XenDesktop 5.5 Resource Calculator

Of course after a weekend of creating my own spreadsheet to calculate my storage requirements for XenDesktop Andre Leibovici created a XenDesktop version of his View calculator. This is a great resource for sizing your storage, calculating IOPS, number of datastores, and other details. A sample of the fields is below.

If you are using XenDesktop MCS, this is a must-use calculator. He says a PVS version is coming as well, so if you aren’t a MCS user then check back with his site for an update.

Tips for measuring Windows 7 VDI IO Requirements

When sizing your storage subsystem for a VDI implementation, it’s extremely critical to understand how your VMs will behave and the resulting IO load. Miscalculate and you will suffer poor performance and angry users. Oversize your array and you will waste money. However, measuring your VM performance may not be as straight forward as you think.

A few months ago I posted a script here that let you dump basic IO performance stats for a VM on vSphere 4 and 5. But as you will see, the applications you load into your image and when you measure the performance has a significant impact on the collected metrics.

For my first round of tests I wanted to focus on the boot performance of Windows 7 64-bit. Booting can be one of the most taxing events (aside from full virus scans) on your VDI storage subsystem. Even if you stagger your VM boots over a few hours as a normal practice, what if you have a power outage or significant hardware failure and you need to rapidly power on hundreds of VMs? Will your storage array melt under the load? Will Windows boot so slowly that it will blue screen (hint: Windows 7 VMs should boot under 5 minutes to avoid problems.) SLAs play an important role here and you need to be mindful of them and verify they can be met.

The test environment is pretty basic and includes vSphere ESXi 5.0, XenDesktop 5.5, and Windows 7 64-bit. The IO measurements were performed over a five minute period after powering on the VM, and metrics were collected via my script. The measurements are only for boot IOs, as no user logged into the VM during the collection process. Tests were performed four times for each scenario and the results averaged. Five scenarios were tested:

  • Base Image: Windows 7 64-bit, Office 2010, VMware tools, joined to a domain
  • VDA Only: XenDesktop 5.5 Virtual Desktop Agent
  • VDA/Symantec: Citrix VDA and Symantec End Point Protection 12.1
  • Optimized: Quest vWorkspace Desktop Optimizer applied with all settings enabled except 15, 26, 27, 30; most VMware Windows 7 optimizations applied.
  • XenDesktop VM: VM created with XenDesktop 5.5 MCS from the optimized template

Drum roll for the results please!

As you can see in the table above, the base Win7 image required an average of nearly 15,000 IOs to boot. 15% of those IOs were writes, while the remainder were reads. Simply installing the XenDesktop VDA decreased the number of write IOs, but increased overall IOs by 17% over the base image. Next up is installing Symantec 12.1, and wow look at those numbers jump! 212% increase in IOs over the base image. Using the Quest and VMware recommended optimizations IOs dropped a bit, but nothing substantial.

What I found to be very interesting is what happened to the IOs when the optimized VM template was cloned by XenDesktop MCS and booted as part of a desktop pool. Zero changes were made to the VM, so the only difference is how the VM behaves when under the control of the Citrix Desktop Studio. Approximately 8000 more IOs are required during the boot process, and a lot more writes are taking place. I would not have guess that large of a delta, so this is an interesting find. The read/write ratio also drops to approximately 80/20.

So what does all of this mean? First, every environment is very unique and you should not use my results, or anyone elses, to estimate the IO load for your environment. Second, take your metrics from a provisioned VDI VM (VMware View, XenDesktop, etc.) and don’t just take measurements from your VM template. Third, booting a VM is very IO intensive and if you only size your storage for steady-state IOPS, then boot storms will cause you major headaches.

Depending on the script/method you use to gather the VM IOPS stats, VMware may not always return the read/write stats in the same fashion resulting in the same order, so you may see inverted data. From my observation this happens on a per-VM basis, even through reboots and power on/off cycles. So if your data looks odd, question it, don’t assume everything is legit.

Automate VMware VMX Security Lockdowns

When building vSphere VM templates best practices would recommend that a number of security lockdowns be incorporated into the template. There are a variety of sources for recommended lockdowns, such as the VMware vSphere 4.1 Hardening Guide. But what if you already have VMs in production that you need to lock down, or want a simple way to configure your VM template settings?

Using some PowerCLI examples I modified them and the result is the script below. The script is called with a single argument, which can be the name of a VM or a wildcard so you can do mass changes. As always, TEST, TEST, TEST! Before you lock down all the settings below, make sure you understand what they do and determine if you really want to disable the feature.

This script can be very handy for XenDesktop 5.0 deployments, as their MCS engine does not properly copy custom VMX settings from the template, so you are left with unsecured VMs. Use the wildcard feature to hit all of the VMs. Also note that many of the settings require the VM to be power cycled, not just rebooted, to read the new values.

Before you run the script you will of course need to use the connect-viserver command to establish a secure connection to vCenter or an ESX(i) host. After the connection is established you can then run the script and monitor the progress in the vCenter recent tasks pane.

# Configure client VM VMX security settings.
# Version 1.0, August 14, 2011
# Argument can be a single VM or a wildcard

$ExtraOptions = @{
 “isolation.device.connectable.disable”=”true”;
 “isolation.device.edit.disable”=”true”;
 “isolation.tools.copy.disable”=”true”;
 “isolation.tools.paste.disable”=”true”;
 “isolation.tools.setGUIOptions.disable”=”true”;
 “Isolation.tools.Setinfo.disable”=”true”;
 “Isolation.tools.connectable.disable”=”true”;
 “isolation.tools.diskShrink.disable”=”true”
 “isolation.tools.diskWiper.disable”=”true”;
 “isolation.tools.hgfs.disable”=”true”;
 “isolation.tools.commandDone.disable”=”true”;
 “isolation.tools.getCreds.disable”=”true”;
 “isolation.tools.guestCopyPasteVersionSet.disable”=”true”;
 “isolation.tools.guestDnDVersionSet.disable”=”true”;
 “isolation.tools.guestlibGuestInfo.disable”=”true”;
 “isolation.tools.guestlibGetInfoDisable.disable”=”true”;
 “isolation.tools.haltReboot.disable”=”true”;
 “isolation.tools.haltRebootStatus.disable”=”true”;
 “isolation.tools.hgfsServerSet.disable”=”true”;
 “isolation.tools.imgCust.disable”=”true”;
 “isolation.tools.memSchedFakeSampleStats.disable”=”true”;
 “isolation.tools.runProgramDone.disable”=”true”;
 “isolation.tools.StateLoggerControl.disable”=”true”;
 “isolation.tools.unifiedLoop.disable”=”true”;
 “isolation.tools.upgraderParameters.disable”=”true”;
 “isolation.tools.vixMessages.disable”=”true”;
 “isolation.tools.vmxCopyPasteVersionGet.disable”=”true”;
 “isolation.tools.vmxDnDVersionGet.disable”=”true”;
 “isolation.tools.setOption.disable”=”true”;
 “isolation.tools.log.disable”=”true”;
 “log.rotateSize”=”100000”;
 “log.keepOld”=”10”;
 “Tools.setinfo.sizelimit”=”1048576”;
 “tools.synchronize.restore”=”false”;
 “time.synchronize.resume.disk”=”false”;
 “time.synchronize.continue”=”false”;
 “time.synchronize.shrink”=”false”;
 “time.synchronize.tools.startup”=”false”;
 “vmci0.unrestricted”=”false”;
 “guest.command.enable”=”false”;
 “tools.guestlib.enableHostInfo”=”false”;
 “isolation.tools.dnd.disable”=”true”;
 “RemoteDisplay.maxConnections”=”1”;
 “Guest.command.enabled”=”false”;
 “devices.hotplug”=”false”;
 “vmxnet.noOprom”=”true”
}
$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
Foreach ($Option in $ExtraOptions.GetEnumerator()) {
    $OptionValue = New-Object VMware.Vim.optionvalue
    $OptionValue.Key = $Option.Key
    $OptionValue.Value = $Option.Value
    $vmConfigSpec.extraconfig += $OptionValue
}

# Get all VMs per the argument

$VMs = get-VM $args[0] | get-view

foreach($vm in $vms){
    $vm.ReconfigVM($vmConfigSpec)
}

Load Balancing XenDesktop 5 with NetScaler 9.3

As I mentioned in yesterday’s blog post, any enterprise VDI deployment needs redundant broker services for high availability. Other enterprise applications such as Exchange, Lync, and SharePoint can all benefit from a load balancer, be it virtual or physical. Building on yesterday’s post about configuring SSL on the NetScaler, it’s now time to configure load balancing for the XenDesktop DDCs and Web Interfaces.

I’m making a few assumptions here. First, you already have XenDesktop 5 installed and functioning in your environment, hopefully with redundant WI and DDC servers. Second, you’ve configured the WI servers for SSL. Third, you’ve already deployed the NetScaler and using at least version 9.3. Fourth, you’ve installed a SSL certificates on the NetScaler for the DNS names you’ve assigned to your WI and DDC virtual IPs.

Environment:
XenDesktop combo DDC/WI: 192.168.0.200 and 192.168.0.201
Web Interface virtual IP: 192.168.0.100
DDC virtual IP: 192.168.0.101

1. Download the Citrix AppExpert template for the Citrix Web Interface here.
2. In the NetScaler open the AppExpert folder, right click on Templates and select Manage Templates.
3. Click on the Upload button and locate the XML file you downloaded in step one.
4. After the template imports click on Load Balancing in the NetScaler GUI. You should now see two new wizards under Getting Started.

5. Start the XenDesktop wizard and enter the appropriate information in the WI server wizard screen. The IP addresses are pretty self explanatory. I would recommend configuring a health monitoring service account. This will allow the NetScaler to actively attempt to authenticate to ensure the WI is actually functional. One critical change you need to make to the form is the site path. You MUST remove site/default.aspx, as shown below.

6. For the DDC configuration page it’s pretty clear what you need to input. Remember you will need to use a unique IP address for the DDC virtual server. And again, I’d configure a service account for health monitoring. You could use the same account or a different one.

7. Close the wizard and if everything is correct, it will create the virtual servers, service groups, monitors, and servers for you. It is very likely though that the WI monitor will show a down status, while the DDC monitor may show as UP. If that happens, it’s probably an SSL issue which we can easily resolve.

8. Open the WI virtual server  and see if you see the error below, certkey not bound,  you are in luck as this is an easy fix.

9. Click on the SSL Settings tab and select the appropriate WI SSL certificate that you either created from my blog yesterday if you are just testing, or your real one if this is a production deployment. Click on Add to move it to the configured column.

10. Close the window and now your WI State should be UP and 100% health.

11. Repeat the SSL assignment exercise for the DDC monitor using another certificate which matches the DDC DNS name you chose earlier.

Next up, open your browser and go to the FQDN for both virtual servers and verify that the XenDesktop login screen appears with no SSL warnings. If so, you’ve now created two VIPs for load balancing critical XenDesktop services and enabled health monitoring. High availability baby!

Creating a SSL certificate for Citrix Netscaler

A high availability VDI deployment, such as XenDesktop 5, demands that you use multiple servers to provide broker redundancy. As such, a load balancer such as the Citrix Netscaler comes in mighty handy. The NetScaler can also act as an ICA proxy between a trusted and untrusted network, such as the internet and your corporate network. Now that I’ve gotten XenDesktop 5 running in my lab, I wanted to see what it takes to configure the NetScaler Access Gateway feature to allow external inbound connections and serve up a nice VDI desktop.

As the configuration is somewhat complex, let’s start with the easy part, creating your own SSL certificate and importing it into the NetScaler. Now in the real world you’d need to use a trusted CA like Verisign, or your clients won’t trust the Access Gateway and the Citrix receiver will not launch. However, if you are in a lab or home environment you can use your own CA just to get the flavor how it works.

In my lab I’m using the latest NetScaler VPX release, which is v9.3 build 48.6.nc. First we need to use OpenSSL to create a private key, then a certificate request, convert the private key, then submit to my Microsoft CA, and finally import into the NetScaler. Figuring out this process was a bit easier than VMware makes it for importing certs into an ESXi host, so you have that going for you.

1. Login to the NetScaler and click on the SSL folder in the left pane.
2. Generate a private RSA key by clicking on Create RSA Key. Use a filename that is easily associated with the FQDN of the certificate and I would use a .key extension to denote it’s the private key. 2048 bits is the maximum keysize, so I’d go for that. Change the format to DER. Click on Create then Close.

3. On the NetScaler SSL page click on Create CSR. Type in a file name for the certificate request (I’d suggest a .req extension), then browse to the private key file you just created. In the Common Name field enter the FQDN you want your certificate to be bound to. Fill in the other information as needed. Click on Create then Close.

4. Back on the SSL page click on Manage Certificates then locate the REQ file, highlight it, then click on View. Copy the contents to the clipboard. Close the window.
5. Assuming you are using a Windows Server 2008 R2 CA, perform these steps:

  • Go to the certificate home page and click on Request a certificate.
  • Select Advanced certificate request.
  • Select Submit a certificate request by using a base-64-encoded….
  • Paste the certificate into the window and change the template to web server.
  • Download a DER encoded certificate (not the certificate chain) using a logical name like xd-contoso-net.cert.

6. Back on the NetScaler and open the SSL folder then click on Certificates.
7. Right click in the SSL window and select Install.
8. I would suggest the FQDN for the pair name, browse locally to the certificate file name, then browse on the appliance for the private key, and change the certificate format to DER.

9. Click on Install and hope that the certificates import successfully. Once the certificate imports, you should delete the certificate from wherever you downloaded it to on your workstation.

 
And there you have it! You’ve created your own private key, certificate request, generated a SSL certificate, then imported it to the NetScaler. The private key and public key file names are important, since the files are stored on the NetScaler and each certificate must have a unique name. You can repeat this process for any number of certificates, as needed. 

VDI Storage – Right size it or anger your users!

According to Citrix, the number one mistake people make when deploying VDI is not sizing their storage properly for performance. However, calculating a target IOPS for your environment is far from easy and you really need to understand VDI, its unique workloads, and your storage subsystem. There are many blogs about this topic, so I’m not going to rehash them. But I will provide a good list of what I think is required reading if you are going to deploy VDI in production on any scale, above a few dozen VMs.

Great links to check out:

VDI Calculator
Finding a Better Way to Estimate IOPS for VDI
Windows 7 IOPS Deep Dive 
Virtual Desktop Resource Allocation
Improper Storage Design for Virtual Desktops is a Killer
Deciding on Local or Shared Storage for your Desktop Virtualization Solution
Estimate IOPS for Virtual Desktops
Data Storage for VDI – Part 2 – Disk Latencies
Data Storage for VDI – Part 3 – Read and Write Caching
Data Storage for VDI – Part 4 – Impact of RAID on Performance
Video: Storage Infrastructure Design Guidelines
HP XenDesktop 1,000 User Reference Architecture
RAID options with Desktop Virtualization
Local or Shared Storage – that is the question
Does Cache Trump IOPS
VDI Storage Calculator Spreadsheet
VDI & Storage – Deep Impact
VDI IOPS Calculator

Interesting VDI storage products:
xiotech Hybrid ISE
WhipTail
Atlantis Computing
IntelliCache and the IOPS Problem

Below is a sample calculation for 2,000 users using a moderate IOPS profile. According to these calculations for RAID-1 your disk array would need 273 15K disks!

Enjoy!

Use a Service Account with XenDesktop 5

During the testing process of Citrix XenDesktop 5, we were using the built-in SQL express database so we can do a quick lab setup. Of course a production deployment would use a full blown SQL 2008 R2 enterprise edition instance with database mirroring. During the XenDesktop 5 SP1 upgrade process, we ran into an interesting error that was related to how I did the original XD5 installation.

During the original installation process I logged into the freshly provisioned VM that was soon to become our all-in-one XD5 server. So of course I logged in with my admin credentials and performed the installation, using the free built-in SQL Express option. All was fine and dandy, until another administrator tried to install SP1.

During the SP1 installation process the other administrator ran into a problem, that was tracked down to SQL. As it turns out, my account (original installer) was automatically configured as the CitrixXenDesktopDB DBO.

So naturally when someone else came along and ran SP1, which needs to update the XD database, he ran into problems. While there are several solutions to the problem, I will propose a solution that solves two problems at once.

As we also discovered, the original installer’s account credentials are also used to connect to vCenter. As it turns out I’m a vCenter admin, so that process was transparent. However, when my password expired, XenDesktop broke because it couldn’t contact vCenter. Bad!

One elegant solution is prior to installing XD5 is to create a service account, and configure it for a non-expiring password. Next, give that service account local admin rights on your XenDesktop server. If you are using MCS with vCenter, give that account the required vCenter rights. Finally, login with that service account on the XenDesktop server and proceed with your installation process. This way both SQL Express and vCenter credentials are using those of the service account, not your personal admin account.

I really wish the XD5 installer prompted for service account credentials, so both of these problems could be automatically avoided. Only after several weeks of testing and a new service pack release did we run into these issues.

XenDesktop 5.0 Service Pack 1 Released

A couple of weeks ago Citrix released  XenDesktop 5.0 SP1, which fixes a number of bugs. Last week I attended Citrix Synergy 2011, and talked to an employee about SP1. He said that the service pack fixes many more bugs than the short list included in the release notes. In fact, his comment was that more than 600 bugs were fixed. So given my experience with the GA release, which you can read about here, I thought I’d give SP1 a whirl to see if the bugs I encountered were fixed.

  1. Using a PVSCSI controller in the guest VM on ESX 4.1. Fixed!
  2. Using a dvSwitch (such as the Nexus 1000v) on ESX 4.1. Fixed!
  3. Guest VM does not unmount an ISO image when cloned on ESX 4.1. Fixed!
  4. ESX 4.1 VMX/nvram settings not copied from master VM template to clones. Still Broken.

It’s great to see Citrix fixing many of the VMware related bugs. SP1 officially supports ESX 4.1 U1, so that’s also good news. But it is disappointing that the VMX and nvram settings are not copied to the cloned VMs. This is a big security issue, since our VMX files contain dozens of required lockdowns. The nvram settings also control floppy drive settings. Since that gets reset all of our VDI VMs have a floppy drive shown. Yes we can use a GPO to hide the floppy, but this really should be handled by Citrix.

In case you missed, it, Citrix also has publicly released a ‘technology preview’ of the next version of XenDesktop, likely version 5.5. Details on this release can be viewed here. Cool new features include Windows Aero redirection, enhanced Flash redirection, improved WAN scanner support, and HDX 3D Pro support. You can download the full package from here.

XenDesktop USB Filtering the easy way!

In XenDesktop 5.0 you can configure HDX policies to block or allow certain types of USB devices. For example, you could block flash drives but allow USB printers or webcams. Unfortunately, Citrix doesn’t give you an easy to to discover class IDs, vendor IDs, or other identifiers that can be used in their policies. Citrix has a good article here on USB filtering in XenDesktop 5.0.

Instead of digging through the registry to discover this critical USB data, I found a great tool that makes it a snap. Nirsoft has a free USB viewer you can download here.

To create the appropriate rules I did the following process:

1. In Citrix Desktop Studio open the Users HDX policy and navigate to USB DevicesClient USB device redirection. Edit the policy and change the value to Allowed.

2. Using the Citrix receiver connect to a virtual desktop, then from the menu bar click on the USB button.

3. In my case I have a flash drive connected to my physical computer, so I selected that from the drop down menu. I then heard the Windows USB disconnect/connect sounds and saw my flash drive ready to use in the VM.

4. Download the USB viewer tool and run it inside the VM. In the list of USB devices, locate your connected device and double click on it. Here’s what comes up for my USB stick:

5. Take note of the USB class ID and USB subClass IDs, as you will need these for the HDX rules.

6. Back in Citrix Desktop Studio open the Users HDX policy and navigate to USB DevicesClient USB device redirection rules. Edit the policy and create a new rule, for example:

7. Accept the rule, then log out of your virtual desktop then log back in. If you try and connect your thumb drive now, nothing happens. Unfortunately XD5 doesn’t provide the user any feedback why you can’t connect the device. It would be most useful if a warning popped up saying that device was administratively prohibited, so the user didn’t call the help desk wondering why it wasn’t working.

You can use the same basic procedure to build up allow or deny device lists as required. Some devices can be tricky, such as multi-function USB printers/scanners/fax machines. So a single composite device might need a few allow entries to make it properly function. But using USB device view, you can pretty easily figure out what you need to do.

XenDesktop 5 Machine Creation failure: VMware PVSCSI driver fix!

During my testing of Citrix XenDesktop 5 I ran across yet another bug, which set me back in my testing. Apparently if the VM template that you want machine creation services (MCS) to use has been configured with the VMware pvscsi controller, creating the VMs will fail when you generate a catalog.

The error that XenDesktop Desktop Studio will give you is:

The specified master VM snapshot could not be found. No machines have been created.

If you look in the Windows application log you see:

Provisioning scheme creation workflow operation failed : System.InvalidOperationException: VM not Found —> Citrix.HypervisorCommunicationsLibrary.InvalidVmConfigurationException: No disk controller found

As mentioned in my previous blogs, I always use the VMware pvscsi controller since it’s more efficient than the emulated legacy SCSI controllers. But, it seems that Citrix didn’t test this use case, since it fails miserably. The fix is to not use the VMware pvscsi controller, and use something like the LSI Logic SAS controller. But what if you already have a VM template built with the pvscsi controller, like me, and you don’t want to rebuild it because of a Citrix bug?

There’s an easy fix! While your VM template is running and using the pvscsi controller, open an elevated powershell and type the following:

Set-ItemProperty “HKLM:SYSTEMCurrentControlSetservicesLSI_SAS” -name “Start” -Value 0 -type “DWORD”

I then rebooted the VM (still with the pvscsi controller), shutdown the VM, then in vCenter changed the SCSI controller type to LSI Logic SAS. Next time the VM boots the LSI Logic SAS driver will be active at boot time and your VM won’t blue screen.

I hope Citrix can fix this bug in their next update for XenDesktop 5. It’s a bit disturbing that this scenario, just like the FIPS bug, wasn’t tested prior to GA.

Update: This issue is now fixed in XD5 SP1. Check out my post here.