Goodbye vSphere C# Client

So for as long as I’ve been using VMware, the Windows C# client has been a staple of my workflow. Even when VMware started transitioning to the icky Flash based interface, I know many MANY people still used the C# client. Between dodgy performance, reliance on Flash (and all it’s security problems), re-jiggered UI, difficulty in finding objects, no VUM interface, etc. The Flash based interface went over like a lead balloon in the vSphere community. To VMware’s credit they did make improvements over the years, but it was still Flash based and slow.

Today VMware is announcing that in their upcoming release of vSphere, the Windows C# client will no longer be offered. Yes, after years of warning us about the client going it, it is now dead. Buried, and one for the history books.

Now you ask, what will it be replaced with? Yes, they will now offer a full HTML5 client.  A while back VMware released a HTML5 ‘fling’ (which is unsupported for production usage) for embedded host management. Frankly I’ve been too busy to try it, plus customers can’t use it in production. Although it does appear to have made it into vSphere 6.0 U2.

Other enterprise products have had HTML5 interfaces for years (e.g. Nutanix), and I’m so glad I can stop installing Flash on servers. So I do welcome this change in VMware management. But the proof will be in the pudding, on how well they implement it. Will it be performant? Will it be intuitive? Can we manage VUM, SRM, and third party products? How about third party plug-ins that still rely on Flash? Only time will tell how these are addressed. I was on the vExpert call earlier this week that VMware hosted, and the community was very concerned about the usability and knowing which plug-ins will or won’t work.

I welcome the change, but only time will tell how well VMware can execute. As a side note, Nutanix never has had a vCenter plug-in. We have a comprehensive HTML5 interface called PRISM that manages our HCI solution. So unlike other vendors, you won’t have to play a waiting game with vSphere .Next and wait for any updated Nutanix plug-in. Once our QA tests vSphere v.Next and we whitelist the ISO, you will be good to go.

Critical VMware Security Patches Released

In case you missed it, VMware has released a number of product updates to address critical vulnerability in JRE. JRE is used in many products, so a wide range of products are affected. You can read the full bulletin here. The bulletin details which product version you need to be running to be patched. In many cases patches are “pending” such as vCenter 6.0, and SRM. So keep your eyes out for another VMware announcement when the patches become available. The full patch matrix seems to include products that didn’t made the affected products list. So carefully review the full security bulletin, as a majority (if not nearly all) VMware products are affected.

(Some) Affected Products:

Horizon View 6.x or 5.x
Horizon Workspace Portal Server  2.1 or 2.0
vCenter Operations Manager 5.8.x or 5.7.x
vCloud Automation Center 6.0.1
vSphere Replication prior to or
vRealize Automation 6.2.x or 6.1.x
vRealize Code Stream 1.1 or 1.0
vRealize Hyperic 5.8.x, 5.7.x or 5.0.x
vSphere AppHA Prior to 1.1.x
vRealize Business Standard prior to 1.1.x or 1.0.x
NSX for Multi-Hypervisor  prior to 4.2.4
vRealize Configuration Manager 5.7.x or 5.6.x
vRealize Infrastructure 5.8, 5.7

Sample VCDX-DCV Architecture Outline

This post covers my approach to writing my VCDX-DCV Architecture Guide. I’ve been debating in my mind for a while whether I should write this post or not. I hesitated for a few reasons. First, I’m just a regular guy that happened to jump through the VCDX hoops and have no “insider” information on how they score. Those that do know the scoring rubric can’t disclose it anyway. Second, there are 1000 different ways to write your VCDX-DCV architecture document. Third, there’s no “magic template” or “sure fire” outline that ensures your design gets accepted. Do not view this post as shortcut or cheat sheet.

What matters is your content, how it aligns to the VCDX blueprint, and that you convey expert level knowledge to the reader. It’s NOT about speeds and feeds, but rather the full traceability of customer requirements, constraints,  assumptions and risks throughout your design. Who cares if you’ve thrown every VMware product and feature at a solution if you haven’t met the business requirements? #Fail

So why did I publish this article? I know when I started the VCDX process it was a bit daunting to read the DCV blueprint and try to come up with an architecture guide that hit all the areas in a logical manner. I’ve heard from other candidates they experienced the same “VCDX writer’s block.” In fact several of us have scrapped our first attempts, and started over. Bottom line is you need to do what feels right to YOU, and what works for YOUR design while covering all the blueprint areas. You may not like my methodology or outline, which is perfectly fine and a valid way to feel.

I’ve also heard comments from VMware customers (like myself when I went through the process) that think since they aren’t a partner and don’t have access to the VMware SET templates that they are at a disadvantage. That’s not true,  IMHO. Yes the VMware SET docs are structured and may help you, but they aren’t directly aligned to the VCDX blueprint and need augmentation.

With all those caveats, I wanted to share my DCV architecture guide outline. Maybe it will help someone with writer’s block, or enable you to see some the areas that a VCDX design could cover. Your design may need additional areas, or less coverage. This is certainly not all inclusive, and it’s guaranteed your outline will be different. It is your responsibility to ensure your documents cover all blueprint areas, makes sense for your design, and something you feel comfortable with. Own your documentation.

Before I go any further, let me state that how I chose to incorporate the specific VCDX bootcamp book recommendations is somewhat unique to my style. Of the submissions I’ve seen none did it exactly this way, which proves that there is no “magic” template or style for VCDX submissions. I just felt it gave a better overall flow to the document.

You will see some common sub-sections in all design areas (e.g. cluster, storage, compute, etc.). For example, in most areas I had specific conceptual, logical and physical sections. This helped me show the traceability of customer input through the entire design process. Each major section also concludes with a Design Justification which is a summary of how I met the customer requirements and sites all of the applicable requirements, assumptions, constraints, and risks.

At the end of the Design Justification section I had two tables to help distill down the critical information. First, I had a summary table, shown below. All of the design quality items (e.g. C02) were referenced elsewhere in that section as applicable. Possibly overkill, but I liked the compact summary.


The second table was that of the applicable design decisions, each with the decision, impact, decision risks (after all, nearly every decision has a risk), and risk mitigation. A sample design decision is below.


WordPress was not cooperating with me for a clean outline format, so I’ve inserted a series of screen captures to maintain formatting.

Sample VCDX-DCV Architecture Outline


2014-10-06_16-50-08  2014-10-06_16-52-34



Downgrade your ESXi Host

On rare occasions you may need to downgrade your ESXi host. In my case I was working on a VMware certification test and my host was using a newer build than what was called for. Or, maybe you run into a situation like the NFS bug earlier this year and needed to downgrade back to a stable ESXi version. Either way, it’s a fairly simple process do downgrade your ESXi host, all without losing your settings.

1. On a computer with PowerCLI installed run the following command. From the long list of profiles, pick the profile which you want to downgrade to. Some profiles have build numbers, while others have dates. So it may take a little digging to downgrade to exactly the build level you want. In my case I wanted ESXi 5.1 GA.

Get-EsxImageProfile | Sort-Object “ModifiedTime” -Descending | format-table -property Name,CreationTime



2. Enable SSH on your ESXi host and then enter the following command, but replace the profile name with the one you want to downgrade to. After the profile downloads and installs, reboot the ESXi host.

esxcli software profile update -d -p ESXi-5.1.0-799733-standard –allow-downgrades

And that’s it! I don’t know the official support stance on this, so just don’t go doing it on random production servers. But it saved my bacon today. If you are a Nutanix customer, this command will also work and preserves our CVM and custom ESXi host configuration information. Also remember that you might be able to use “shift r” during ESXi boot to restore your ESXi host to a prior installed version. This could negate the need to re-download the profile that you want. In my case the host was freshly imaged with a newer build so the “shift r” at boot was not of assistance.

VMware has a KB article on a similar procedure, and a support statement that you can check out here.

Automate Sysprep on vSphere w/o Custom Specs

I’m a huge fan of using vCenter customization specifications to automate the sysprep process for deploying new Microsoft VMs. The sysprep process ensures a unique Windows SID, sets the VM’s hostname, and can even join a VM to the domain, among other things. However, the customization specifications can only be triggered when you clone a VM. While that may be good for a vast majority of use cases, I recently ran across a scenario where that was not possible since my VMs already existed.

For a VDI project I am looking at a software storage appliance to offload a lot of the IOPS from our back end storage system, to increase performance and reduce costs. One feature the our particular solution has is called “fast clone”, which allows the storage appliance to create a VM clone in just a few seconds, instead of several minutes using the vCenter clone method. Internally it adjusts some pointers to the VMDK, and de-dupes, so it doesn’t have to copy every block when you create a new VM. In fact, very few blocks are copied during the cloning process.

However the “fast clone” process literally cloned the master VM and did not have any method to trigger vCenter customization specs. As a result all the Windows hostnames were the same as were the SIDs. I certainly did not want to run sysprep manually on hundreds of VMs. The vendor workaround was far too complex and cumbersome to consider. So I developed the script below which automates the major tasks which the vCenter customization specifications perform and easier (IMHO) than what the vendor suggested.

Script Features

  • Copies an existing sysprep unattend XML file to the VM via the VMware tools VIX interface
  • Each unattended XML file is automatically customized with the VM’s name as it appears in vCenter, so sysprep will change the Windows hostname appropriately
  • Deletes the residual unattended XML files which may contain sensitive passwords or product keys
  • Auto-joins the VM to the domain assuming an appropriately configured unattend XML file and DHCP is available
  • Accepts a command line argument for easy testing against one VM, but it will also read a CSV file for mass processing

It’s up to you to supply an appropriately configured Windows sysprep unattended XML file for the operating system in question. If you include domain join parameters then it will join the VM to the domain as well, all without prompting for a username or password. To delete the residual XML files, the script will upload a setupcomplete.cmd file to c:\windows\setup\scripts. It will not over-write any existing file, so make sure it doesn’t exist. Windows knows to automatically run that script after the sysprep process.

In order to customize the unattended XML file with the VM’s hostname, the script does a simple replace on a string called “CHANGEHOSTNAME”. When you create your XML file be sure to use this name for the machine name, so the search and replace will work properly. Otherwise all the VMs will have the same hostname!

5-18-2013 6-35-18 PM

Using the Script

When you want to run the script against several machines, use the csv option. The csv file must have the vCenter VM name, one VM per line, without any header or empty lines at the end. There’s limited error checking, so I would urge you to take a snapshot of your target VM so you can revert back until you work out the kinks with your unattend file. In the vCenter console you will see some authentication errors when sysprep kicks off and invoke-script can no longer connect to the VM , but those are harmless messages.

In the example below I executed the script on the vCenter server using a PowerCLI console. I had configured the CSV input file with two hostnames. First I entered my password (for my current user account), then the administrator credentials for the guest VM. The script assumes all VMs have the same credentials as you will only be prompted once.

vsphere sysprep windows

If you watch the vCenter console you will see a bunch of entries. As I mentioned earlier, once sysprep kicks off vCenter is unable to connect to the guest so some authentication errors appear.

vsphere sysprep windows

After minute or so the VM rebooted and the sysprep process kicked off. A few minutes later my VM was joined to the domain with its new name and ready for use. Depending on the complexity of your unattended sysprep file you do could a lot of customization within the guest, install software, etc. the sky is really the limit. This script just gives you an easy way to run sysprep against dozens or hundreds of existing VMs if you can’t use vCenter customization specifications.

5-18-2013 6-52-35 PM

# This script will copy a sysprep unattend XML file to the guest VM and execute it,
# using the VM's vCenter name. Input can be a single arguement on the command line,
# or a csv file. The CSV must have one VM name per line and no blank lines or header.
# The setupcomplete.cmd deletes the two copies of the unattend XML file, which may
# contain sensitive passwords or product keys.
# Derek Seaman

# Your vcenter server name
$vCenter = ""

# Your master sysprep unattended file. It will not be modified.
$MasterSysprep = "d:\sysprep-master.xml"

# Optional CSV input file. Only called if no VM argument is provided.
# One vCenter VM name per line with no header
$CSV_File = "D:\vms.csv"

# "Hostname" in the master unattended sysprep file that will be replaced for each VM

# Resulting sysprep file with the custom hostname, overwritten for each VM. Do not change.
$CustomSysprep = "D:\sysprep.xml"

# Don't change anything below here

#Validates VMware PowerCLI snap-ins are loaded

$xPsCheck = Get-PSSnapin | Select Name | Where {$_.Name -Like "*VMware*"}
If ($xPsCheck -eq $Null) {Add-PsSnapin VMware.VimAutomation.Core}
if ($args[0] -eq $null ) {$list = import-csv $CSV_File -header name} else { $list = $args[0] }

# Function to mask password input
function Read-HostMasked([string]$prompt="Password") {
$password = Read-Host -AsSecureString $prompt;
$BSTR = [System.Runtime.InteropServices.marshal]::SecureStringToBSTR($password);
$password = [System.Runtime.InteropServices.marshal]::PtrToStringAuto($BSTR);
return $password;

} # function Read-HostMasked([string]$prompt="Password")

# Connects to vCenter
$currentUser = ([System.Security.Principal.WindowsIdentity]::GetCurrent()).Name
$currentUsePassword = Read-HostMasked "Enter the password for the current user"
Connect-VIServer -Server $vCenter -User $currentUser -Password $currentUsePassword | out-null

# Guest OS administrator credential input
$guestuser = read-host "Enter guest administrator username"
$guestpassword = read-HostMasked "Enter guest administrator password"

Foreach ($vm in $list) {

# Cleans up prior local sysprep output file and replaces hostname in sysprep.xml
remove-item $CustomSysprep -ErrorAction SilentlyContinue

$content = Get-Content $MasterSysprep
$content | foreach { $_.Replace($ReplaceHost, $ } | Set-Content $CustomSysprep
write-host $ Custom sysprep file created

# Creates setupcomplete.cmd file to delete sysprep XML files post-sysprep. File must not already exist.
$Script1 = "echo `"del /F /Q c:\windows\panther\unattend.xml c:\windows\system32\sysprep\sysprep.xml`" | out-file -encoding ASCII c:\windows\setup\scripts\setupcomplete.cmd"
invoke-vmscript -scripttext $script1 -VM $ -guestuser $guestuser -GuestPassword $GuestPassword | out-null
write-host $ setupcomplete.cmd uploaded

# Copies sysprep.xml to guest and executes asynchronously
$script2 = "c:\windows\system32\sysprep\sysprep.exe /generalize /oobe /unattend:c:\windows\system32\sysprep\sysprep.xml /reboot"
copy-vmguestfile -source $CustomSysprep -destination c:\windows\system32\sysprep -VM $ -localtoguest -guestuser $guestuser -guestpassword $guestpassword
invoke-vmscript -scripttext $script2 -VM $ -guestuser $guestuser -GuestPassword $GuestPassword -scripttype bat -runasync | out-null
write-host $ Sysprep executed

VMware releases vSphere 5.0 and 5.1 security patches

Yesterday VMware released security and bugfix patches for both ESXi 5.0 and ESXi 5.1. It has also updated the vCenter Server Appliance to address security issues as well. For the full security bulletin, check out this link. The new build numbers are shown below:

ESXi 5.0 Build 914586 (Update 2)
ESXi 5.1 Build 914609

As always, you can download the patches from their patch manager site here.

Are VMFS and Datastores going the way of the dodo bird?

At VMworld 2012 San Francisco there was some information publically shared about “vVols” (VMware Virtual Volumes) which is an entirely new and radical concept for VM storage. At VMworld 2012 Barcelona there seems to be a lot more talk about it, as major vendors are now blogging about their future support for vVols and their benefits.

vVols will entirely replace the datastore concept (for both NFS and block storage) and VMFS with what I would call VM-aware storage. The VM now becomes an object that the storage array understands and can apply policies to such as snapshots, replication, and SLAs against. You can manage capacity through capacity pools. Capacity pools can span storage chassis or even datacenters.

No more deciding how big or how many datastores you need to create. No more storage vMotioning a VM to another datastore because you are running low on space. No more wondering how many VMs you can place on a VMFS datastore before you run into contention issues. No more datastore clusters. No more VMFS!

This has the potential to really change how you view and consume storage in a VMware environment. It also will also impact how you do backups, disaster recovery, and manage your storage on a day-to-day basis. In fact, storage should take less management. This also combines the benefits of NFS and block storage into a single way to communicate to the array.

For additional details from various vendors and VMware, check out these links:

HP vVol Demo with 3PAR
VMware Blog/Video on vVols
EMC VPLEX and VMAX vVol Demo
IBM XIV vVol Discussion
Duncan Epping on vVols
Erik Zandoer on vVols
Julian Wood on vVols
Stephen Foskett on vVols
VMworld 2011 EMC/vVol Preview Demo
LogicalBlock on vVols

When will VMware release this technology? Who knows, but my bet is on the next major release of vSphere, probably due out the end of 2013, if they stick to their yearly releases. vSphere 6.0?

Get your free 2 socket Veeam Backup 6.0 License key

Veeam is once again running a Christmas special where you can get a free NFR (not for resale) Veeam Backup 6.0 license key for 2 sockets which is good for 1 year. You can select VMware, Hyper-V, or both. All you need to do is fill out this form and wait for the email with the license key.

For home labs or just trying out Veeam without their more limited timed trial versions, this is a great opportunity. Even if you don’t think you will use the key, I’d grab one anyway since you never know what may come up over the next year where it could come in handy.

Veeam is targeting the offer at certified VMware professionals such as VCP, but they don’t require any identifying information.

Get Your Nerd On T-shirt now!

A friend of mine, Chris McCain, has some cool t-shirts for sale on his site, Get Your Nerd On.

Check ’em out and get one today!

Align your partitions with VMware Converter 5.0 Beta

Update: VMware released the GA version of 5.0 and you can download it here.

A few days ago VMware released a significant update to their standalone converter utility, Converter 5.0 beta (download here). One of the cool new features is the ability to re-align partitions. By default Windows Server 2003 and Windows XP do not have properly aligned partitions. This can cause addition IOs and poor VM performance. Windows Vista and Server 2008 and later are smarter and automatically align partitions on a 1MB boundary.

So I decided to try out the new feature and verify that a conversion process did in fact align the partitions. To perform the test I already had a Server 2003 VM in VMware Workstation 7 that had an improperly aligned partition:

Here you can see a starting offset of 32,256 bytes which is 31.5KB. No good! You ask how did I get that information? Simple…from a command prompt type:

wmic partition get blocksize, startingoffset, name, index

Next I fired up Converter 5.0, ran through the wizard to convert it to an ESXi VM, and saw this nifty screen:

The ‘create optimized partition layout’ appeared when I selected a volume copy option. Whoohoo! I ran through the rest of the wizard, waited 4.5 hours (gotta be a beta bug to take this long), and viola, ended up with a newly converted VM on my ESXi host. Now did the converter actually work? Let’s see:
By George, yes the starting offset is now divisible by 32K. You can now sleep better knowing that your disk subsystem is working as efficiently as it can. This tweak can be really important in a Windows XP VDI environment where there’s a lot of disk IO and any savings can be substantial when multiplied by hundreds or thousands of VMs.
If you want to get really fancy and change the NTFS cluster size during the conversion process, you can click on the Advanced button in the figure above and tweak as shown below: