Archives for February 2011

Lync Server 2010 Install failure on Server 2008 R2 SP1

During my free time I thought I’d install Lync Server 2010 and check out the new features. If you want a great guide for installing Lync Server 2010, check out Jeff Schertz’s blog here. I used Windows Server 2008 R2 SP1 slipstream media, since it was just released to the public a few days ago. Why not use the latest and greatest?

Everything was going fine until I got:

Error: Prerequisite installation failed: Wmf2008R2

After a few minutes of digging I spotted the problem, and it’s directly related to Server 2008 R2 SP1. The issue is that the Windows Media format package name was changed in SP1 so the installer bombs since it can’t find the RTM name. The old version was 6.1.7600.16385 and the new version is 6.1.7601.17514. Houston we have a problem!

To fix the problem you can manually install the component using the following command line:

C:Windowssystem32dism.exe /online /norestart /add-package
age~31bf3856ad364e35~amd64~~6.1.7601.17514.mum /ignorecheck

REBOOT the computer, then you can continue with the Lync Server 2010 installation.

Configure custom Default Profile in VMware VM Templates

Over the past year I’ve been developing Windows Server 2008 R2 and Windows 7 VM templates for my VMware environment. However, the process has not been without its challenges. One of the features I wanted was a customized default user profile so that things like WMP, IE, and other settings were configured to our standards.

However, using the VMware customization specifications you can’t easily change the XML file it feeds sysprep to perform a profile copy operation. So how do you get a customized default profile with VMware vCenter?

Here’s the process that I use which works like a charm:

1. Install Windows into a VM using whatever method you wish (autounattend or manual).
2. Customize the Administrator’s profile however you wish (desktop icons, launch WMP, modify the toolbar, etc.).
3. Install any additional software or other tweaks you want to make to your image.
4. Install User Profile Manager then use the Copy To feature to copy the settings to the default profile, as shown below.

5. De-install User Profile Manager.
6. Shutdown your VM and turn it into a VM template.
7. Use vCenter to provision a new VM from the template, and use the customization specification to modify the VM such as hostname, administrator password, etc.

When vCenter provisions your VM and you log into it for the first time the default profile contains your customizations, and is configured how you wanted it to be. Easy as pie!

Windows Recovery Environment VMware Driver Injection

This post will show you how to configure Windows recovery environment VMware drivers. In a previous blog post here I described how to inject VMware pvscsi and VMNET3 mass storage drivers into your Windows Server 2008 or Windows 7 image. However, that did not cover injecting the same drivers into the Windows Recovery Environment, which is a separate WIM within the install.wim, thus requiring extra work.

Here’s how to inject drivers into the winRE.wim file and repackage the install.wim with the updated recovery environment.

1. Follow the first five steps at my blog here to install WAIK, find the right drivers, and create a scratch directory.

2. Mount the install.wim file from your Windows installation DVD:
dism /Mount-Wim /WimFile:D:\install.wim /Index:1 /MountDir:D:\mount

3. Copy the winRE.WIM to a working folder:
copy D:\mount\windows\system32\recovery\winre.wim D:

4. Create another mount directory, D:Mount2, then run this command:
dism /Mount-Wim /WimFile:D:\winre.wim /Index:1 /MountDir:D:\mount2

5. Inject the pvscsi and VMXNET3 drivers:
dism /image:D:\mount2 /Add-Driver /driver:d:\drivers\pvscsi.inf
dism /image:D:\mount2 /Add-Driver /driver:d:\drivers\vmxnet3ndis6.inf

6. Unmount the winRE image:
dism /unmount-wim /mountdir:d:\mount2 /commit

7. Copy the modified winRE.wim file to:

8. Unmount and commit the changes to the install.wim:
dism /unmount-wim /mountdir:d:\mount /commit

At this point you now have a modified install.wim file that you can copy back into your Windows OS ISO. Depending on your install.wim file, you may have multiple operating systems that need to be modified. To do this you would serially mount all OS indexes, inject the new winRE.wim then unmount the image. For a typical Windows Server 2008 R2 DVD, you could have 8 images that need to be modified to cover all of your bases. So this process can be a bit tedious and would lend itself to scripting.

vCenter 4.1 U1 and FIPS encryption: Verify your IE Settings

During my regression testing of my vCenter 4.1 U1 installation instructions on Windows Server 2008 R2, I came across a problem that made me scratch my head. I was updating the vCenter SSL certificates, per my blog here. However, when I opened IE and tried to connect to the vCenter default home page would not come up. I got Internet Explorer cannot display the webpage.

OK I thought, maybe I goofed up the SSL certificates. I regenerated them, and nope, no good! The Windows Server 2008 R2 template that I’m using is locked down and has many security features enabled, including FIPS compliant encryption.

You can connect to vCenter with the vSphere client, but it appears the web services on port 443 are broken. For example, as I mentioned, the vCenter home page would not come up, the vCenter Service Status screen would not open, and performance graphs were also broken.

After additional research since my original post, the root cause appears to be the combination of two security settings: FIPS compliance, AND restricting what encryption algorithms IE is allowed to use.

The IE settings that cause the problem is the unchecking of TLS 1.0, as shown below.

This in combination with enabling FIPS on the server, as shown below, create a situation that doesn’t allow the TLS handshake to complete, so web based services that rely on IE settings break.
The lesson here is that if you have FIPS encryption enabled on the computer that you are accessing vCenter from, ensure your IE settings allow TLS 1.0. Normally TLS 1.0 is checked, so this won’t be a problem for most people. But if you are trying to enhance security by only allowing TLS 1.1 or higher, then you will run into issues.

VMware VUM 4.1 U1 SSL Certificate Replacement

One of the continuing pain points with VMware vSphere is the unnecessarily complicated procedure to install trusted SSL certificates in ESXi, vCenter and VUM. Up until 4.1 Update 1 (released 2/10/11), VMware had no public procedures to update the VUM SSL certificate, over 1.5 years after vSphere 4.0 hit the streets. Plus I’ve found that even the published procedures for ESX(i) and vCenter were convoluted and incomplete.

So over the last couple of years I’ve written several blogs about how to replace your ESXi, vCenter and VUM certificates. VMware made a little progress with VUM 4.1 Update 1, in that they now have a GUI utility that performs the behind-the-scenes reconfiguration of VUM to use a new SSL certificate. This new tool is called VMware Update Manager utility and does more than just update your VUM SSL certificates. You are still left with a painful process for ESXi and vCenter, so maybe in vSphere 5.0 VMware will wake up and provide a more streamlined procedure.

Even with the new tool in 4.1 U1, I found the associated KB article less than helpful and even tells you to leverage openssl on an ESX (not ESXi) host to generate new self-signed certificates. The second half of their article has instructions for using a trusted commercial CA certificate, but they still have you leveraging ESX to generate the certificate requests. This boggles my mind for several reasons:

1) ESX 4.1 is the last release and is a dying code branch. VMware has stated ESXi is the future and the only option in vSphere 5.0.

2) OpenSSL is not included in ESXi so you can’t follow the KB article if you only run ESXi . Why publish a KB article that doesn’t apply to organizations that only use ESXi?

3) OpenSSL is open source and widely available for free for many platforms including Windows. vCenter and VUM only run on Windows, so it makes a lot more sense to have customers download Windows OpenSSL and generate the certificates on a Windows computer.

4) Even if you have ESX, VMware always lags in incorporating the latest version of OpenSSL into ESX, so you could be using a version with known vulnerabilities.

I’m not trying to bash VMware, but come on guys, please get with the program. As a testiment to the SSL problems VMware has not addressed, my SSL blog posts get alot of hits. Until VMware “gets it right” I’ll continue to help the community at large. So to that end, let’s get on with how to update VUM SSL certificates in VUM 4.1 U1.

1. Download OpenSSL Windows binaries here. I recommend the full v1.0.0c package. Install OpenSSL using all default values on any Windows computer. I put OpenSSL on my vCenter server since I need it for ESXi and vCenter SSL certificate generation.

2. Generate a 2048-bit RSA private key (you could use 1024 bit as well, but I like stronger keys):
openssl genrsa 2048 > rui.key

3. Create a certificate request based on the previously generated private key:
openssl req -new -key rui.key > rui.csr

For the certificate request parameters (in green) use the values appropriate for your organization. The critical parameter, the common name (in red), should be the FQDN of your VUM server. Do not use a challenge password.

4. At this point you have a valid certificate request and you can submit it to a commercial CA, or your internal trusted CA. For the purposes of this article I will leverage a 2008 R2 Microsoft CA, so some steps may vary if you use a commercial cert.

5. Use NotePad and copy the contents of rui.csr to the clipboard.

6. Navigate to your Microsoft CA, click on Request a certificate, click advanced certificate request, click Submit a certificate request by using a base-64-encoded CMC….

7. On the Saved Request screen paste the contents of the clipboard, and change the certificate template to Web Server (or your organization’s web server template name).

8. Submit the certificate request and download it as base-64 encoded WITHOUT the certificate chain, and save it with a filename of rui.crt.

9. Type the following command (and use a blank password when prompted):
openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -out rui.pfx

10. Stop the VMware vCenter Update Manager service.

11. Backup the existing VUM certificates located your VUM directory, by default it’s :
C:Program Files (x86)VMwareInfrastructureUpdate ManagerSSL.

12. Copy your new rui.crt, rui.key and rui.pfx files to the SSL directory above, replacing the existing files.

13. Navigate to C:Program Files (x86)VMwareInfrastructureUpdate Manager and launch VMwareUpdateMangerUtility.exe. Login with your vCenter administrator credentials.

14. Click on SSL certificate then check the box under the instructions and finally click Apply.

15. Restart the VMware vCenter Update Manager Service and pray it starts.
16. If you’ve left any of your certificate files laying around the file system, except in the VUM SSL directory, you should back them up to a secure location then delete them. You need to protect the private keys, so don’t leave them laying around just anywhere.
17. Launch the vSphere client and connect to vCenter. Verify that the VUM tab appears and that you can access VUM without any errors. It would also be smart to check the vCenter Service Status from the vCenter home page to ensure everything looks healthy.

Potential CommVault licensing gotchas for VMs

Some backup software manufacturers are offering a capacity based licensing model, instead of a agent based model. Depending on your situation, this may be a dramatically easier and cheaper licensing model. Traditionally you had to buy per-server licenses, per-agent licenses, per-library licenses, and maybe other options as well. Backup software licensing could get very complex and very expensive. But it’s extremely important to understand how their model works, for both physical and virtual servers or you may be in for sticker shock. CommVault licensing may surprise you how they count capacity in your environment.

CommVault, Symantec, and others have introduced capacity based models. With this model, you have unlimited number of agents and servers, but the total amount of data you want to be backed up must be licensed. Depending on the vendor, they may have slight variations on this model. Tivoli capacity licensing is nearly as complex as their agent based model.

When can this model be more cost effective? Generally if you have a lot of servers with minimal data on them, this model works well. Per-server and per-application agents can get expensive. If you have a small number of servers with huge data stores, then a per-server/agent method could be more cost effective.

But, you need to be VERY careful and understand how the backup product measures capacity, if you go with that model. In a physical environment, it’s generally very simple. If your pizza box server has 4TB of storage but you’ve only written 500GB of data, 500GB would count towards your capacity license. The remaining 3.5TB is ‘free’, until you start to physically use it.

In a virtual environment, this can get more complicated, and you could be in for some nasty surprises. Let’s say you do a P2V migration of that same pizza box server, but you downsize the virtual disks to just 1.5TB. Now the million dollar question is, how much capacity counts towards your backup license? 500GB or 1.5TB, or something in between?

With CommVault Simpana 9.0, the answer is ‘it depends’. CommVault counts the VMware VMDK disk size against your capacity license, regardless of how much physical space the VM is using. If you use VMware thin provisioned VMDKs, then at least 500GB comes out of your capacity license. If you use thick VMDKs and rely on your storage array to do the thin provisioning (which nearly all modern arrays support), the full 1.5TB counts against your license!! This is because Simpana is not intelligent enough to look inside the VM for actual disk usage and just charge you for the allocated amount. It looks at the VMDKs as a big blob, and charges you accordingly.

As a result, licensing for a virtual server can be significantly more expensive per GB than a physical environment, at least with Simpana 9.0. I don’t know how NetBackup or other capacity based products count VMDK storage. I’d hope they are more consistent and use the physical server logic. If my VM has a 2TB hardware thin provisioned disk, but I’ve only written 500GB of data, only 500GB should count.

Using VMware thin provisioned disks doesn’t fully solve the problem. Why? Let’s say you have 2TB software thin provisioned disk, and the allocated VMDK space is 500GB. If you copy 1TB of data into the VM, then delete it, the VMDK is now 1.5TB but only contains 500GB of data. So you pay 3x the price for the same 500GB of data, unless you somehow shrink the VMDK.

Finally, if you leverage VMware fault tolerance, you simply can’t use VMware thin provisioned disks. You must use EZT (eager zeroed thick) disks. So regardless of how much or how little disk space you use, the total VMDK disk size counts against your license.

This can be very confusing. CommVault has a two methods for counting capacity that you need to be aware of. One model for physical servers and another for virtual servers that can catch you off guard depending on how your VMDKs are configured.

Authentication Denied: Unable to logon to an ESXi 4.1 console

Over the last couple of days I’ve been performing ESXi 4.0 to 4.1 build 320137 upgrades. During the upgrade process today one of my servers had a hiccup and the web services was not responding. After a couple of reboots and non-responsive web services I iLO’d in (it’s an HP server) so I could get to the ESXi DCUI (direct console user interface), AKA the yellow screen. To my shock when I pressed F2 I got:

Authentication Denied: Direct console access has been disabled by the administrator for

At first I thought OK, maybe someone enabled lockdown mode and I didn’t know it. Checked a few things, nope, no lockdown mode. After more poking and prodding, I found the root cause of the problem. But it’s a mystery to me why this is occurring. The only consistent theme is that the server was on ESXi 4.0 and they were upgraded to ESXi 4.1 build 320137.

So what was the problem? New to ESXi 4.1 is the Security Profile configuration screen. Here you can stop/start several low-level system services. On 25% of my upgraded boxes the “Direct Console UI” service was in the stopped state as shown below.

The solution is to reconfigure the service to Start and Stop with host, which is the ESXi 4.1 default configuration. After I started the service, viola, DCUI access was restored!

Since this happened on several boxes, but not all, I’ll chalk it up to another VMware bug. So in my upgrade procedures I’m adding a check to verify the service status before we bless an upgrade as being complete.