Archives for June 2010

VDI Smackdown..who wins?

If your organization is looking at deploying VDI (Virtual Desktop Infrastructure), I highly recommend you check out the VDI Smackdown report by PQR. It has a very lengthy and highly detailed feature comparison matrix of leading VDI solutions such as Citrix XenDesktop 4.0 SP1, Microsoft Server 2008 R2 Remote Desktop Services, and VMware View 4.0.1. I’ve never seen such a long comparison table of features between VDI vendors.

There’s no one best solution, as requirements vary widely between organizations and situations. After you think about your VDI use cases, the Smackdown will really help you see how each product stacks up. The full 33 page report is worth reading in its entirity.

RMS Automatic Template Downloading on Windows 7

Recently the project I’m supporting is looking at RMS to provide information rights management (IRM) on some documents. Windows RMS provides two means to let users protect content. First, there is the ad hoc method that lets a user specify what protections they want to put on their content, and what users/groups it applies to. Second, an RMS administrator can configure standard templates (i.e. “Company Confidential”) which all users in the enterprise can use. In most organizations both content protection methods have their place.

However, I’m disappointed in how Microsoft implemented these templates. You’d think the RMS client would dynamically query the RMS server for available templates when you want to protect content and present them to the user for selection. However, it’s much more brain dead and less dynamic. In Windows XP and Vista RTM, the administrator had to ‘manually’ copy the XML templates to a special directory on each and very computer for every user. Most used a GPO or logon script. Still a kludge if you ask me.

Starting with Vista SP1 and later, including Windows 7, Microsoft included a scheduled task called “AD RMS Rights Policy Template Management” which discovers the RMS servers in the environment and downloads the templates for each user. It’s triggered to run every day at 3AM or at user logon time.

However, the default configuration of this task is brain dead. Under HKCUSoftwareMicrosoftMSDRMTemplateManagement there’s a key called “lastUpdatedTime” which gets populated each time the scheduled task runs. There’s also another key called “UpdateFrequency” which is set to 30. What does the 30 mean? It will only download templates once every 30 days. Even if you manually run the task it won’t touch the RMS servers. The minimum you can set the frequency to is once a day (1). You can, however, delete the “lastupdatedtime” key and it will check the RMS server and re-populate the key.

Also another very important point is to add the RMS FQDN to each user’s Local Intranet security zone in IE. If you don’t do this then the task won’t authenticate to the RMS IIS server and you will get a Last Run Result of (0x8004CF43).

If the task worked the scheduled task probably has a Last Run Result of (0x4CF04). To confirm the templates actually downloaded, go to your profile directory and under C:Users%username%AppDataLocalMicrosoftDRMTemplates you should see one XML document for each template defined in the RMS console. If not, make sure you have invoked a document protection attempt in office so that it discovers your RMS server.

Another annoyance with RMS is that Office isn’t smart enough to look in this templates folder by default. NO! Let’s make it harder on our admins to get all of this working. Under:


you need to create a REG_EXPAND_SZ value with a name of AdminTemplatePath with a value of:


Why Microsoft needs to make this so difficult is beyond me. Personally I think the embedded RMS client should make a dynamic web services call to the RMS server when a user wants to protect content, get the latest templates, and cache them locally. Office needs to look at the default template location too. Also remember the scheduled task is NOT enabled by default. So if your organization is going to use RMS, you need to configure a GPO or script to enable the task on all Vista SP1 and later clients. If you are going to use Remote Desktop Services (RDS) or XenApp, enable the scheduled task on your servers.

PowerShell command to change Windows Cipher Suite Order

While journying down the whole cipher suite road this weekend, I put together a little one liner that reconfigures the cipher suite order that Windows will try and use. As I mentioned in a previous blog, you can configure this via GPO. But, maybe you want to build in the configuration to a golden image. You probably have other PowerShell scripts to configure your golden image, so you can throw this command in to tweak the cipher suite order.

The command only works on Windows Server 2008 R2 and Windows 7. If you use Vista or Server 2008, look at your existing registry key for the list of cipher suites then modify the script. Many of the new cipher suites are not availabile on 2008/Vista.

After you cut and paste the script to your computer remove all line breaks and spaces in the cipher suite string. This is important, as we are at the 1024 character limit of a PowerShell string.


Easily configure your Windows Cipher Suites!

After many hours of digging around the Windows registry and experimenting with various keys to enable TLS 1.2 on Windows Server 2008 R2 and Windows 7 (see my blog post here), I found this free tool that gives you one click access to configuring your Windows Cipher Suites. The Harden SSL/TLS tool is in beta, but worked great for me.

The tool lets you enable/disable protocols, hashes, key exchanges, and prioritize the cipher suites. One click exporting to registry files, and PCI DSS compliance. The author also has a few other tools and whitepaper as well.

The SSL hardening tool seems to make system changes in real time. My suggestion is to use a VM, take a snapshot, run the tool, export the registry settings, and then look at the registry file. Compare the registry file to the clean snapshot then decide what registry changes you want to push to production servers.

The tool is still in beta, and I had it lock up on me a few times. So using it to see what registry key changes you need to make manually is a pretty safe bet. Don’t run it on production systems!

I also found a web site that would let you query SSL enabled site, score them on security, and show the cipher suites they support. Great for checking out your bank or other secure sites you need to trust. My bank scored an 81, which was pretty good. Check out Qualsy SSL Labs.

Enable TLS 1.2 Ciphers in IIS 7.5, Server 2008 R2, Windows 7

Some industries, like Government, require the use of certain cryptography algorithms. One of the great features of Windows Server 2008 R2 and Windows 7 is the support for TLS 1.2 ciphers. TLS 1.2 ciphers support AES-256 encryption with SHA-256 hashes. Unfortunately, Microsoft did not enable these protocols out of the box. I wanted IIS 7.5 to negotiate TLS 1.2 connections with my Windows 7 clients. After some registry hacking I was successful, as shown by a Wireshark trace.

I created a simple PowerShell script that enables TLS 1.2 for both client and server communications. It also disables SSL 2.0 server responses, in case you need to be PCI compliant. The lines are pretty long, so pay attention to the wrapping. After you run the PS script (with elevated rights) you must reboot the client and server.

Please note that all values must be DWORD. This is very important, as any other value type will NOT work and you may be pulling your hair out wondering why it’s not working.

# Enables TLS 1.2 on Windows Server 2008 R2 and Windows 7

# These keys do not exist so they need to be created prior to setting values.
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2"
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server"
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client"

# Enable TLS 1.2 for client and server SCHANNEL communications
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server" -name "Enabled" -value 1 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server" -name "DisabledByDefault" -value 0 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client" -name "Enabled" -value 1 -PropertyType "DWord"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client" -name "DisabledByDefault" -value 0 -PropertyType "DWord"

# Disable SSL 2.0 (PCI Compliance)
md "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Server"
new-itemproperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Server" -name Enabled -value 0 -PropertyType "DWord"

After you run the PowerShell script, you should see DWORD entries like those shown below. Also, go into the Advanced properties of IE and check the box next to TLS 1.2.

If you start a WireShark capture on a TLS session you will know it’s v1.2 by two easy methods. First, the Protocol column will show TLSV1.2. Secondly, if you open the ServerHello packet you should see burried in the packet:

Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)

You can also configure the advanced settings in IE8 on Windows 7 or Server 2008 R2 to only use TLS 1.2 by de-selecting all other SSL/TLS options. If your browser can connect with just TLS 1.2 selected, then you are golden. But for 100% verification, use a packet sniffer.

Another tweak you can do to your system is change the order of the cipher suites that Windows negotiates. This is possible in Windows Vista and higher. I changed the order such that TLS_RSA_WITH_AES_256_CBC_SHA256 is at the top of the list. Next to elliptical curve ciphers, this is the strongest that Windows offers.

To change the cipher suite order, open the GPMC on a Server 2008 or higher DC and navigate to: Computer\ Configuration\Policies\Administrative Templates\Network\SSL Configuration Settings. Enable the policy, then copy the cipher suites to Notepad and change the order as you wish. I just flipped the first two entries. So my first two entries look like:


I haven’t done extensive testing to know what types of compatibility problems enabling TLS 1.2 may create. So as always, test, test, test! I confirmed with WireShark that the Windows RMS client in Windows 7 will use TLS v1.2 to contact the root Windows Server 2008 R2 RMS server.

During my research I stumbled upon a cool Microsoft web site that lets you test various cipher suites with your browser. If you click on the cipher suites on the server authentication line you can see what your browser will support.

Lastly, Microsoft has a good reference of which cipher suites are associated with which protocols.

PXE Booting a Wyse zero client, Part 1

In some industries end point security is a significant concern, so organizations have been moving away from a traditional fat client to thin clients, and even to zero clients. Traditionally thin clients have some security and maintenance issues of their own. If they run Window Embedded you still need to do patching, customize the image, and remotely manage the devices. There’s also a chance you can store data on the device, depending on the configuration.

Recently there has been a trend towards ‘zero clients’ which either eliminates or radically changes what runs on the client device. A true zero client would have no operating system or software to ever update. Panologic has true zero clients. But if you are moving towards a mainstream VDI solution like VMware View or Citrix XenDesktop, your client hardware needs a little OS to provide the connectivity into the VDI environment. Alternatively you could use Wyse WSM to “stream” Windows XP or Windows 7 to one of their devices, but that is only a LAN based solution. We needed a solution that would work across the WAN.

Wyse supplies ThinOS for some of their thin clients, which has a smaller attack surface area than embedded Windows or Linux operating systems. ThinOS is just 4MB in size, so it’s very small. For some organizations though, having any embedded flash memory is a security risk, even if end users can’t easily write data to it.

For Wyse products you have two solutions that completely remove the flash memory from the thin client, yet provide you connectivity into VMware View, Citrix XenDesktop, and Microsoft Remote Desktop Services. Both involve PXE booting the Wyse zero client to download the micro OS, which then runs from RAM. When you power down the device all memory is lost, and there’s no local storage.

One is their V00LE and the other is Xenith. What Wyse doesn’t tell you on their public web site is that both of these models can be ordered with zero flash memory and PXE booted to download their operating system. The V00LE uses Wyse ThinOS and the Xenith uses a special XenDesktop only operating system. The project I support purchased a few V00LEs for a proof-of-concept. Once you get the hang of it, PXE booting the V00LE is easy, but in trying to make it work I encountered a few issues.

1. You don’t need to use WSM to PXE boot ThinOS or Xenith OS. In fact, Wyse recommends against using WSM to “stream” their embedded operating systems.

2. You cannot use the ThinOS BIOS downloads that are available to registered users. You need to contact your Wyse rep to get their PXE bootable versions of the firmware. If you try and use the stock downloads it will either hang or say the image is too big.

3. The PXE boot process requires the use of a TFTP server to download the PXE boot strap binary and the ThinOS firmware. For my purposes I used the Windows Deployment Services (WDS) that comes with Windows Server 2008 R2, since it comes with a PXE and TFTP server.

4. ThinOS needs to download a text configuration file once ThinOS has booted. This can be from a FTP site or HTTP(S) source. I chose a HTTP source since I could later apply SSL if need be. The configuration file controls the security and connection details of the device.

After I got over all of these hurdles, I was able to PXE boot our V00LE with ThinOS. The boot time is just a matter of seconds and its really slick. For part 2 I will provide more technical details on how to configure DHCP and the TFTP server to make the PXE boot process work.

XenDesktop 4 Provisioning Services Firewall Rules

A couple of weeks ago Citrix released Provisioning Services 5.6, which is a component of their XenDesktop 4.0 suite. Provisioning services is supported on Windows Server 2008 and Windows Server 2008 R2, however it does not automatically add firewall rules. In fact, when you configure provisioning services it tells you to either disable the firewall (not smart!) or manually configure the rules. Personally I think it should automatically configure the rules since it’s not rocket science.

Until Citrix automates the firewall rule creation process, I wrote a little script that opens all of the Citrix default ports to support PXE, TFTP, SOAP, and the streaming services. Of course if you change the default ports or installation paths, you will need to tweak the script.

Unfortunately the commands are pretty long so they will line wrap. Just paste these into a .cmd file and run them from a command prompt. If you want to increase the security of your server, you can limit the remote IPs to particular IPs or subnets that will be accessing these services.

@echo off
:: Configures Windows Server 2008/R2 firewall for Citrix Provisioning Services.
:: Includes PXE and TFTP services.

Echo Configuring Windows Advanced Firewall for Citrix Provisioning services.

netsh advfirewall firewall add rule name=”Citrix PXE Services (UDP-in)” dir=in action=allow protocol=UDP Profile=domain localport=67,4011 program=”%ProgramFiles%CitrixProvisioning ServicesBNPXE.exe” description=”Allows inbound PXE boot connections.”

netsh advfirewall firewall add rule name=”Citrix TFTP Services (UDP-in)” dir=in action=allow protocol=UDP Profile=domain localport=69 program=”%ProgramFiles%CitrixProvisioning ServicesBNTFTP.exe” description=”Allows inbound TFTP connections.”

netsh advfirewall firewall add rule name=”Citrix SOAP Services (TCP-in)” dir=in action=allow protocol=TCP Profile=domain localport=54321,54322 program=”%ProgramFiles%CitrixProvisioning ServicesSoapServer.exe” description=”Allows inbound SOAP connections.”

netsh advfirewall firewall add rule name=”Citrix Streaming Services (UDP-in)” dir=in action=allow protocol=UDP Profile=domain localport=6905-6930,10802-10803 program=”%ProgramFiles%CitrixProvisioning Servicesstreamprocess.exe” description=”Allows inbound Citrix Streaming connections.”

After you run the script if you look in the Windows Advanced inbound firewall you should now see the following rules created.

Extracting VMware ESXi pvscsi boot floppy images

When VMware released ESX(i) 4.0 U1, they officially supported booting your OS drive from a paravirtual SCSI controller. Prior to U1 you could only use one of their other SCSI controllers such as BusLogic. However, when you try and install Windows with a PVSCSI controller it will not detect any mass storage devices since it doesn’t have the drivers. But where do you find the drivers?

I exclusively use ESXi so there are a few extra steps required to find the VMware boot floppies. Instead of installing ESX on a host to extract the tools, I found a better way for ESXi users that only takes a few minutes and doesn’t require using ESX.

Note that the PVSCSI drivers in ESX 4.0 Update 2 (U2) are the same as the ones in U1, so you don’t need to get new floppy images when you build VMware vSphere Update 2 VMs. The boot floppies work equally well on ESX.

1. Go to the VMware patch download page, select ESXi, then click on the search button.

2. vSphere 4.0 Update 2 was just released a few days ago, so I downloaded the upgrade-from-esxi4.0-4.0_update02 zip. You should be able to download any update that includes vmware tools.

3. Download 7Zip, if you don’t already have it and install it.

4. Using 7Zip open the zip archive and navigate deep into the archive tree to this path:


5. Extract all three floppy images (pvscsi_1.0.0.5-signed-Windows2008.flp, etc.) to a folder on your computer.

6. Using the vSphere client you can now attach these floppy images to a VM and Windows can use them during the installation process. You could also copy the floppy images to a VMFS datastore and mount them from there.

Now you can easily extract the PVSCSI controller floppy images from any ESXi patch bundle that includes updated VMware tools and use them to build Windows VMs. If you want more information on support and limitations of the PVSCSI controller type, check out this VMware KB article. For some performance data on PVSCSI controllers, check out this VMware technical paper.

Update: For vSphere 4.1, download the ESXi upgrade bundle then go to the following path using 7ZIP:

Update 2: If you want to directly integrate the drivers into your Windows 7 or Server 2008 R2 ISO image so you don’t need to mount a virtual floppy during installtion, check out my blog post here.

Update 3: For people wanting to use ESXi 4.1 U1, grab the upgrade package ( then use 7ZIP to navigate to: embeddedEsx4.1.0ESXi-4.1.0-update01vmware-esx-tools-light-4.1.0-1.4.348481.i386.vibdata.tar.gzdata.tar.4.1.0floppies. These floppy images have a datestamp of 1/12/2011.

SIA338: Authentication & Passwords, The Good, The Bad & The Really Ugly

One of the sessions that I attended today was by the very popular Marcus Murray. He’s a Swedish security expert and Microsoft security MVP. I heard him speak a couple of years ago at TechEd Orlando and was really blown away by the content and his presentation style. Thankfully Microsoft let him come back and do another session this year. Some of the content was from his Orlando session, but also included some great new content. You can actually download his whole presentation from his blog. Since you can access the presentation, I’ll just summarize a few of his attack demos:

Pass the Hash. This was one of the demos he did in Orlando, which scared the heck out of everyone. Although it’s not a new attack vector, the results are scary. In a nut shell, this is how it works. Whenever someone, such as a domain administrator, logs onto a Windows machine their password hash gets stored on the computer. If another user or attacker can get local admin access, there are tools which can dump that hash. You can then take that hash (with another tool) and impersonate that use to other servers on the network, compromising them. If you can grab a domain admin account hash, you have instantly 0wn3d the domain. If you can access a domain controller you can now dump the hashes for everyone in the entire domain and you can impersonate anyone at will.

After TechEd Orlando I went through the entire exercise in my home lab and was able to duplicate the results and escalate from a local workstation administrator to a domain admin in less than 30 seconds. If you combine these tools with a web-based attack and local privilege escalation techniques, an attacker could remotely 0wn your entire corporate domain in a matter of seconds. Think about that for a little bit! No password cracking required, and password length is irrelevant.

Passing the Dutchie. This is similar to pass the hash, but the attacker uses an untrusted computer that is not part of the domain. The attack works by using a man-in-the-middle attack using an SMB relay to use NTLM credentials to access resources on domain joined computers.

Computer Account default password. Sometimes deployment tools or other methods of creating a computer account in AD improperly configure the password and set it to the name of the computer. If for some reason the password doesn’t get changed (machine never joined to the domain, etc.) this computer account can be used to access network resources. Marcus said 95% of his clients have this problem with at least one computer account in AD. So chances are your network is vulnerable to this attack!

Passing the Google Cookie. In this attack Marcus snatched Google docs cookies from the network (WiFi or wired), and injected them into IE and was able to impersonate the user and access their documents. This is similar to pass the hash, but it used cookies and the Google docs web site. Google doesn’t send their cookies over SSL, so beware! You can see a video of this attack on the his blog link I provided above.

One of the new tidbits of information that was brand new to me is the concept of Authentication Assurance in Windows Server 2008 R2 using smart cards. In a nut shell this is a new feature in Windows Server 2008 R2 that lets Windows dynamically add you to a security group if you logon with smartcard using a particular certificate template. If you use this security group to ACL resources, such as a file share, then attacks like pass the hash and pass the dutche are foiled since a smartcard was not used for authentication and you aren’t a member of the special security group. This security group could be extended to IPsec to only allow users that authenticate with a smartcard to access any resources on a particular computer.

Another countermeasure that he mentioned were some new settings in Windows 7 and Server 2008 R2 that can audit and even eliminate the use of NTLM on the network. There are some new group policy settings that you can use to block all NTLM authentication. This is easier said than done since some applications or methods of accessing resources don’t work with Kerberos, such as accessing a file share via IP address vice a hostname. Even if you can’t fully disable NTLM for all use cases, you can at least start auditing to get a handle on how widely its used on your network. This is one reason why I always try to Kerberos enable ANY application on the network, such as SQL, SharePoint, IIS, vCenter, etc.

DAT401: SQL 2008 HA/DR case studies

This session talked about several proven methods of high availability and disaster recovery for SQL 2008. It focused on several case studies of real-word companies, their HA/DR approach, and metrics from their environments. It didn’t cover any new wizbang technology or third party products. With the proper design, processes, procedures, and highly skilled people, it’s really mind boggling what companies have done.

There are a few common HA/DR architectures:

Failover clustering for HA and database mirroring for DR
– Synchronous database mirroring for HA/DR and log shipping for additional DR
– Geo-clustering for HA/DR and log shipping for additional DR
Failover clustering for HA and SAN-based replication for DR
– Peer-to-Peer replication for HA and DR

Each architecture has its own pros and cons. Your business requirements will determine which solutions you will want to employ. The remainder of the session was discussing various case studies.

One case study, bWin, is an online gambling company in Europe. They process 1 million bets a day on over 90 sports. Their SLA is zero data loss, 99.99% availability 24×7, and they have an unlimited IT budget (no kidding). Their design had to take into account a full datacenter failure and complete data loss within that datacenter. Total data is in excess of 100TB, 100 SQL instances, and the environment processes over 450K SQL statements per second.

Their solution, which is highly complex with extreme levels of redundancy, has enabled them ZERO downtime in three years, zero data loss, and near 100% verified data availability. Their backup and storage architectures are really mind blowing. It is well worth reading the case study, here. If you want to read about their backup architecture, you can find the case study here. They can backup a 2TB database in 36 minutes. People were starting to laugh in the session at the extreme lengths this company went to ensure verified zero data loss.

The key take way from this case study is that you need to document everything, have processes and procedures in place for every scenario, and have extremely highly skilled people. The technology is just one small piece of the entire design. It’s really the processes and people that enable these extreme levels of up time and data availability. You can have all the technology in the world but if your document is poor and you don’t have extremely highly skilled people, you will end up in a world of hurt and miss your SLA targets.

Another case study was ServiceU. In summary, they were able to upgrade from SQL 2005 to SQL 2008, Windows Server 2003 to 2008, a new SAN, and new server hardware, with less than 16 minutes of total downtime. This was accomplished without any virtualization product and through careful planning and orchestrating of the upgrades.

Other case studies include QR Limited, Progressive Insurance, and an Asian travel company. Bottom line is that SQL can provide highly robust HA/DR if you have the right architecture, documentation, processes, and highly skilled people.