Safely virtualizing Windows Server 2012 Active Directory via Generation-ID

Windows Server 2012 generation ID is a great new feature that will allow use to safely virtualize a domain controller, on specific hypervisors. One of the really great features that hypervisors have had for ages is the ability to perform snapshots, then roll back to a prior state with a click of a mouse. Invaluable feature in both the lab, and in production.

I know during all my (failed) vSphere 5.1 installs I practically wore out the revert to snapshot button in vCenter. But, there is at least one class of VMs that you almost NEVER want to roll back from a snapshot with, those which are vector-clock synchronized software such as Active Directory.

Why is rolling back AD bad? I mean why is rolling back AD *REALLY* bad? Microsoft has these little things called USNs, or Update Sequence Numbers. A USN is an Active Directory database instance counter which gets incremented each time an update to AD is made. USNs are unique to each DC, and use a monotonically increasing value. USNs are used to determine what changes need to be replicated to other DCs.

When you revert to a snapshot a USN rollback occurs. What can happen if a USN rollback occurs? Lots of bad things, such as missing AD objects, wrong security group memberships, passwords are reset, and re-appearing AD objects. Also, DCs that are rolled back may accumulate many changes which never get replicated to other DCs. In short, the AD consistency of your forest is SHOT.  Starting with Windows Server 2003 SP1 and later, an event log ID 2095 is generated if a USN roll-back is detected, but it’s up to you to fix the mess. Microsoft has a great KB article here that goes into a lot more detail.

What has Microsoft done in Windows Server 2012 (and Windows 8) to address this problem? They’ve introduced a safeguard called a VM-Generation ID, which can be implemented by any hypervisor. This generation ID can be used by applications and operating systems to detect if a virtual machine has been rolled back in time, and take appropriate measures.

So what happens when AD detects that the Generation IDs have changed? First, it dumps the RID pool, then does a non-authoritative synchronization of the SYSVOL folder. AD replication is then re-established to other DCs, to bring the reverted DC back into a consistent state with the rest of the forest.

Sounds great right? Well it is, but only a very limited number of hypervisors support VM-Generation ID. As of this writing the hypervisors are Hyper-V 3.0, vSphere 5.0 U2, and vSphere 5.1. Since a USN rollback is quite unpleasant, you of course want to verify that WS2012 and your hypervisor are playing nice and using the Generation-ID feature. If you look in the Directory Service event log, you will see event ID 2168 and 2172. In the screenshots below they have the same Generation-ID, since the VM was not reverted to a previous snapshot.

To test out this new feature I fired up my vCenter 5.1 web console and took a snapshot of my WS2012 domain controller. After the snapshot completed, I created a new group on another DC, then reverted the WS2012 DC back to my snapshot. Let’s look in the event viewer and see what happened:

Yes, AD realized it was reverted back to a prior snapshot…

Microsoft even tells you that snapshots are not backups, and silly, use an AD aware backup program to restore AD.

And now life is almost good…

Let’s freshen up FRS a little bit while we are at it…

Nothing like a new database to start off the day with…

A touch of USN cleanup…

And a few minutes later, everything is back in sync! As you can see from the screenshots, Microsoft is very verbose in the logs on exactly what is happening and why. In a very large forest with a lot of DCs the recovery process could take longer.

So under what circumstances does the Generation-ID change and not change? Here’s a list:

Generation-ID NOT changed when:
VM is paused or resumed
VM is rebooted
VM host reboots
VM is vMotioned/Live Migrated

Generation-ID IS changed when:
VM starts executing a snapshot
VM is recovered from a backup
VM is failed over in a disaster recovery environment
VM is imported, copied, or cloned

This feature alone should be a huge driver for deploying WS2012 based DCs on all of your hypervisors. Never thought I’d say this..but happy snapshotting your domain controllers! For even more detailed information on virtualized domain controllers, Microsoft has a great series of articles here you can read.

P.S. This feature does NOT work with array-based snapshots. The hypervisor tracks and creates the new Generation-IDs. So DO NOT revert a domain controller back to a prior state by reverting to a previous snapshot that your array created vice your hypervisor. With the forthcoming VVOLS in vSphere .Next, Generation-ID could be supported with hardware-snapshot offloads but we will have to wait and see if that’s the case.

MS Security Compliance Manager 3.0 Hits the streets

One of the absolute best tools for managing security group policy settings in a Microsoft environment is their Security Compliance Manager. Hot off the presses is version 3.0, which is a major step forward in both functionality and OS/product support.

The full product announcement from Microsoft is here. The most exciting news for me is full support of Windows Server 2012, IE 10, and configuring stand-alone machines. Oh yes, Windows 8 support, but who’s even using that?

Not new to the 3.0 release, is the ability to compare different baselines, archive baselines, and create your own custom baselines that you can export to a GPO. Your IA guys should love it!  And in case you missed it, there’s a beta version of a SCM baseline for SQL Server 2012 you can find here.

Create Trusted Remote Desktop Services (RDP) SSL Certificate

For Windows environments that want extra security, one of the features that has been around for ages is requiring TLS 1.0 for Windows RDP (Remote Desktop) connections. This functionality requires a certificate on the server, since TLS is based on the usage of X.509 certificates. Installing a RDP SSL certificate is easy.

By default Windows will create a self-signed certificate automatically for use with RDP. But as we all know, self-signed certificates are nearly worthless, and could easily be intercepted for man-in-the-middle attacks. So one should reconfigure Windows to use a trusted certificate. Thankfully this is fairly easy, and once configured, pushed down to all servers via GPO for automated deployment.

I’ve validated that this procedure works both on Windows Server 2008 R2 and Windows Server 2012. It may work on Windows Server 2008.It requires the use of a Microsoft enterprise online certificate authority. Again, I’ve used both Windows Server 2008 R2 and Windows Server 2012 CAs with success. Not surprising, since certificates are industry standard. For the purposes of this article I’ll use Windows Server 2008 R2 CA, and Windows Server 2012 “target” server.

The general process is first creating a new Certificate Authority certificate template that has an extended key usage to limit its use to only Remote Desktop TLS sessions. Second, we configure a GPO setting to automatically configure servers to request a certificate via this template, and use it for RDP TLS. Refresh GPO on the target server, and finally we attempt to connect via a stand-alone computer to verify it sees the certificate that we deployed.

Installing a RDP SSL Certificate

1. On your Microsoft certificate authority server open the Certificate Templates console.

2. Duplicate the Computer template and use the Windows Server 2003 Enterprise format (Server 2008 v3 templates will NOT work).

3. Change the template display name to RemoteDesktopComputer (no spaces). Verify the Template Name is exactly the same (no spaces). You can use a different name if you want, but both fields must match exactly.

4. Now we need to create an application policy to limit the usage to RDS authentication, then remove the other application uses for the certificate. On the extensions tab click on Application Policies then click on Edit.

5. Click on Add, then click on New.  Set the value of Name to Remote Desktop Authentication. Change the object identifier to

6. From the Application Policies list, select Remote Desktop Authentication.
7. Back on the certificate template properties, remove all other entries. Only Remote Desktop Authentication should be present.
8. If you wish, you can modify the validity period of the certificate, making it say two years instead of the default of one.
9. You probably want to secure your domain controllers as well, so for that we need to modify the security setting on the template. Open the Security tab and add the group Domain Controllers and give the group Read and Enroll (not Autoenroll).
10. Open the MMC snap-in for managing your Certificate Authority and locate the Certificate Templates node. Right click, select New, then Certificate Template to Issue. Choose the RemoteDesktopComputer template.
11. Next up is configuring the GPO to utilize the new template. You can modify any GPO you wish, or create a new one. Obviously the scope of the GPO should cover any servers that you want to secure with TLS. This could be a server baseline GPO, domain GPO, or whatever you want.
12. In the GPO editor locate the node Computer Configuration\Policies\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Session HostSecurity. Modify the Server Authentication Certificate Template setting. Enable the policy and enter the certificate template name that exactly matches what you created in your CA.

13. In the same GPO node, configure the Require use of specific security layer for remote (RDP) connections to use SSL (TLS 1.0).

14. Wait for the GPO to replicate, then refresh the GPO on a test server. Wait a minute, then open the Certificates MMC snap-in for the computer account. Look in the PersonalCertificates store for a certificate that has the Intended Purposes of Remote Desktop Authentication. If it’s not there, wait a minute, and refresh. If it never appears, something is wrong. Look at the gpresult to make sure your GPO is being applied to the server.

15. Once the certificate appears, double click on the certificate to open it. On the Details tab look at the first few characters of the thumbprint value and remember them.

16. To make sure the RDP service is aware of the new certificate, I restart the Remote Desktop Services service.

17. Open an elevated PowerShell prompt and run this command:

Get-WmiObject -class “Win32_TSGeneralSetting” -Namespace root\cimv2\terminalservices -Filter “TerminalName=’RDP-tcp'”

Validate that the Security Layer value is 2 and that the thumbprint matches the certificate. If both of those settings are correct, then you are good to go!

As a quick test I attempted to connect to this server from a non-domain joined computer that did not have the root certificate for my CA. I configured the RDP client to warn on any security issues. As expected, the client threw errors about the CRL not being available, and that it didn’t trust the chain. I also viewed the certificate and verified it was the correct one.

It seems Windows 8 has much more stringent certificate checking than Windows 7. The screenshots below are from Windows 7, in case you didn’t recognize the chrome. When using a Windows 7 non-domain joined computer to access the same TLS protected server, I got NO certificate warnings. That was even with the RDP 8 add-on hotfix. I’m glad to see Win8 does thorough certificate validation.

Connecting to the same server from a domain-joined computer that trusted the root CA resulted in no security warnings and a successful connection. If you look at a Wireshark capture you can also validate that CRL information is being exchanged between the computers, which means TLS is being used.

WMI GPO Filters for Windows Server 2012 and Windows 8

When deploying Group Policies in a Windows environment, often you may have different GPOs for different versions of the operating systems. With the recent release of Windows 8 and Windows Server 2012, it’s likely you will have new GPOs just for these operating systems. You could build out new OUs for the each OS type, but that can get messy rather quickly.

My personal preference for most cases is to use WMI filtering to limit which operating systems a GPO applies to. This way you can dump all your member servers in one OU, and filter GPOs based on OS type.

To create a WMI filter, first you need to open the GPMC and locate the WMI Filters node. Start the new WMI filter creation wizard, and enter a name of your WMI filter. I always put the OS type, so it’s clear what OS the filter is for.

Now you need to add the actual WMI filter by clicking on the Add button. Next up is the tricky part! You need to type in or paste the WMI query for your operating system type. There are several ways to do this, but I like using the OS version number, since that is independent of the OS flavor (enterprise, datacenter, professional, etc.). See the bottom of my post for all the OS WMI queries you can choose from.

After you have created the WMI filter, you now need to configure one or more GPOs to use the filter. At the bottom of the Scope tab on any GPO you will see the WMI Filtering option. From the drop down select the appropriate WMI filter.

And that’s all there is to it! You can create more complex WMI queries, that could cover multiple operating systems, or filter on almost any other computer property such as memory, particular application, etc. If you can query it with WMI, then you can probably filter a GPO with it.

You can also export/import WMI Queries from the GPMC as well, if you want to easily transport them between environments. As always, test them out before applying a GPO that may hose up an OS if they get the wrong settings.

Windows XP
select * from Win32_OperatingSystem WHERE Version LIKE “5.1%”

Windows 7
select * from Win32_OperatingSystem WHERE Version LIKE “6.1%” and ProductType = “1”

Windows 8
select * from Win32_OperatingSystem WHERE Version LIKE “6.2%” and ProductType = “1”

Windows Server 2003 R2
select * from Win32_OperatingSystem WHERE Version LIKE “5.2%”

Windows Server 2008
select * from Win32_OperatingSystem WHERE Version LIKE “6.0%” AND ( ProductType = “2” or ProductType = “3” )

Windows Server 2008 R2
select * from Win32_OperatingSystem WHERE Version LIKE “6.1%” AND ( ProductType = “2” or ProductType = “3” )

Windows Server  2012
select * from Win32_OperatingSystem WHERE Version LIKE “6.2%” AND ( ProductType = “2” or ProductType = “3” )

Inject Cisco UCS Drivers into Windows Server 2012 ISO

A few days ago I blogged about injecting VMware vSphere drivers into a Windows Server 2012 image, so you can seamlessly install WS2012 inside a VM with PVSCSI and VMXNET hardware. Next up is injecting Cisco UCS drivers into Windows Server 2012, in case you need to install Windows Server 2012 on a physical Cisco UCS blade. The process is the same as injecting the VMware drivers, but requires a little more effort to gather up all the right drivers.

Noteworthy is that you really need to do this on a Windows 8 or Windows Server 2012 computer/VM, and use the Windows 8 ADK. The DISM software for Windows Server 2008 R2 doesn’t understand the SHA256 digital signatures on some of the drivers and will barf unless you use an override switch.

1. Download the Cisco UCS Drivers disc v2.0.4 or later, which contain the Windows Server 2012 drivers. You can download the 2.0.4a (Oct 26, 2012) version here. Note: Cisco TAC account is required to download the software.

2. Find a Windows 8 or Windows Server 2012 VM that you can use to copy the drivers to and install the Microsoft ADK on. WS2012 comes with many drivers built in, so the only Cisco drivers we need are Network and Storage. All LSI drivers are built-in to WS2012, so you don’t have to worry about the built-in local storage controller on the server. For the Cisco UCS drivers, I created a folder called D:Boot Drivers64-bit.

3. Now that you have a folder to put the WS2012 drivers, you need to drill down into the Cisco driver ISO image and pull out the “W2K12” folders. In my case I only wanted the 1280 and MLOM drivers. Remember Cisco adapters are converged network adapters, so you need both the storage and network folders.

Under each adapter copy the W2K12 folder to your D:Boot Drivers64-bit folder and rename it to make it more clear what the driver is for. 
4. After I completed the copy process, I ended up with this folder structure. Folder names are not important, so call them whatever you want.
5. Download the Windows 8 ADK from here. Run through the installation wizard until you get to the feature set. All you need are the Deployment Tools. Finish the wizard and wait until it is installed.
6. Mount a virgin Windows Server 2012 ISO image and under the Sources directory copy boot.wim and install.wim to the root of your D drive.
7. Open the DISM command prompt and enter the following commands, or save them to a batch file and run the batch file. This assumes your ISO image has five WIMs on it, one boot WIM, and four OS WIMs. If you are unsure how many WIMs are in your install.WIM file, run:
dism /get-wiminfo /wimfile:d:install.wim
:: Creates mount directory for DISM
mkdir D:mount
:: Modifies the boot WIM
dism /Mount-Wim /WimFile:D:boot.wim /Index:2 /MountDir:D:mount
dism /image:D:mount /Add-Driver “/driver:d:boot drivers64-Bit” /recurse
dism /unmount-wim /mountdir:d:mount /commit
:: Modifies all of the Operating System WIM images
dism /Mount-Wim /WimFile:D:install.wim /Index:1 /MountDir:D:mount
dism /image:D:mount /Add-Driver “/driver:d:boot drivers64-Bit” /recurse
dism /unmount-wim /mountdir:d:mount /commit
dism /Mount-Wim /WimFile:D:install.wim /Index:2 /MountDir:D:mount
dism /image:D:mount /Add-Driver “/driver:d:boot drivers64-Bit” /recurse
dism /unmount-wim /mountdir:d:mount /commit
dism /Mount-Wim /WimFile:D:install.wim /Index:3 /MountDir:D:mount
dism /image:D:mount /Add-Driver “/driver:d:boot drivers64-Bit” /recurse
dism /unmount-wim /mountdir:d:mount /commit
dism /Mount-Wim /WimFile:D:install.wim /Index:4 /MountDir:D:mount
dism /image:D:mount /Add-Driver “/driver:d:boot drivers64-Bit” /recurse
dism /unmount-wim /mountdir:d:mount /commit
rmdir d:mount

8. Review all of the output and verify no errors occurred. If you see any invalid signature issues, then you probably aren’t using the Windows 8 ADK on Windows 8 or Windows Server 2012.
9. In your favorite ISO editing tool, such as UltraISO, over-write the boot.wim and install.wim with your customized version. Boot the server from your new media and it should now automatically recognized any vNICs and vHBAs you have provisioned via the service profile in UCSM.

HP Service Pack for ProLiant supports Windows Server 2012

Just released is the October 2012 HP Service Pack for ProLiant, which has full support for Windows Server 2012. Current support for Windows Server 2012 is focused on G7 and Gen8 servers. G6s and earlier will have limited to no WS2012 support. The 2012.10.0 release has these major changes:

  • Added support for Microsoft Windows Server 2012 and Microsoft Windows Server 2012 Essentials
  • Added offline support for HP Diagnostics and Array Configuration Utility (ACU)
  • Modified the user interface when booting a server to the SPP
  • Updated to HP Smart Update Manager 5.3.0
  • Added custom baseline functionality
  • Reports in comma-separated values (CSV) format
  • Linux RPM support
  • Fibre Channel switch firmware update support (B-series and H-series only)
  • Support for HP Integrity I/O card online firmware updates
  • Schedule pull from web repository downloads (not applicable to Fibre Channel switches)
  • Support for 16G Fibre Channel QLogic HBA
  • The ability to use the UNC format to identify file paths

You can download the full set of release notes and documentation here. To download the SSP you can go here. For more information on HP’s Windows Server 2012 support, go here. They also have what appears to be a late breaking supplement for the SPP you can find here.

Update: Find my blog post about the February 2013 HP Service Pack for ProLiant here.

Cisco UCS now supports Windows Server 2012

If you want to run Windows Server 2012 on Cisco UCS blades, fear no more! As of the 2.0(4a) release they now have official support in firmware, and an updated drivers disc. Just login to Cisco TAC, get the firmware and new driver disc. As you can see in the image below, they have directories for Windows Server 2012. You can check out my blog article about firmware 2.0(4b) which contains minor bug fixes, unrelated to Windows Server 2012.

New Cisco UCS Firmware – 2.0(4b)

Today Cisco release a new firmware bundle for their UCS servers, 2.0(4b). No major fixes in the bundle. You can find the full Release Notes here. If you want to run Windows Server 2012, 2.0(4a) is the minimum supported version you need to be running. You can check out my blog post about that here.

KVM Viewer no longer fails to establish a connection to the KVM Server if the trusted.certs keystore password is not the default. (CSCuc48582)

User passwords with more than three consecutive characters or numbers are no longer accepted by Cisco UCS Manager. (CSCtq09466)

When polling ipAdEntIfIndex, “No Such Instance currently exists at this OID to the MDS boxes” error is no longer received. (CSCub84958)

SNMP no longer returns a “No such instance” error when VRF context is configured. (CSCub90031)

Chassis with four PSUs and n+1 power redundancy no longer displays “Power state on chassis X is redundancy-failed” error. (CSCub84671)

The Cisco UCS Manager GUI no longer displays “Unable to authenticate this site certificate” messages. (CSCub94755)

The Cisco UCS B200 M3, B22 M3, and B420 M3 Blade Servers no longer experience `Server Hardware Not Supported’ or discovery errors when you are upgrading from Release 2.0(2) to Release 2.0(3) or 2.0(4) and the blades are inserted into a UCS DC chassis. (CSCuc35326)

The KVM Java client will no longer display an error/warning message stating that the KVM certificate to the blade has expired. (CSCuc26360)

The IOM upgrade no longer fails and gets into a continuous reboot after the IOM is activated by the FI. (CSCuc15009)

Cisco UCS Manager is no longer truncating the last digit of the license file id from the license. (CSCuc32555)

Inject VMware Drivers into Windows Server 2012 ISO Image

A while back I wrote a blog article on how to inject VMware drivers (PVSCSI and VMXNET3) into a Windows Server 2008 R2 and Windows 7 image. You can check out that article here. But given those are now legacy operating systems, I’m refreshing the procedures for Windows Server 2012 (they’d work on Windows 8 too).

One of the performance optimizations that I always include in our Windows VM templates is the VMware paravirtual SCSI driver. This is a high performance mass storage driver that is optimized for virtual environments and gives you the best disk I/O performance. Unfortunately Microsoft does not include it out of the box on any OS install disk. Plus, I like to include the VMXNET3 driver, so that the VM can use the high performance virtual NIC that VMware provides, without first having to install VMware tools.

The process below injects the required drivers into the Windows Server 2012 installation boot files, and the actual Window Server operating system, for a fully VMware aware image. Unlike Windows Server 2008 R2 and Windows 7, there is no separate Windows Recovery Environment WIM to modify.

1. Download the Windows 8 ADK (Assessment and Deployment Kit) from here. Never mind that it says Windows 8, as it will work with Windows Server 2012 since they are the same code base.

2. Start the installation process and after a long download select the two options below (Deployment Tools and Windows Preinstallation Environment (Windows PE)). WinPE is optional, but in case you need it in the future, I’d install it anyway.

3. Mount the Windows Server 2012 ISO. Navigate to the Sources directory and copy boot.wim and install.wim to your computer, say on the D: drive.

4. VMware provides both 32-bit and 64-bit PVSCSI/VMXNET3 drivers, and you must use the right one depending on what CPU architecture you are injecting the drivers into. Since Windows Server 2012 is 64-bit only, find a 64-bit VM for this exercise and go into the C:\Program Files\VMware\VMware Tools\Drivers\pvscsi folder.

Copy those files to D:\Boot Drivers\64-bit. Do the same for the VMXNET3 drivers, putting them in the same folder. The result should look something like this:

If you need the 32-bit drivers for Windows 8, then find a 32-bit VM running on vSphere and do the same copy process, but put those drivers into a Boot Drivers32-bit folder. To verify the supported architecture of the drivers, open the pvscsi.inf file and scroll down to the [Manufacturer] section. If you see NTamd64, you have 64-bit drivers. If you see NTx86, you have 32-bit drivers. The 64-bit pvscsi.sys file is also larger than the 32-bit version.

5. Create a folder on the D: drive called Mount.

6. To modify the boot.wim file type the following commands in the Deployment Tools Command prompt:

dism /Mount-Wim /WimFile:D:\boot.wim /Index:2 /MountDir:D:\mount
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\pvscsi.inf"
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\vmxnet3ndis6.inf"
dism /unmount-wim /mountdir:d:\mount /commit

7. Depending on your Windows Server 2012 ISO image, it may have varying amounts of images included. The VL ISO I have contains four indexes, or images. You can list the indexes with the following command:

dism /get-wiminfo /wimfile:d:\install.wim

9. Just to be safe, I want to modify all of the images just in case down the road I want to use another image, such as Windows Server Core.

dism /Mount-Wim /WimFile:D:\install.wim /Index:1 /MountDir:D:\mount
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\pvscsi.inf"
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\vmxnet3ndis6.inf"
dism /unmount-wim /mountdir:d:\mount /commit

dism /Mount-Wim /WimFile:D:\install.wim /Index:2 /MountDir:D:\mount
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\pvscsi.inf"
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\vmxnet3ndis6.inf"
dism /unmount-wim /mountdir:d:\mount /commit

dism /Mount-Wim /WimFile:D:\install.wim /Index:3 /MountDir:D:\mount
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers64-Bit\pvscsi.inf"
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers64-Bit\vmxnet3ndis6.inf"
dism /unmount-wim /mountdir:d:\mount /commit

dism /Mount-Wim /WimFile:D:\install.wim /Index:4 /MountDir:D:\mount
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\pvscsi.inf"
dism /image:D:\mount /Add-Driver "/driver:d:\boot drivers\64-Bit\vmxnet3ndis6.inf"
dism /unmount-wim /mountdir:d:\mount /commit

13. Create a backup of your OS ISO file, and then use your favorite ISO editing tool (such as UltraISO) and replace the boot.wim and install.wim files in the Sources directory. Now you can use the new ISO image to create a VM which uses the pvscsi controller for the boot drive and the VMXNET3 NIC driver.

Windows Server 2012 File Share Deduplication in action!

A great new feature in Windows Server 2012 is data deduplication. Data deduplication uses sub-file variable-size chunking and compression, which together deliver optimization ratios of 2:1 for general file servers and up to 20:1 for Hyper-V virtualization data. Microsoft has some great articles on dedupe, such as this one and a list of Powershell cmdlets here.

Major features include:

  • Scale and performance. Windows Server 2012 data deduplication is highly scalable, resource efficient, and nonintrusive. It can run on dozens of large volumes of primary data simultaneously without affecting other workloads on the server. Low impact on the server workloads is maintained by throttling of CPU and memory resources consumed.

  • Reliability and data integrity. Windows Server 2012 leverages checksum, consistency, and identity validation to ensure data integrity. And, for all metadata and the most frequently referenced data, Windows Server 2012 data deduplication maintains redundancy to ensure that the data is recoverable in the event of data corruption.

  • Bandwidth efficiency in conjunction with BranchCache. Through integration with BranchCache, the same optimization techniques are applied to data transferred over the WAN to a branch office. The result is faster file download times and reduced bandwidth consumption.

  • Optimization management with familiar tools. Windows Server 2012 has optimization functionality built into Server Manager and PowerShell. Default settings can provide savings immediately or fine-tune the settings to see more gains. Easily use PowerShell cmdlets to kick off an optimization job or schedule one to run in the future. Turning on the Data Deduplication feature and enabling deduplication on selected volumes can also be accomplished using an unattended .xml file that calls a PowerShell script and can be used with Sysprep to deploy deduplication when a system first boots.

Data deduplication involves finding and removing duplication within data without compromising its fidelity or integrity. The goal is to store more data in less space by segmenting files into small (32KB to 128 KB) variable-sized chunks, identifying duplicate chunks, and maintaining a single copy of each chunk. Redundant copies of the chunk are replaced by a reference to the single copy, the chunks are organized into container files, and the containers are compressed for further space optimization.

According to Microsoft’s blog article here, real-world performance impact is very minimal. Optimization rate for a single job runs about 100GB/Hr. Multiple volumes can be processed in parallel.

Let’s see how easy it is to configure data deduplication, and what kind of space savings we see. As a test I have the original 3.6GB ISO image of Windows Server 2012, and I have a modified version of the ISO that has additional files and software shoved inside. So 99% of the content is the same, but as you can see from the file sizes they are different. This file share is being shared from my file share cluster, created via my blog article here.

Now let’s configure data deduplication and see what kind of savings we see.

1. First we need to locate the volume on the file share cluster that is hosting my data. After locating the volume, just right click on it and select Configure Data Deduplication.

2. A wizard opens up where you can configure various parameters, such as minimum file age (which can be set to 0 to ignore age limits), configure file type exclusions, and setup the all important dedupe schedule.

3. The scheduler has several options, and lets you configure dual schedules. Perhaps one schedule for week days, and another schedule for the weekend.

4. After configuring the dedupe schedule, I came back to the system a day later (to let the scheduler kick in and my minimum file age of 1), and viola! 54% dedupe rate and saved 4.26GB.

You can do whole volume backups, restores, and even restore the volume to a different server and the dedupe database is preserved and the volume is automatically recognized and accessible. There are also APIs backup software can use to do even more intelligent backup/restore operations as well. A great space saving feature, that is fully compatible with VMs, vMotion, sDRS, etc.