vCenter 5.5 SSL Certificate and SQL Toolkit Updated

11-17-2013 7-03-32 PMFresh off the press is an updated version of my vCenter 5.5 SSL certificate Toolkit script. Last year when I did my popular vCenter 5.1 install series the posts contain a series of scripts and CLI commands to replace the SSL certificates. While that process worked for many people, it still was not as easy as it should be.

So for vCenter 5.5 I wrote a PowerShell script that did all the SSL certificate creation ‘magic’ in one place. In the intervening weeks since the first version went up, I’ve made a number of changes based on user feedback (and code submission) and my own development effort. I want to develop it further, but that will have to wait for a number of weeks while I complete a big project I’m working on. But for those that did download the first version and haven’t seen my Tweets about updates, I wanted a dedicated post to highlight the full feature set of v1.41 (November 10th).

The script is designed to be used in conjunction with the VMware vCenter certificate automation tool, NOT replace it. While that tool will create CSRs, I find it a bit cumbersome and does not help you in minting the certs. Regardless of what kind of CA you have, the script will help. The degree of automation varies, as the script is targeted for an online Microsoft CA. Once you use my tool to mint all of your certificates, then it’s a straight forward matter of using the VMware certificate tool to replace the self-signed certificates with your freshly minted ones.

As you will see in the feature list, the script goes beyond just SSL assistance and can also aid in your SQL database and DSN creation.

The script has the following features:

  • Downloads and installs the proper version of OpenSSL (0.9.8.Y) if it’s not already installed
  • Creates 2048 bit RSA private keys in the proper format
  • Creates a directory for each service bundle of SSL certificates
  • Generates ten OpenSSL configuration files, one for each certificate, in the appropriate directory
  • Creates certificates for AutoDeploy, Dump Collector and Syslog collector
  • Downloads both root and subordinate root public certificates
  • Submits the CSRs to the online CA and downloads the certificates
  • Creates the needed service PEM files for the vCenter certificate automation tool
  • Creates the required root/subordinate PEM files
  • Handles the special SSO 5.5 certificate requirements
  • Assumes all vCenter components are on one server
  • Automatically uses the hostname of the server you run the script on for all certificates
  • Creates a pre-filled vCenter Certificate Automation environment script – Just run!
  • Works with offline CAs
  • Creates SSO 5.5 certificate replacement files – Only used if manual replacing certs
  • Creates customized SQL vCenter and VUM database creation script
  • Creates SQL ODBC DSNs for vCenter and VUM
  • Automatically downloads and installs SQL 2008 R2 or SQL 2012 client package
  • Linux vCenter Server Appliance support for online minting and offline CSR creation
  • Creates certificates for Auto Deploy, Dump Collector and Syslog collector
  • Support Microsoft CAs that require manual certificate approval

On the potential roadmap is replacing the ESXi 5.x host certificates, and a bit more robust Linux VCSA support. A screenshot of the main menu is shown below.

As always you can download the latest version from: vExpert.me/toolkit55 It’s gotten over 1,500 downloads in the few weeks that its been available, which is great. Hopefully it is helping people install vCenter 5.5 and more easily configure trusted certificates. For instructions on how to use the tool and a change log, start in Part 8 of my vCenter 5.5 install series.

11-10-2013 5-29-56 PM

vSphere 5.5 Install Pt. 10: Replace SSO Certs

10-11-2013 6-44-21 PMJust like replacing a hip joint, replacing vCenter SSO SSL certificates can induce some pain, is a bit complex, and the outcome can be questionable. The replacement process in SSO 5.5 is pretty much like that in 5.1, but now we have the VMware certificate automation tool and my vCenter toolkit to make this a safer operation. Outcome is not guaranteed, and there may be side effects.

My approach to replacing SSL certificates in this series is replacement right after the service is installed. Basically you build up a trusted infrastructure from the get-go. VMware would likely say to build it up all untrusted, then go back and replace the certs. Certainly a valid point. If the VMware automation tool was more automated, then I might agree. But by building up trusted layers there are less replacement steps and chance of messing up, IMHO.

I do strongly recommend using the VMware certificate automation tool instead of manual replacement steps. But for the SSO service I’ll show you both ways, each using my vCenter Toolkit script to prepare the needed files.

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction 
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 

vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips 
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Replace SSO Certs
vSphere 5.5 Install Pt. 11: Install web client
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

Permalink to this series: vexpert.me/Derek55
Permalink to the Toolkit script: vexpert.me/toolkit55

Automated Certificate Replacement

1. Download the vCenter Certificate Automation Tool v5.5. You can find it under the Drivers and Tools section of the vSphere 5.5 downloads. Direct link is here. Unzip the contents on your vCenter server.

10-9-2013 6-10-08 PM

2. At the beginning of my toolkit script are two variables that set the SSO and vCenter administrator account names. Now during the SSO install you can’t change these names, so you shouldn’t need to change them here. But just FYI, I wanted to point out they are defined in the toolkit script.

10-9-2013 6-23-42 PM

3. Re-run my vCenter 5.5 Toolkit script, but this time we want to create the automation batch file. The menu number may change, but in today’s version of the script it is option 4.

10-9-2013 6-17-27 PM

4. The script will run in a split second and provide you the path to the batch file. The batch file sets the same variables as the stock VMware ssl-environment.bat file. I stripped out all of the comments and set the paths to where your certificate files are stored, assuming you used my script for part 8 or 9, minting your SSL certs.

5. Copy the ssl-environment.bat file that the toolkit created and overwrite the one in the VMware tool directory. You don’t need to run the batch file as the main updater script will execute it in the background.

10-9-2013 7-14-01 PM

6. Run the VMware ssl-updater.bat file. On the main menu select Option 3.

10-9-2013 6-27-30 PM

7. On the next menu select option 1.

10-9-2013 6-29-58 PM

8. When you select option 1 it will ask you a series of questions (in yellow below). Most of the information should be pre-populated for you. But you do need to input the administrator@vsphere.local password and answer if you are using a load balancer or not (don’t use one, VMware does not recommend it). Cross your fingers and toes, and watch for success messages.

10-9-2013 6-53-45 PM

9. Skip down to the verification section to validate that your certificate was replaced and that the service is in a healthy state. Let’s hope for no side-effects of this delicate operation.

Manual Replacement

If you for some reason you can’t use the VMware certificate tool (I recommend you DO use it, since it provides some certificate checking and is less error prone. But it could have issues that prevent you from using it. If so, then you can follow the steps below. They are directly from KB 2058519, with a couple of corrections (I’ve submitted fixes, so hopefully they will correct them).

Thanks to my vCenter 5.5 toolkit script 90% of the tedious work in that KB article is already done for you. Whoohoo. All we need to do is issue three commands to update the lookup service, then copy over our new certs. That’s it!

1. Open an elevated command prompt (not PowerShell) and enter the following command, replacing the vCenter FQDN, paths, and password as needed.

"C:\Program Files\VMware\Infrastructure\VMware\cis\vmware-sso\ssolscli" updateService -d https://d001vctr01.contoso.net:7444/lookupservice/sdk -u administrator@vsphere.local -p YourPassword -si D:\certs\vCenterSSO\gc_id -ip d:\certs\vCenterSSO\gc.properties

2. Enter the following command:

"C:\Program Files\VMware\Infrastructure\VMware\cis\vmware-sso\ssolscli" updateService -d https://d001vctr01.contoso.net:7444/lookupservice/sdk -u administrator@vsphere.local -p YourPassword -si D:\certs\vCenterSSO\admin_id -ip d:\certs\vCenterSSO\admin.properties

3. Enter the following command:

"C:\Program Files\VMware\Infrastructure\VMware\cis\vmware-sso\ssolscli" updateService -d https://d001vctr01.contoso.net:7444/lookupservice/sdk -u administrator@vsphere.local -p YourPassword -si D:\certs\vCenterSSO\sts_id -ip d:\certs\vCenterSSO\sts.properties

4. If all goes well then your screen should look similar to the one below, with three success messages. If not, you did something wrong. Depending on what’s goofed up, SSO may be in a hosed state and require a re-install.

10-9-2013 8-16-46 PMA

5. Now that the services have been updated, we need to overwrite some certificates. Navigate to the C:\ProgramData\VMware\CIS\runtime\VMwareSTS\conf directory. Backup the ssoserver.crt, ssoserver.key and ssoserver.p12 files. In the vCenterSSO directory that the toolkit script created, copy the ssoserver.crt, ssoserver.key and ssoserver.p12 files and overwrite the old versions.

6. In an elevated command prompt type:

net stop vmwarests
net start vmwarests

Verification

To verify that the SSO service is using the new certificate and didn’t suffer fatal stab wounds, open your favorite browser and go to the lookup service URL. That should be https://vCenterName:7444/lookupservice/sdk It should open without any SSL errors and if you look at the certificate by clicking on the lock icon, it will be issued by your CA. The XML response below is normal (yes even the Unexpected EOF), since we didn’t give the SDK service any input data.

10-9-2013 7-02-19 PM

10-9-2013 7-05-34 PM

Summary

With the help of my Toolkit script and the VMware automation tool, replacing the SSO certificate is not as painful as it was in the early months of vCenter 5.1. Even if you need to replace it manually, the toolkit does a lot of the tedious work to make success more likely. Next up in Part 11 is installing the Web Client, updating the SSL certificates, and configuring IE 10.

vSphere 5.5 Install Pt. 9: Offline SSL Minting

10-4-2013 6-19-17 PMNot everyone has an online Microsoft Certificate Authority, or maybe my toolkit script has issues in your environment. So in this installment we will go over manual SSL minting. By that I mean we will use my Toolkit script to create the CSRs, you will download the certificates yourself, then run my Toolkit script again to create all of the required files. So in reality the only manual process is getting the certificate.

Even if you don’t have an online Microsoft CA, I suggest reading through Part 8. It will familiarize you with my vCenter 5.5 Toolkit script and has the change log. If have an online Microsoft CA and ran the script in the previous post then you can skip this installment and go to Part 10 (coming soon).

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction 
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 

vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips 
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Replace SSO Certificates
vSphere 5.5 Install Pt. 11: Install Web Client 
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

Permalink to this series: vexpert.me/Derek55
Permalink to the Toolkit script: vexpert.me/toolkit55

Offline SSL Method

1. Download my vCenter 5.5 toolkit script from the link above. Open it in the PowerShell ISE (or favorite editor). The PowerShell script requires a few variable modifications before you run it. In the first block of variables you need to setup the directory where you want all of the certificates to go. If OpenSSL is already installed, change the path so the script knows where the root directory is. If that directory does not exist OpenSSL will be downloaded and installed for you. Next up are the certificate properties. Change those to suite your environment. If you want the server’s IP address in the SAN field, then uncomment the line and change the IP.

10-10-2013 7-04-44 PM

2. Execute the PowerShell Toolkit script. Unlike part 8 where we selected option 1 and everything was automated, here we need to select the option behind door number 2. This will create all of the required directories, private RSA keys and CSRs for you.

10-9-2013 4-52-21 PM

2. The first screenshot are the seven service directories which get automatically created. Inside each directory are three files. In the second screenshot the rui.key file is your private 2048 bit RSA key. The .cfg file is the OpenSSL configuration file that was used to generate the CSR. The .csr file is what you will submit to your CA.

10-4-2013 6-51-01 PM

10-4-2013 6-44-06 PM

3. Now you need to take each of the seven CSR files and submit it to your CA. In case you have an offline Microsoft CA or there are strong security measures in place so the vCenter can’t access your CA directly, I’ll cover the manual issuing and downloading process with a Microsoft Windows Server 2012 CA. If you have a non-Microsoft CA, then just skim over the Microsoft CA section, save your certificates as rui.crt in each directory, and pick back up at step 8.

4. Go to the URL of your Microsoft CA. The default address is https://hostname/certsrv. Make sure you are accessing the CA page with credentials that can request VMware-SSL certificates. Click on Request a certificate.

10-4-2013 7-00-57 PM

5. Select the second option, Submit a certificate request by using a base-64-encoded….

10-4-2013 7-03-33 PM

6. Copy and paste the CSR information from the first service into the top pane. Make sure the VMware-SSL template is selected. If that template is NOT listed then you probably goofed up one of three things 1) You accessing the CA web site with your non-admin account 2) You didn’t properly publish the VMware-SSL certificate template 3) You don’t have enroll permissions on the VMware-SSL template. Do not enter any additional attributes.

10-4-2013 7-05-27 PM

7. After you submit the certificate request you need to download the Base-64 encoded version WITHOUT the certificate chain. Name the file rui.crt and save it back into the same service directory that you submitted the CSR from. These certificates are NOT interchangeable, so don’t get the rui.crt files mixed up. The system will barf later on and you will lose some hair. Each certificate must match the service it was intended for.

10-4-2013 7-09-29 PM

8. After you’ve done this for all seven certificates, each service directory should now look like the following, with a rui.crt file now present.

10-4-2013 7-15-23 PM

9. Next up we need to create one or two root CA files, depending on your CA architecture. Double click on one of your .crt files and go to the Certification Path tab. In my example below we have two CAs: A root and a subordinate. The CA at the top is the root and the next one down is the subordinate. vCenter needs the public certificate from both, so that it can properly chain.

10-4-2013 7-17-20 PM

10. If you are using a Microsoft CA then go back to the Home page of the CA. But this time select the last option, Download a CA certificate…

10-4-2013 7-22-02 PM

11. Click on Download CA certificate chain if you have a Root/subordinate CA architecture. If you have just a root CA click on Download CA Certificate. If you are downloading the chain, just save it to your desktop with any ole name and skip to step 12. If you have just a root CA, then save the file as Root64.cer in the root of your certificate directory (screenshot below).

10-4-2013 7-23-33 PM

Root only CA:

10-4-2013 7-40-48 PM

12. For those that downloaded their chain (and ball), double click the certificate and locate the two certificates. Right click on your ROOT (see step 9), select All Tasks, and Export. Save the certificate as a Base-64 encoded file and name it Root64.cer. Put it in the root of your certificate directory as show in step 11.

10-4-2013 7-37-28 PM

13. Repeat the process on the subordinate CA, but save the file as interm64.cer. You should now have a directory that looks like:

10-4-2013 7-47-04 PM

13.  Re-run the Toolkit script but now we select Option 3. This will process all of the files and create the exact same output as the online option in Part 8. Review the screen events for any errors.

10-10-2013 7-31-55 PM

A sample of the screen output is below.

10-10-2013 8-12-40 PM

Output Validation

1. Assuming no errors occur, you should now see additional files in the root of your certificate directory. A chain.cer file should now appear if you have an intermediate CA. A hash file (which ends in 0) for each root certificate will also be listed.  If you only have a root CA then you will have one hash file.

10-9-2013 5-05-54 PM

2. If you take a peek inside one of the folders you will see a series of files. Each service, except SSO, will have the same set of files (except the .csr and .cfg with are uniquely named). The

  • chain.pem: Used for the VMware vCenter certificate automation tool
  • rui.crt: Public half of your SSL certificate
  • rui.key: Private half of your SSL certificate
  • rui.pfx: Combined private and public SSL keys
  • *.cfg:  Certificate signing request file
  • *.csr: Certificate signing request

10-9-2013 5-09-43 PM

3. In the vCenterSSO you will see a plethora of files. Depending on how you replace your SSL certificates, you may only use some of these files. But to help you out as much as possible, all the SSO files that are tedious to create manually are created for you. If you are missing files, then something went wrong. Please match up all filenames to validate the toolkit script worked. Some files are copies of each other, but they are needed to avoid confusion and more easily follow the KBs.

  • *.properties: Use for manual SSO SSL replacement
  • *_id: Use for manual SSO SSL replacement
  • ca_certificates.crt: Use for manual SSO SSL replacement
  • root-trust.jks: Used for SSO/STS certificate validation
  • server-identity.jks: Same file as above with a different name (per VMware KBs)
  • ssoserver.p12: Same functionality as rui.pfx, but VMware changed the name and format for SSO 5.5
  • ssoserver.crt: Copy of chain.pem
  • ssoserver.key: Copy of rui.key

10-9-2013 10-06-14 PM

Certificate Validation

Now that your certificates are minted, let’s quickly validate all of the properties are present. Even if your CSR requests a property, that doesn’t mean your CA will honor it. The OU in each subject name should be unique and match the directory it’s in.

10-10-2013 7-17-04 PM

The Subject Alternative Name should contain the short name and FQDN. Optionally it can contain your IP address too.

10-10-2013 7-18-18 PM

Enhanced key usage should show server and client authentication. Client authentication can be missing if the CA template is wrong.

10-10-2013 7-18-59 PM

Key usage should contain digital signature, key encipherment and data encipherment.

10-10-2013 7-19-43 PM

Summary

After a bit more work than the automated method, you now have all of the required certificate files to either use the vCenter certificate automation tool, or try the complex manual replacement method. Next up in Part 10 we update the SSO service SSL certificates.

vSphere 5.5 Install Pt. 8: Online SSL Minting

10-3-2013 8-04-58 PM

In the last installment of the vSphere 5.5 series we installed the SSO service. Now that the Java JRE is installed (via the SSO installer), we have the tools ready to create our vCenter SSL certificates via an online Microsoft CA. If you don’t have an online Microsoft CA that can issue your VMware SSL certificates, skip this section and go to Part 9 (coming soon), where we go through the manual process.

I’m recommending you create the SSL certificates now, so that you have a variety of methods at your disposal to use these certificates. VMware’s stance is that you fully install vCenter 5.5 with self-signed certificates then use their free certificate automation tool to replace all of the certificates as the last step. That is certainly a good route to go, and I would not dissuade you from that method. Or, you could also replace certificates as you install components so each layer is trusted as you go. Either way, I recommend the VMware certificate tool even if it is a bit primitive. It is flexible enough to let you incrementally replace certificates.

In order to make life easier for installers I’ve written a “toolkit” PowerShell script to help with the SSL process. More details are below, but I would like to give credit to Chrissy LeMaire for some of the (modified) building blocks of this script. She wrote a vCenter 5.1 PowerShell SSL replacement script that was more automated than VMware’s batch script. My script does not replace VMware’s automation tool, but helps you prepare the files it needs.

Download Toolkit Here

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction 
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 

vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Update SSO Certificate
vSphere 5.5 Install Pt. 11: Install Web Client 
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

…possibly more to come…

Derek’s Toolkit Script

The PowerShell script performs several tasks and is menu driven. It’s an all in one script, meaning it handles online/offline CAs, and will also do other install tasks like create your ODBC connectors. That functionality is not yet in there, but will be added in the coming weeks. The full feature list will change and so will the menus. But I’ll try and keep this updated as often as I can.

The CSRs are in strict accordance with VMware KB articles regarding certificate requirements, including an optional IP address in the SAN field. I want to strictly color within the KB article lines, so if you do use this script and then have to call up VMware support they won’t roll their eyes and have you re-do your certs because some blogger got it wrong.

The script has the following features:

  • Downloads and installs the proper version of OpenSSL if it’s not already installed
  • Creates 2048 bit RSA private keys in the proper format
  • Creates a directory for each service bundle of SSL certificates
  • Generates seven OpenSSL configuration files, one for each certificate, in the appropriate directory
  • Downloads both root and subordinate root public certificates
  • Submits the CSRs to the online CA and downloads the certificates
  • Creates the needed service PEM files for the vCenter certificate automation tool
  • Creates the required root/subordinate PEM files
  • Handles the special SSO 5.5 certificate requirements
  • Does NOT require PowerCLI
  • Assumes all vCenter components are on one server
  • Automatically uses the hostname of the server you run the script on for all certificates
  • Creates a pre-filled vCenter Certificate Automation environment script – Just run!
  • Works with offline CAs
  • Creates SSO 5.5 certificate replacement files – Only used if manual replacing certs
  • Creates customized SQL vCenter and VUM database creation script
  • Creates SQL ODBC DSNs for vCenter and VUM
  • Automatically downloads and installs SQL 2008 R2 or SQL 2012 client package
  • Linux vCenter Server Appliance support for online minting and offline CSR creation
  • Creates certificates for Auto Deploy, Dump Collector, Syslog collector, Authentication Proxy
  • Support Microsoft CAs that require manual certificate approval
  • Requires PowerShell 3.0 or higher

Download Toolkit Here

Toolkit Change Log

v1.56, January 19, 2014

  • Fixed bug when no subordinate CA was present
  • Changed Microsoft “renewal” default to 0 for root/sub CA

v1.55, January 12, 2014

  • Added additional CA/subordinate error checking

v1.50, December 22, 2013 Changes:

  • Added support for ESXi hosts

v1.42, December 3, 2013 Changes:

  • Modified how the certificate hash files are created
  • Added Authentication Proxy certificate creation
  • Changed MS CA download parameter Renewal from 0 to 1

v1.41, Nov 14, 2013 Changes:

  • Changed the root/intermediate CA download order and added more error checking

v1.4, Nov 10, 2013 Changes:

  • Added Auto Deploy, Dump Collector and Syslog Collector SSL certificates
  • Added support for manually approved CA certificates (Windows & Linux)
  • Added SHA512 request in CSR
  • Added request for vCenter FQDN in Option 3

Special thanks to Ryan Bolger from Trace3 for the CA manual approval code.

 

Online SSL Minting

1. Download my vCenter 5.5 toolkit script from the links above. Open it in the PowerShell ISE (or favorite editor). The PowerShell script requires a few variable modifications before you run it. In the first block of variables you need to setup the directory where you want all of the certificates to go. If OpenSSL is already installed, change the path so the script knows where the root directory is. If that directory does not exist OpenSSL will be downloaded and installed for you. Next up are the certificate properties. Change those to suite your environment. If you want the server’s IP address in the SAN field, then uncomment the line and change the IP.

10-10-2013 7-04-44 PM

2. The script is semi-intelligent about using only a root, or one subordinate and root. Simply comment out $SubCA with a # if you only have an online Microsoft root CA. If you have two or more subordinates, then you will need to follow VMware SSL KBs or modify my script. Sorry!

10-7-2013 8-17-46 PM

If for some reason the script can’t download your CA certs automatically, try changing the download protocol. The default is HTTPS. If that still fails, you can manually place the Base-64 encoded root (and intermediate) certificates into your $Cert_Dir path. The root should be called Root64.cer and the intermediate called interm64.cer.

10-10-2013 7-08-34 PM

10-10-2013 7-12-20 PM

3. The next section are the details for your issuing CA and the template. The issuing CA is your online CA that will actually mint your certificates. If you only have one CA, then clearly that is what you should use. The $ISSUING_CA field can be a little tricky. The first field is the shortname (or FQDN) of your CA (e.g. d001dc01). Next up is your CA name. This can be anything, so you must open the Certificate Authority MMC on your CA to find out what it’s called. As you can see from my screenshot below my CA name is contoso-D001DC01-CA.

10-3-2013 9-06-22 PM

10-9-2013 4-35-46 PM

Now if you don’t have MMC access on the CA to look up the actual CA name, then you can find it another way. Open a browser and go to https://yourCAserver.domain.com/certsrv and select Download a CA certificate, certificate chain, or CRL. Select Download CA Certificate Chain. Open the files, then open the certificate(s) in the file. Now depending on how your CAs are setup, it may take a little thought to correlate the “Issued to” and “Issued by” to your root/subordinate CA. In my case contoso-D001DC02-CA is my root CA, and contoso-D001DC01-CA is my subordinate. I want my VMware certs issued from the subordinate, so just like the MMC screenshot above, the CA name that corresponds to D001DC01 is contoso-D001DC01-CA.

12-3-2013 6-43-46 PM

4. Next up is the template name. This can also be any value, but if you followed my guide then it will be called VMware-SSL. This is the Template Name not the Template display name.

10-3-2013 9-08-57 PM

10-9-2013 4-37-42 PM

5.  Your account must have the required CA permissions to enroll for the VMware-SSL template. If it does not, then find a CA administrator, have them logon the vCenter server and run the script. If your online Microsoft CA is configured for manual certificate approval, you can still use Option #1.

1-9-2014 9-01-08 PM

6. Since you have properly configured the script variables and have one or more online Microsoft CAs, you should select option 1. New to v1.3 and later it will ask you to confirm your vCenter FQDN. If the FQDN is correct, just press ENTER. If it is wrong, just enter the right FQDN.

10-22-2013 8-30-54 PM

After confirming/entering your FQDN the process is automated. If you do get errors, then you either goofed up the variables, have insufficient permissions, or my script is broke and needs fixing. If its broke, now is an excellent time to learn PowerShell. Script is provided as-is, and bugs/issues may or may not be fixed.

Below is a screenshot with a sample of the script output as it runs using an online CA with automatic approval. A lot more has scrolled off the screen, but you get the idea. There is limited error checking, but subtle issues could fly by on the screen. Review the output for any issues.

10-9-2013 5-00-02 PM

7. If your CA is configured for manual approval you will get a list of the request IDs for all 10 certificates. Have your CA administrator approve the requests, then run Option #3 to complete the process. You will only see the Request IDs when manual request approval is required. When you run Option #3 the output should look like the previous screenshot.

11-10-2013 7-10-23 PM

Output Validation

1. When the script completes you should have eleven directories, and either one or three certificate (.cer) files in the root of your working directory. If you have a subordinate CA then you will have three files. If you have a single CA you will only have a Root64.cer file. The two files with funny names are hash files of the root and intermediate CAs. If you only have a root CA you will see a single hash file.

12-3-2013 6-52-23 PM

2. If you take a peek inside one of the folders you will see a series of files. Each service, except SSO, will have the same set of files (except the .csr and .cfg with are uniquely named). The

  • chain.pem: Used for the VMware vCenter certificate automation tool
  • rui.crt: Public half of your SSL certificate
  • rui.key: Private half of your SSL certificate
  • rui.pfx: Combined private and public SSL keys
  • *.cfg:  Certificate signing request file
  • *.csr: Certificate signing request

10-9-2013 5-09-43 PM

3. In the vCenterSSO you will see a plethora of files. Depending on how you replace your SSL certificates, you may only use some of these files. But to help you out as much as possible, all the SSO files that are tedious to create manually are created for you. If you are missing files, then something went wrong. Please match up all filenames to validate the toolkit script worked. Some files are copies of each other, but they are needed to avoid confusion and more easily follow the KBs.

  • *.properties: Use for manual SSO SSL replacement
  • *_id: Use for manual SSO SSL replacement
  • ca_certificates.crt: Use for manual SSO SSL replacement
  • root-trust.jks: Used for SSO/STS certificate validation
  • server-identity.jks: Same file as above with a different name (per VMware KBs)
  • ssoserver.p12: Same functionality as rui.pfx, but VMware changed the name and format for SSO 5.5
  • ssoserver.crt: Copy of chain.pem
  • ssoserver.key: Copy of rui.key

10-9-2013 10-06-14 PM

Certificate Validation

Now that your certificates are minted, let’s quickly validate all of the properties are present. Some users have reported corrupted/incorrect root and subordinate certificates, so please do NOT skip this section. Also, even if your CSR requests a property (such as client authentication), that doesn’t mean your CA will honor it. The OU in each subject name should be unique and match the directory its in.

10-10-2013 7-17-04 PM

The Subject Alternative Name should contain the short name and FQDN. Optionally it can contain your IP address too.

10-10-2013 7-18-18 PM

Enhanced key usage should show server and client authentication. Client authentication can be missing if the CA template is wrong.

10-10-2013 7-18-59 PM

Key usage should contain digital signature, key encipherment and data encipherment.

10-10-2013 7-19-43 PM

We also need to validate that the root and intermediate certificates are in the right format. Some users have reported corrupted root or intermediate certificates. In Notepad open both your root64.cer and interm64.cer files. They should both start with —–BEGIN CERTIFICATE—–. If they contain HTML or any non-printable characters, then they are corrupted or in the wrong format. STOP, as they will most certainly NOT work.

1-9-2014 8-52-00 PM

If your root or subordinate CA certs contain HTML, and you are using the script to automatically download them from a Microsoft CA, I have a suggestion that may work. Locate the code snippet below and change Renewal=1 to Renewal=0. That tweaks how the script downloads the certificate from the Microsoft CA. In addition, if the script downloads an old subordinate certificate, change renewal=0 to renewal=1 in the DownloadSub function.

1-9-2014 8-55-17 PM

Summary

Assuming you have an online Microsoft CA and you were successful in running the script, you now have all of the files needed to use the VMware certificate automation tool, or go through the manual certificate replacement process. In Part 9 I cover how to use an offline or non-Microsoft CA. At the end of that article you will have exactly the same files as you do from this installment. Starting in Part 10 we will resume our configuration and installation of vCenter components.

vSphere 5.5 Install Pt. 6: Certificate Template

10-5-2013 3-51-55 PMNow that you understand what type of SSL certificates you need and how many vCenter 5.5 requires, we need to create a Certificate Authority template that will mint proper certificates. You may very well be able to get away with the Microsoft “Web server” template, but it is missing a few properties that VMware still lists as a requirement. So to ensure you don’t run into any problems, this installment shows you how to setup those properties.

I’m assuming you are using a Microsoft CA for this exercise. Technically you can use any CA, so don’t think that you are just limited to Microsoft’s implementation. Certificates are standardized in the X.509 format. In a real enterprise environment CAs should be heavily locked down and you probably won’t have permissions to change anything on the CA. Find your CA administrator and have them complete this section. If you aren’t using a Microsoft CA, then the steps below won’t exactly apply to you. But research how to configure your CA for the “required” properties.

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 
vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Update SSO Certificate
vSphere 5.5 Install Pt. 11: Install Web Client 
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

Permalink to this series: vexpert.me/Derek55
Permalink to the Toolkit script: vexpert.me/toolkit55

Certificate Template

1. Logon to your Microsoft root CA. In this case I’m using Windows Server 2012. Launch the Certification Authority console. I’ve already created a custom template, VMware-SSL. But ignore that for now (just like a cooking show I have my template already done in the oven) and locate the Certificate Template container. Right click and select Manage.

10-2-2013 7-17-46 PM

2. Locate the Web Server template, right click and duplicate it.

10-2-2013 7-19-36 PM

3. Don’t change anything on the compatibility tab. Don’t think you are clever and try changing the default value to something like Windows Server 2012. #Fail. On the General tab rename the template. I like using VMware-SSL because it has no spaces, so the template name and display name are the same. This avoids confusion down the road where a script requires the template name as a parameter. Spaces are allowed, but let’s not confuse the situation anymore than needed…we are already confused enough.

10-2-2013 7-22-52 PM

4. Click on the Extensions tab then highlight Application Policies. Click Edit and add Client Authentication.

10-2-2013 7-26-23 PM

5. Click on Key Usage and check the box to allow encryption of user data. Close out of all the certificate properties windows.

10-2-2013 7-28-40 PM

6. Back in the CA window issue the new VMware-SSL template, by selecting the menu item shown below. A list of available templates will appear, and just click on VMware-SSL. It should now appear in the right pane, as you can see below. Sometimes CAs can be slow, and it could take a couple of minutes to appear. Do not panic; be patient. Once it appears you now have a good template to use for VMware certificates (vCenter, ESXi hosts, etc.).

10-2-2013 7-31-03 PM

Summary

Creating a certificate template is not tricky and only takes a couple of minutes. It may take a few minutes for the new certificate type to replicate in AD. So don’t be too surprised if you can’t immediately see it. The steps are pretty much the same on Windows Server 2008 and later, so don’t worry if you aren’t yet using Windows Server 2012.

In Part 7 we (finally) get to mount the vCenter 5.5 ISO and install the SSO service. So yes, this install series is finally getting to the point were we can install something. But hopefully you are better educated about vCenter 5.5 than you were before you stumbled on this series. Impress your friends at your next cocktail party about SSL OU values and PEM files.

vSphere 5.5 Install Pt. 5: SSL Deep Dive

SSL certificateNow that you’ve read up on upgrade best practices and SSO enhancements in the first four installments, we can get to the fun stuff! Understanding vCenter 5.5 SSL certificates. Ok maybe not fun for all, but fun to me. Last year in my 5.1 install series I just plowed right into the minting process, but didn’t really explain the big SSL picture. Since SSL is probably the biggest point of confusion in vCenter 5.1 and later, this post will serve to illuminate one of my favorite subjects. After all I have to keep up my “SSL guy” nickname. Do not fear, this series will eventually get to actually mounting the vCenter ISO and doing a software install. But I hope these foundational pre-install articles enable you to better understand the product.

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 
vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Update SSO Certificate
vSphere 5.5 Install Pt. 11: Install Web Client 
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

Permalink to this series: vexpert.me/Derek55
Permalink to the Toolkit script: vexpert.me/toolkit55

Who cares about SSL?

Why should you go through the headaches of replacing all the VMware self-signed certificates? What’s the risk of using untrusted certificates? What can happen if the SSL connection is compromised?

Hypervisors are likely the underpinnings of your business critical apps and intellectual property. If your hypervisor is compromised then it’s just a few short commands to access your critical business data. Unless you like your infrastructure being p0wn3d, then you don’t want your VMware infrastructure compromised. If you don’t use trusted certificates, and just click through all the VI client SSL warnings (you have clicked Ignore and trust this certificate many times…haven’t you?) then you won’t know that a man-in-the-middle attack has taken place.

A man-in-the-middle attack is where a third party intercepts your “secure” communications and relays data between you, the attacker’s device, and the end host (like an ESXi server). This can be accomplished by ARP spoofing, or other means. If it’s second nature to ignore and click through all VI certificate warnings, you will have no idea your credentials have been intercepted….in clear text. No fancy brute force decryption required. Just sit back, grab a coke, and enjoy cleartext flowing across your screen. An interesting article on attacking VMware is here.

The diagram below shows the man-in-the-middle attack, and the following graphic is the clear text credentials from the VI client. With those credentials the bad guys can now own your infrastructure.

10-2-2013 9-04-19 PM

10-2-2013 9-23-20 PM

If you use trusted SSL certificates and untrain your staff to ignore SSL warnings, then it’s significantly harder to spoof the communications.  Think of this this way: Would you enter your credit card details or social security number on a site that threw up constant SSL warnings? No. So why trust potentially billions of dollars of IP to untrusted certificates that your grandma could spoof. There’s a reason why financial institutions, Government and other high value targets require real certificates. Don’t think that just because you run Nana’s Cookie shop that the bad guys aren’t after you…they are.

To be clear, man-in-the-middle attacks and SSL certificate spoofing are not unique to VMware or because they did something wrong. It’s a problem across the industry with self-signed certificates. I hope you are sufficiently scared to take certificates seriously and replace them. Even if you bake cookies in your garage and sell them on the corner stand, I’d update your SSL certs.

Certificate Requirements

Last year with vCenter 5.1 there was some confusion around certificate requirements. Ok, A LOT of confusion. How many certs do you need? What CA template options? What is a certificate OU?  I was stumbling around in the dark last October since official guidance at that time was a wee bit skimpy. Thankfully that situation has been largely remedied by VMware. The list below is a combination of the vCenter Certificate automation tool requirements and of the certificates themselves:

  • Platform Support: Windows Server 2008 R2 SP1 or later, including Windows Server 2012 (not R2)
  • Private Key Algorithm: RSA with 1024 bits or higher (I use 2048). No fancy elliptical curve support.
  • Recommended Signature Algorithms: SHA256, SHA384, SHA512
  • NOT Recommended algorithms: MD2, MD5, SHA1
  • Path or certificate filename can’t contain: ^ (caret), % (percent), & (ampersand), ; (semicolon), ) (closing parenthesis)
  • Use PEM certificate format, with a header of —–BEGIN CERTIFICATE——
  • Must use a complete certificate chain
  • Use OpenSSL v0.9.8y (Do NOT use OpenSSL 1.x !!). Recommend 32-bit version.
  • Does NOT support wildcard certificates
  • Have the subject alternative name (SAN) field included in them
  • Have unique OrganizationalUnitNames for the components (which in turn creates a unique Distinguished Name (DN))
  • Key Usage: Include digitalSignature, keyEncipherment, dataEncipherment
  • Seven unique certificates: InventoryService, SSO, UpdateManager, vCenter, WebClient, LogBrowser, Orchestrator

As you can see, there are a number of very specific requirements to meet in order to have success with replacing your vCenter SSL certificates. These are essentially unchanged from vSphere 5.1, but are now documented. Thank you VMware! Official sources of the requirements are here and here, if you have a burning desire to read more…before you fall asleep.

One final requirement that can trip people up is the format of the RSA private key file (rui.key). VMware states that only a key file with the —–BEGIN RSA PRIVATE KEY—- string will be accepted. Newer versions of OpenSSL may format it differently, creating problems down the road.

RSA private key

Certificate Subject Properties

One of the areas of confusion is exactly what certificate properties are important for a successful trusted certificate implementation. If we look at a SSL certificate which we will be generating in an upcoming post, we see a number of properties that may be Greek to most people.

In the example below we see properties common to all our SSL certificates. For example, we can see the signature hash algorithm is the recommended SHA256. You can also see validity dates, which are important. The primary point of confusion, and what took a lot of detective work when 5.1 went GA, is what needed to be in the Subject field. The SSO service in 5.1 and carried through to 5.5, requires a unique DN (distinguished name) for each certificate. The most common way to make the DN unique is to modify the OU value for each certificate. You could modify another field, but I would not recommend it. For all practical purposes the OU must be unique.

As you can see in the first certificate the OU value is vCenterSSO. In the second certificate the OU is VMwareUpdateManager.  While you don’t have to use these exact strings, they must be unique within the SSO certificate store. I’m opting go with the ‘official’ OU descriptions. Yes, even if you do the simple install with all components on a single VM, every component on that server must have a different certificate. You can NOT use wildcard certificates, either.

Commercial CAs can also have problems minting these properties. Although I’m not sure why you’d want to spend the money on a public CA certificate for an internal service. But some people try it…and are usually disappointed with the results.

vCenter SSO certificate VMware Update manager certificate

Certificate Key Usage

There also has been some lively discussion around other certificate properties, and what is and is not required. Certificate key usage defines how the OS or application can use the certificate. For example, can you only use it to digitally sign an email? Or only for code signing? Or just to secure RDP traffic? Those ‘features’ are enabled by Key Usage and Enhanced Key Usage fields.

For the purposes of this series I’m using KB 2037432 as a reference. As you can see in the graphic below, which is a VMware provided certificate request template, you see various fields. Highlighted are the KeyUsage and extendedKeyUsage fields. Of some controversy are whether dataEncipherment and clientAuth options are really required. Those are not properties normally allowed if you use the Microsoft “web server” template. For the purposes of this series I’ll assume they are required and will show you how to create a custom CA template to mint them. YMMV, and possibly the “web server” template will work fine for you. VMware must have a good reason to include them, so I’m not going to argue. My cooler is stocked with bigger VMware fish that need to be fried.

VMware certificate request

If you look at the properties of a properly formatted certificate you will see I used the correct template which enabled the minting of a certificate with said key usage requirements.

SSL data encipherment SSL authentication

Subject Names

Another important certificate property is the subject name and the subject alternative name (SAN). The subject name (commonName) is the FQDN of your vCenter server (assuming all components are on a single VM). Even if you are using the simplified deployment model all seven certificates will have the same commonName. The subjectAltName are alternative names that the host is known by. For those of you familiar with Microsoft Lync or Exchange deployments, you will be familiar with SANs.

There’s a little debate on what needs go to into the SAN field. Specifically, do you include the IP address or not. In the official VMware certificate template request they include the IP (shown below). However, at VMworld they said not to include the IP. Authentication uses Kerberos, which requires hostnames (can’t use IPs). Since the certificate replacement process is challenging, having to re-issue certs if the IP changes is not remotely appealing. My stance is to NOT include IPs in the certificate.

An interesting note I found in KB2037432 was that the FQDN in the SAN field was moved to the last value to support SRM. Curious that the order of SAN fields would matter, but apparently it does for SRM. Sounds like a bug to me.

SSL subject alternative name

Certificate Signing Request (CSR)

In order for a CA to mint you a certificate you must submit what is called a CSR, or certificate signing request, to your CA. This request contains all of the properties in the request template that you see above. It also contains data which is tied to the private RSA key. In our case we will use OpenSSL to process configuration files for each of the seven certificates, then submit the CSR to the CA, where we can then download our freshly minted certificate. The CSR will be in Base64 format, meaning it’s a neatly formatted text file you can open with Notepad and peek at. Although you will need to be Neo to decode it on the fly.

Certificate Authority Hierarchy

In many environments you will have more than one certificate authority. Typically an enterprise would have an offline root CA that is tightly controlled and secured. You then have one or more online issuing CAs which are joined to the domain and issue your certificates. Or, you may have a simplistic model where a single online CA does everything. If you have two or more tiers of CA, then you have some extra work cutout for you. Why? vSphere needs information about all tiers so that it can properly validate the chain of trust all the way to the root. Subsequent installments will show you exactly what extra steps are needed.

Certificate Creation Process

The certificate creation process is fairly simple once you understand the process. First, OpenSSL creates a private RSA key, usually 2048 bits long. That is used in conjunction with the certificate configuration file by OpenSSL to generate a CSR. The CSR is submitted to the CA, and out pops an SSL certificate at the other end. Nifty!

SSL certificate process

PEM Certificates

The vCenter Certificate automation tool requires certificate files to be in a very specific format. If they are not in that format (PEM), then you will have show-stopping problems and wonder why everything is a smoking pile of ashes. PEM encoded files are a series of Base64 certificates in a specific order. First up the tool needs the public certificates for your CA(s). If you have a subordinate CA, then this file (chain.cer) needs to have both the root and subordinate CA BASE64 certificates concatenated together. Do not add any spaces or lines, as that will not work. An example is shown below.

PEM certificate

In addition to the root CAs, each of the seven services needs a chain.pem file that is a combination of your root CA(s) and the certificate for your service. Again, don’t add spaces or lines in the file or you will run into problems.

PEM certificate

Don’t worry about creating the files now or where they go. Coming up is a PoweShell script that does all of the work for you. But if you are troubleshooting, then understanding the file format should help. I know I got tripped up on it last year and embarrassingly opened a support ticket. But I probably wasn’t the only one. Now you won’t have to be one!

Summary

Now that you hopefully understand a bit more about the certificate requirements, you will be in a better position to mint all of our SSL certificates. SSL certificates are complicated, and even in vSphere 5.5 they remain largely unchanged. Paths where VMware stores the certs are somewhat different, but the big overhaul that was needed did not happen. There are some cool ‘under the covers’ changes to certificates that I haven’t seen publically talked about yet which I’ll cover in the future. The teaser is that it sets the foundation for better certificate management in the future.

Next up in Part 6 is how to configure your Microsoft CA to issue SSL certificates with all the required properties.

VMware SSL pain? vCert Manager to the Rescue

VSS LabsOne of the most critical aspects to securing your VMware vSphere environment is replacing the self-signed certificates with trusted certificates. VSS Labs is releasing a new product, called vCert Manager, which will vastly improve the SSL certificate management experience for VMware customers.

Over the years I’ve written several articles detailing the process, which got exponentially harder with vSphere 5.1. I often hear users complain about the SSL certificate replacement process. Feedback on my blog has been quite verbose and colorful…some posts had to be censored. If you are struggling with the vSphere 5.1 installation, you can check out my vSphere 5.1 resources here.

While VMware did release the vCenter Certificate Automation tool earlier this year for usage with vCenter 5.1, it is command line/menu driven and does not fully automate the entire process. Given this tool was the only game in town I recommended my readers use the tool. You can find my multi-part usage series for the tool here.

The real game changer is the enterprise-scale VSS Labs vCert Manager. This goes way beyond the basic vCenter Certificate Automation tool, and provides full certificate lifecycle support for securing your vSphere environment with trusted SSL certificates. By full lifecycle I mean from automated vSphere infrastructure discovery, CSR generation, certificate minting, certificate installation, email expiration notices, automatic expired certificate updating, auditing, reporting, plus a lot more.

The entire process is managed via a web interface (securable with SSL), and requires no knowledge of OpenSSL, CSRs, PEM files, or leaf certificates. Plus it supports vSphere 4.x and 5.x environments, including ESXi hosts. The full support matrix is below.

If you are attending VMworld 2013 in San Francisco, be sure to stop by the New Innovator Pavilion and find the VSS Labs booth. Once you see the tool in action, you will be chomping at the bit to get a copy.

vCert Manager Features

Since the product has a very long feature list I’ve put together three tables that compare and contrast the features of the VSS Labs vCert Manager product and the VMware Certificate Automation Tool v1.01. The first table covers the VMware product/feature support of each tool. The second table covers certificate support, and the third table compares the tool core feature set and licensing.

VMware Product Support

VSS Labs VMware Support

As you can see from the table above, vCert Manager supports a broad range of VMware infrastructure. The VMware vCenter Certificate Automation tool is limited to just replacing vCenter 5.1 certificates. Version 1.01 has no support for prior versions of vCenter or ESXi hosts. Good tool for a specific use case, don’t get me wrong.

In contrast the vCert Manager product supports vSphere 4.0 through 5.1, including their respective ESX/ESXi host versions. I expect both companies to release updated certificate management products as new vSphere releases hit the streets. Neither tool has yet expanded to vCOPS, vCloud Director, or Horizon View. VSS Labs is targeting vCloud Director and Horizon View support in the near term, likely around the end of September.

Another great feature is that vCert Manager can manage dozens of vCenter instances and thousands of ESXi hosts from a single server and interface. No need to install the tool on each vCenter server. Credentials are server and host specific, so you can manage them all even if they are in different business units with different credentials.

Certificate Support

VSS Labs Certificate Support

Next to comprehensive VMware product support, broad certificate authority support and a high degree of automation is extremely important. This is where the vCert Manager product really shines. The VMware vCenter Certificate Automation tool requires a bit of manual pre-requisite work and using intermediate CAs can be tricky. Version 1.0.1 added CSR generation capability, so I certainly appreciate the added ease of use.

In contrast, vCert Manager automates the ENTIRE process, and you don’t have to know anything about certificate properties, SANs, how many certificates you need to mint, or how to submit and download certs from your CA. It can be configured to interface with an online Microsoft CA, so everything happens behind the scenes. Offline CAs are supported to, and as much of the process as possible is automated given the human intervention needed to mint the certificates.

In short, if you have an online CA the entire certificate request creation, minting, and deployment is executed through a GUI interface in an automated and orchestrated fashion. This makes certificate replacement almost foolproof.

Tool Features

VSS Labs vCert Manager Features

The most striking difference to me between the two tools is the user interface. vCert Manager sports a fully webserver driven GUI interface for all operations. If you haven’t used the VMware vCenter Certificate Automation tool you may not know that it’s completely command window based with text menus that instruct you in what order to perform the various steps. The VMware tool also needs to be installed on each server that has a vCenter component, unlike the vCert Manager which is installed on a single server and can manage thousands of vSphere components.

When updating ESXi hosts it requires DRS be enabled on the cluster so that it can put the host in maintenance mode, update the certificate, and return the host into production. This is all automated and when certificate is about to expire, it will automatically replace the certificates in the same manner.

Other great enterprise features of vCert Manager include syslog, full audit trail, basic reporting, and role-based access controls. Unlike the VMware tool, vCert Manager DOES require a Microsoft SQL database. So that’s a little extra configuration, but the installer does most of that work too. It supports Windows Authentication to the SQL database. SQL 2008, 2008 R2, and 2012 are supported.

Licensing and Support

There of course has to be a discussion about licensing and support. The VMware tool is free and if you have active SnS with VMware, they will support the tool for production usage. Clearly that’s a great value, and does ease much of the pain with vSphere 5.1 certificates. I certainly appreciate VMware releasing the tool, and the improvements in v1.01. I expect more improvements in the future.

vCert Manager is a licensed product, and support is provided by VSS Labs. The licensing model is per vCenter server, and per actively managed component (such as an ESXi host). For pricing details you will need to contact VSS Labs.

How to get a Free License

VSS Labs has a free NFR license available for home lab usage, with a special bonus for current fellow VMware vExperts. The tool will discover all of your components, even if they exceed your license, but you can only actively manage the number up to your licensed threshold.

8-19-2013 8-50-46 PM

Summary

vCert Manager should go GA by the end of August, if not sooner. When it does go live I’ll post a link to the download. I’ve been using the beta versions, and once the code goes GA I will post a multi-part blog post series on using the tool. So stay tuned!

For a version 1.0 product, I’m very excited to see the depth of support vCert Manager has packed into their first release. Throughout the beta process the company took feedback very seriously. After spending countless hours writing blog articles on VMware SSL certificates over the years, you have no idea how glad I am to see such a comprehensive tool. Maybe less blog articles for me to publish, but better for the community. Less hair pulling, and frustration for all!

The small home lab NFR licenses will give anyone interested in the product a good feel for how it works. I really appreciate the vExpert “bonus”, which will help the community test it against various vCenter versions more easily.

Sample Screenshots

The screenshots below are from a beta version, but will give you a little taste of the user interface and features. My installation series will contain a boatload of screenshots, once I get the GA code.

vCert Manager

vCert Manager Summary

vCert Manager Configuration

vCert Manager certificate authority

vCert Manager managed components

© 2017 - Sitemap