VMworld 2015: Certificates for Mere Mortals

Session INF4529

Note: Although not mentioned in this session, I have a SSL toolkit for vSphere 6.0 which makes the replacement process easier. Check out my vSphere 6.0 install guide here for all the details.

Certificate Lifecycle Management

  • VMCA: VMware certificate authority
  • VECS: VMware Endpoint Certificate store

VMCA

  • Dual Operational modes: Root CA and Issuer CA
  • Root CA: Automated, can issue other certs, all solutions and endpoint certificates are created and trusted to this root cert
  • Issuer CA: Can replace all default root CA certificate created during installation. Basically subordinate CA to your enterprise CA.

VECS

  • Repository for certificates and private keys
  • Mandatory component
  • Key stores: machine SSL certs, trusted roots, CRLs, solution users, others (e.g. VVOLS).
  • Managed through veccs-CLI
  • Does not manage SSO certificates

vSphere 6.0 Certificate Types

  • ESXi certificates – autogenerated post-install. New modes in 6.0, one of which can use VMCA certs. Can renew in webclient.
  • Machine SSL certificates – Creates server-side SSL (HTTPS, LDAP, etc.). Each node has its own machine SSL certificate.
  • Solution User certificates – Machine, vpxd, vpxd-extension, vsphere-webclient. Encapsulates one or more vCenter services.
  • Single-sign-on: Not stored in VECS. Stored in filesystem. STS certificate. Renew/update via GUI, not filesystem replacement.

Certificate Replacement Options

  • VMCA as root. Easiest deployment option.
  • VMCA as Enterprise CA subordinate – VMCA will issue certs on behalf of your enterprise CA
  • Custom CA – Only use custom certs all around. Not recommended except for Gov’t/Financial.
  • Hybrid – User facing certs replace, then let VMCA manage solution user and ESXi certs.

VMware vSphere 6.0 Certificate Manager

  • Available on both Windows and VCSA
  • Menu driven (GUI in 6.0 U1)

VMCA as Subordinate

  • RSA with 2048 bits
  • x.509v3
  • SHA256, 384 or 512
  • No wildcards in SubjectAltName
  • Cannot create subsidiary CAs of VMCA
  • Sync time for all nodes

Session videos, slides and scripts: http://vmware.com/go/inf4529

 

Ignite 2015: Encryption, Certificates and PKI

Session: BRK3130

Note: This was a great beginner level session for those not familiar with encryption, certificates or PKI. If you are in that boat, I would urge you to find the session video and watch the whole presentation. If you are a security professional and already know about these topics, then the content is probably too basic. I didn’t capture all the content below, but just took down some highlights what was covered.

Why am I here? Thanks to the NSA. Thanks to Edward Snowden. SharePoint, Lync, Exchange all  need to be secure.

Shows screens of RDP SSL warnings, and browser SSL warnings.

Are you still using passwords? Phishing and fraud, password fatigue, pass the hash attacks

IoT (Internet of things) is adding new concerns of authentication (connected cars, medical, industrial sensors)

Non-repudiation – Ability to bind a human to a digital document

Privacy – Hot topic over the last 2 years due to NSA and Snowden. Challenges are not new.

Encryption – Encryption at rest, in transit, challenges: weak algorithms

Encryption at rest – Bitlocker, EFS, SQL TDE

Encryption in transit – SSL/TLS, IPsec, Office 365 message encryption

Azure RMS – AD RMS for On-Premises. Protect documents from Birth to end of life. Protection regardless of location.

Speaker goes over symmetrical, asymmetrical encryption, hardware security modules (HSM) technologies such as AES and shows how they work.

What is hashing? Uniquely identify a stream of data. It’s a one way function.IMAG0425

Use the tool IIS Crypto to disable/enable and change the order that ciphers are use. FREE.

Good ideas: Remove RC4, reorder suites, Update to 2012 R2, research ECC vs. RSA

Talks about Certificate Authorities, certificates, and their basic properties. Also discusses path of trust, and where to find certificates in Windows.

CA Lifetime planning: End certs – 2 years, intermediate CA – 4 years, root CA – 8 years. Renew certificates when 50% of their life has expired.

S/MIME – For Email encryption and digital signatures

vSphere 6.0 Install Pt. 3: Certificate Management

Introduction

As long as I can recall certificate management in vSphere has been difficult, and for many customers, something they completely ignore. I’m surprised how many customer designs (even those done by VCDXs) I’ve seen where they feel it’s too difficult to deploy vSphere certificates so they accept the risk of using the non-trusted VMware provided certificates. While I don’t think untrusted SSL certificates are the biggest security threat to an enterprise, I do feel that using trusted certificates is the right thing to do and an add extra layer of security. If you work in a highly regulated industry like finance, healthcare or Government you may be mandated to use fully trusted certificate. Most of my career has been in the Government sector, so using trusted certs was not even a question and just a basic security requirement.

Starting in vSphere 5.1, SSL complexity really shot up and was pretty ‘cocked up’ to put it politely. In vSphere 5.5 VMware did address some of the complexity with a command line tool to help replace certificates. That was still complex, so I wrote the widely used vCenter 5.5 toolkit, which made the whole process super easy. Feedback on that effort has been super positive, and kept me motivated to do the same for 6.0. Now with vSphere 6.0 my toolkit script has to do less because VMware has made it easier, but I still want to make it even easier for customers. Fortunately or unfortunately, depending on how you look at it, vSphere 6.0 has new certificate management options which at first look make SSL more complex than in the past. We’ll dig into each option in this article.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Who cares about SSL?

Why should you go through the headaches of replacing all the VMware self-signed certificates? What’s the risk of using untrusted certificates? What can happen if the SSL connection is compromised?

Hypervisors are likely the underpinnings of your business critical apps and intellectual property. If your hypervisor is compromised then it’s just a few short commands to access your critical business data. Unless you like your infrastructure being p0wn3d, then you don’t want your VMware infrastructure compromised. If you don’t use trusted certificates, and just click through all the VI client SSL warnings (you have clicked Ignore and trust this certificate many times…haven’t you?) then you won’t know that a man-in-the-middle attack has taken place.

A man-in-the-middle attack is where a third party intercepts your “secure” communications and relays data between you, the attacker’s device, and the end host (like an ESXi server). This can be accomplished by ARP spoofing, or other means. If it’s second nature to ignore and click through all VI certificate warnings, you will have no idea your credentials have been intercepted….in clear text. No fancy brute force decryption required. Just sit back, grab a coke, and enjoy cleartext flowing across your screen. An interesting article on attacking VMware is here.

There are certainly many other ways to compromise your virtual infrastructure, like stealing the credentials of an administrator and gaining direct access to vCenter. Or using pass the hash, and gaining vCenter access that way. So ‘hacking’ SSL may not be the first choice for an attacker, but it is an attack vector you should consider and secure.

VMware Certificate Authority (VMCA)

This is a new an exciting component in vSphere 6.0 that will radically change how many will issue and deploy SSL certificates in their vSphere environment. SSL certificates are used extensively to secure communications in a vSphere environment. This ensures data confidentiality and integrity. Any attempt to modify data in transit is detected, such as man-in-the-middle attacks.

The VMCA is a built-in certificate authority, which is included in the Platform Services Controller (PSC) service. This is a full blown CA, and can (if you wish) automatically issue certificates to all vCenter 6.0 components and ESXi 6.0 hosts in your environment. The VMCA is mostly command line driven, and does not have a fancy GUI like your Microsoft CA has. But once configured, it’s pretty much a hands off operation. Do take note that VMCA in vSphere 6.0 does NOT support the use of CRLs nor does it have the concept of certificate revocation. If you suspect one certificate was compromised, first remove it then replace all certificates.

VMCA Intermediate Certificate Requirements

If you wish to use the VMCA as a subordinate CA to your existing enterprise CA, take note of the certificate requirements. The requirements are:

  • Private Key Algorithm: RSA with 2048 bits. No fancy elliptical curve support.
  • Recommended Signature Algorithms: SHA256, SHA384, or SHA512
  • NOT Recommended algorithms: MD2, MD5, SHA1
  • Key Usage: Root certificate extension set to true and cert sign must be in the list of requirements
  • Use PEM certificate format, with a header of —–BEGIN CERTIFICATE——
  • Does NOT support Wildcard certificates or more than one DNS name
  • Certificate must be X.509 v3

More about being a subordinate CA later in this article.

Certificate Deployment Options

VMware has come up with four primary certificate deployment options in vSphere 6.0. This is more than any previous release, where you basically had two (use VMware certs, or replace with trusted certs). You need to fully understand all four options, then pick for your environment which one best meets your business and security requirements. Depending on your industry, you may be severely limited in your choices. For example, if you are a U.S. Government agency you are stuck with option 3, using an external CA for all certificates and you won’t care about the new VMCA.

2015-02-07_9-42-07

#1 VMCA Root CA

Option #1 is the simplest option, and probably the one a lot of organizations will go with. This is relying on the new VMCA to provision and manage certificates for vCenter and ESXi hosts. The VMCA is automatically created upon PSC installation, and requires no further configuration. However, for services accessed by a web browser (such as the web client) you will get an SSL warning unless you explicitly trust the VMCA root in your browser of choice. This is akin to the VMware signed certificate method in years past. Except in 6.0 there’s now a central CA managing the certificates and their lifecycle. If you do nothing, this is what you will get. Better than in prior vSphere releases, but still not fully trusted certificates. For better security see option #2.

#2 Subordinate VMCA

This is an entirely new option in vSphere 6.0, and wasn’t remotely possible in prior releases. Basically what happens in this mode is that the VMCA imports a root signing certificate from you trusted enterprise root CA. The VMCA then becomes an official subordinate CA to your enterprise root(s). All the certificates issued by the VMCA are trusted by your organization, even the web services exposed in browsers. As you deploy new vSphere components that are VMCA aware, they will get issued trusted SSL certificates. Since the VMCA now manages ESXi 6.0 host certificates, your ESXi hosts will also be issued trusted certificates without any manual intervention.

The BIG downside to this, is also the big upside. The VMware CA is now issuing fully trusted certificates, which may go against company policy. Or, if you are in a regulated environment like the US DoD, there’s no way in hell they will allow you to stand up a trusted subordinate CA. So I would say this option is good for environments that want more than VMware issued certificates, but aren’t so regulated that a VMware subordinate CA would be strictly prohibited. Call this a good compromise between security and simplicity. Thank you VMware! For even more security see option #3.

2015-02-07_12-07-12

#3 External CA

Using an External CA is not new, and has always been in option in vSphere dating back many versions. It replaces all certificates in the environment by ones issued from the corporate trusted CAs. VMCA is bypassed, so this is a much more labor intensive process and much higher management complexity. This will be the only option in highly regulated environments, and will cause the most customer pain. All of the benefits of the new VMCA will be ignored, in favor of a higher security posture. This process is also totally different from that in vSphere 5.5, so get ready to learn yet another tedious procedure.

2015-02-07_12-08-23

#4 Hybrid

A hybrid scenario features the usage of the VMCA in combination with an external CA. For example, you could use the automated VMCA certificates for all “internal” certificates and ESXi hosts and only replace externally facing certificates (such as web client) with those from an external CA. This adds complexity to the VMCA subordinate option (#2), but is less work than using an external CA for all certificates.  Personally, I don’t see this solution being used too much. I think the other three will be more popular, and the level of regulation and security consciousness will ultimately determine which route to take.

Certificate Types

In the vSphere 5.x era each service was issued its own unique SSL certificate. As you may recall, each certificate had to contain a unique “OU” field otherwise SSO would barf. This does not scale well, as VMware is constantly adding new services to vCenter. Even in vSphere 5.5, my toolkit had to generate 11 certificates for all the services. Whew!

In vSphere 6.0 we now have several types of certificates. As shown in the VMware graphic below, a lot of services are use these ‘common’ certificates. This reduces the total number of certificates needed in the environment.

2015-02-07_11-38-36

The following table lists each of the certificate types used in vSphere 6.0, how they are provisioned, and where they are stored.

2014-11-22_13-36-40

ESXi Certificate: As has been the case for many years, this certificate is used by the ESXi host to encrypt nearly all communications. Nothing new here.

Machine SSL Certificate: Each node (embedded installation, management node, or PSC) has its own machine SSL certificate. All services running on this node use this certificate for end point encryption. The vCenter service (vpxd), VMware directory service (vmdir) also use these certificates.

Solution User Certificates: These certificates are used for authentication to the vCenter SSO service. Once the certificate is presented to SSO, SSO will issue a SAML token. The service, such as vpxd, can then use this token to authenticate to other services. Baseline solution user certificates include vpxd, vpxd-extensions, and vSphere-webclient.

VMware End Point Certificate Store (VECS)

The VMware End Point Certificate store (VECS) is a local repository for certificates and private keys. VECS is a mandatory component, and will be used even if you don’t sign your certificates with the VMCA. Remember that ESXi certificates are stored on the ESXi host and not in the VECS. The VECS includes a number of keystores including machine SSL certificates, trusted roots, CRLs, solution users (machine, vpxd, vpx-extension, vSphere-webclient) and other keystores such as those for vVols.

2015-02-07_12-03-29

Summary

Securing your virtual infrastructure is important. There are many attack vectors, and attacking SSL may not be the highest risk. But with vSphere 6.0 and my Toolkit script, replacing SSL certificates is easier than it used to be. So strongly consider taking the time to understand the new deployment methods, assess your business requirements, then take steps to secure your environment. It’s of little use to secure your SSL connections if you give the ESXi root and vCenter admin passwords to everyone.

Next up in the series will be vCenter upgrade and deployment best practices, in Part 4. You can check that out here.

VMware vSphere 5.5 Toolkit v1.58 Live

As many of you know, one of my passions throughout my IT career has been security. Having worked in the Federal Government space for most of my career, making sure solutions are secure is always a top priority. Securing your VMware infrastructure is very important, and one of the primary tasks is using trusted SSL certificates. So last year I wrote the vSphere 5.5 Toolkit PowerShell script, which has had over 9,000 downloads! I had no idea it would be so popular. Here’s a screenshot of the main menu:

vsphere 5.5 toolkit

Features of the SSL toolkit script include:

  • Downloads and installs the proper version of OpenSSL (0.9.8.za) if it’s not already installed
  • Creates 2048 bit RSA private keys in the proper format
  • Creates a directory for each service bundle of SSL certificates
  • Generates seven OpenSSL configuration files, one for each certificate, in the appropriate directory
  • Downloads both root and subordinate root public certificates
  • Submits the CSRs to the online CA and downloads the certificates
  • Creates the needed service PEM files for the vCenter certificate automation tool
  • Creates the required root/subordinate PEM files
  • Handles the special SSO 5.5 certificate requirements
  • Does NOT require PowerCLI
  • Assumes all vCenter components are on one server
  • Automatically uses the hostname of the server you run the script on for all certificates
  • Creates a pre-filled vCenter Certificate Automation environment script – Just run!
  • Works with offline CAs
  • Creates SSO 5.5 certificate replacement files – Only used if manual replacing certs
  • Creates customized SQL vCenter and VUM database creation script
  • Creates SQL ODBC DSNs for vCenter and VUM
  • Automatically downloads and installs SQL 2008 R2 or SQL 2012 client package
  • Linux vCenter Server Appliance support for online minting and offline CSR creation
  • Creates certificates for Auto Deploy, Dump Collector, Syslog collector, Authentication Proxy
  • Support Microsoft CAs that require manual certificate approval

I’ve now updated the script with some minor modifications for v1.58, dated July 12, 2014:

  • Updated OpenSSL download to 0.9.8za
  • Removed SQL 2012 SP1 client download (link broken)
  • Fixed Database creation script bug
  • Added additional error handling and Powershell-ized more commands
  • Changed the sts.properties file to use sts in the URI per KB2058519

These are incremental updates, and the base functionality has remained the same. I am hoping for vSphere v.Next that VMware will streamline the whole process and give SSL replacement a makeover. I have no idea if this is in the works or not.

As always, you can download the latest version of the toolkit script from: vExpert.me/toolkit55 If you are using an older version I suggest you grab the latest copy. If you want full SSL lifecycle management and a paid solution, I recommend you check out the VSS Labs vCert Manager, which you can find out about here.

Also remember to check out my 20 part vSphere 5.5 series, which covers the usage of the toolkit script and a whole lot more. You can find that series at: vExpert.me/Derek55

vCenter Certificate Automation Tool: Part 4 (Web Client and Log Browser)

Continuing on from Part 3 of the VMware vCenter Certificate Automation Tool series, we are now ready to replace the Web Client and Log Browser SSL certificates. If you want to start at the beginning, check out Part 1.

1. Pressing 5 takes us back to the main menu. Now we press 6 to enter the web client and log browser update process. Pre the pre-planning guide we need option 1. I enter the SSO administrator username and password.

Several minutes later the process was a success.

Step 11 of the pre-planning guide is complete. Check!

2. Now we need to press 2, to trust the inventory service.

Several minutes later the process was a success.

Step 12 of the pre-planning guide is complete. Check!

3. Now we need to press 3, to trust the vCenter server.

Step 13 of the pre-planning guide is complete. Check!

4. Now we need to press 4, to update the web client SSL certificate. Again, the presented paths and files were correct. Enter the SSO administrator username and password.

Step 14 of the pre-planning guide is complete. Check!

5. Next up is pressing 5, to enable the log browser service to trust SSO.

Step 15 of the pre-planning guide is complete. Check!

6. Now press 6, to update the log browser SSL certificate. Again, the certificate and paths looked good. Enter the SSO username and password.

Step 16 of the pre-planning guide is complete. Check!

At this point, since I’m using the vCenter FQDN for the VUM configuration, I am not able to use the v1.0 of this tool to update the VUM certificates. You can check out Part 12 of my vSphere 5.1 Install series for the manual method to update the VUM SSL certificate.

7. To validate some of the certificates I launch the vSphere web client. Using my web browser I view the SSL certificate and validate that my new certificate is being used. I also open the log browser and pull down the logs from an ESXi host to verify that works as well.

Minus the VUM “known issue”, the tool worked flawlessly for me and certainly helped ease the SSL burden. I’m hoping future versions of the tool have the following enhancements:

  • Automated execution of the pre-planning steps, so I don’t have to keep referring back the 18 step list and checking off each one (assuming an all-in-one server).
  • Ability to create CSRs, submit to a Microsoft online CA, and download the certificates.
  • Ability to create CSRs for an offline/commercial CA, and use the resulting certs
  • Automatically build and verify the CA chain files, to reduce human error and confusion
  • Provide a full GUI with detailed logging, to make the processes even easier
  • Perform full certificate validation to ensure unique DNs
  • Fix the VUM FQDN “known issue”
  • Back-port the tool to work with vSphere 5.0
  • Perform SQL database password validation
  • Cache in memory all required passwords (flushed upon error or exiting)
  • Configure all ODBC/JDBC SQL connection strings to use SSL (if SQL supports SSL)

I think the tool is a decent first stab at helping with the SSL configuration nightmare that 5.1 unleashed on the community. The process could be more fully automated, so I hope future versions can improve on this useful utility.

vCenter Certificate Automation Tool: Part 3 (vCenter and Orchestrator)

Continuing from Part 2 of my VMware vCenter Certificate Automation tool series, we are now ready to replace the vCenter server and vCenter Orchestrator certificates. If you want to start at the beginning, check out Part 1.

1. Per the pre-planning guide step 4 I exit back to the main menu by pressing 5, then press 4. vCenter needs to trust the SSO certificate, so I press 1. The default path and file are correct, so I press enter. Success!

Step 4 of the pre-planning guide is complete. Check!

2. From the same menu I press 2, to update the vCenter SSL certificate. Again, the default paths and files were correct so I accepted them. Now I’m prompted for the vCenter administrator name and password. Next I’m asked to enter the original vCenter server database password, with all kinds of scary warnings if I input the wrong password since no validation is done. I’m also asked to enter the SSO administrator username and password.

After several minutes of chugging away I see a successful message.

Step 5 of the pre-planning guide is complete. Check!

3. Per the pre-planning guide I now must select option 3, to trust the inventory service SSL certificate.

Step 6 of the pre-planning guide is complete. Check!

4. Pressing 5 I get back to the main menu. And I need to go back into the inventory service, so I press 3.  Finally, we now configure the inventory service to trust vCenter by pressing 2.

Step 7 of the pre-planning guide is complete. Check!

5. Pressing 5 I get back to the main menu. I now press 5, to update vCO. Per the pre-planning guide I need to configure vCO to trust SSO, so I press 1. The default SSO filename is correct so I press enter.

Step 8 of the pre-planning guide is complete. Check!

6. Now vCO needs to be told to trust vCenter server, so I press 2 and validate the path is right.

Step 9 of the pre-planning guide is complete. Check!

7. Next up is updating the vCO SSL certificate, so I press 3 and validate the path.

Step 10 of the pre-planning guide is complete. Check!

Check out Part 4 where we update the Web Client and Log Browser SSL certificates.

vCenter Certificate Automation Tool: Part 2 (SSO and Inventory)

Continuing from Part 1 of my VMware vCenter Certificate Automation tool, we are finally at the point where we can review what the built-in planner advises we do, and then replace our certificates. If you missed Part 1, go back and execute all of the steps or you have a better chance of a pig flying by your window and waiving at you than getting new SSL certificates working.

1. In case things go Tango Uniform, I strongly urge you do a full backup of all vCenter databases (SSO, vCenter, and VUM), plus snapshot/backup your vCenter VM(s). If you hose up the certificate replacement process you may be left with a smoking vCenter hole. Backup before proceeding!

2. On your vCenter server run the ssl-updater.bat script. They have a built-in planner which tells you which steps to perform and in what order, depending on what services you want to update. To access the planner type 1.

3. Since we want to update all our services, I pressed 8.

The result of pressing 8, was the following text:

1. Go to the machine with Single Sign-On installed and – Update the Single Sign-On SSL certificate.
2. Go to the machine with Inventory Service installed and – Update Inventory Service trust to Single Sign-On.
3. Go to the machine with Inventory Service installed and – Update the Inventory  Service SSL certificate.
4. Go to the machine with vCenter Server installed and – Update vCenter Server trust to Single Sign-On.
5. Go to the machine with vCenter Server installed and – Update the vCenter Server SSL certificate.
6. Go to the machine with vCenter Server installed and – Update vCenter Server trust to Inventory Service.
7. Go to the machine with Inventory Service installed and – Update the Inventory  Service trust to vCenter Server.
8. Go to the machine with vCenter Orchestrator installed and – Update vCenter Or chestrator trust to Single Sign-On.
9. Go to the machine with vCenter Orchestrator installed and – Update vCenter Or chestrator trust to vCenter Server.
10. Go to the machine with vCenter Orchestrator installed and – Update the vCenter Orchestrator SSL certificate.
11. Go to the machine with vSphere Web Client installed and – Update vSphere Web  Client trust to Single Sign-On.
12. Go to the machine with vSphere Web Client installed and – Update vSphere Web  Client trust to Inventory Service.
13. Go to the machine with vSphere Web Client installed and – Update vSphere Web  Client trust to vCenter Server.
14. Go to the machine with vSphere Web Client installed and – Update the vSphere  Web Client SSL certificate.
15. Go to the machine with Log Browser installed and – Update the Log Browser trust to Single Sign-On.
16. Go to the machine with Log Browser installed and – Update the Log Browser SSL certificate.
17. Go to the machine with vSphere Update Manager installed and – Update the vSphere Update Manager SSL certificate.
18. Go to the machine with vSphere Update Manager installed and – Update vSphere Update Manager trust to vCenter Server.

As you can see, we have to perform 18 steps to fully update all SSL certificates. Due to the “Known Issues” with VUM and using a FQDN, I shall not be performing steps 17-18 since that is not a supported configuration.

4. Getting back to the main menu by pressing 9, I now want to start updating the SSL certificates in the prescribed order per the pre-planner. So I press 2 to start with SSO.

To perform the certificate update I press 1. At this point you can opt to sacrifice a chicken over your vCenter VM to appease the SSL gods and make this go smoother.

After pressing 1 it then asks me where my SSO SSL chain file is stored. And it also wants to know where the SSO private key is, as well. Since we previously configured the environment script, the paths and files it listed were correct. I then typed in my SSO master password (you do remember it, right?). My install did not involve load balancers, so I told the installer no.

At this point the black magic starts, and my heart was thumping hoping that my chicken sacrifice worked. And a minute later….all seems to be well. Chicken worked!

Step 1 of the pre-planning guide is complete. Check!

5. Now that the SSO certificate appears to be successfully updated, it’s time to march on to the inventory service. So I press 3 to return to the main menu. On the main menu I press 3 to update the inventory service. I’m now presented with a plethora of options.

Per the pre-planning guide I need to select option 1. After 30 seconds of disk activity, I get a successful message.

Step 2 of the pre-planning guide is complete. Check! 16 left to go.

6. Slightly illogically the next step is to select option 3, per the pre-planning guide. Again, the certificate paths and files are pre-populated and are correct. Now it wants to know the SSO administrator user. If you aren’t sure what this is, open the Web Client and login. If you can access and modify the Sign-On and Discovery settings, you probably have the right username. In my case this is “sysadmin”, but it will surely be different for you.

A little whirring of my disk drive, and I get a successful message.

Step 3 of the pre-planning guide is complete. Check! 15 left to go.

Next up in Part 3 is continuing the march towards completing all 18 steps by updating the vCenter and Orchestrator certificates.

VMware vCenter Certificate Automation Tool 1.0 Released

Today VMware announced their first stab at helping customers manage the SSL certificate replacement challenge that we face with vSphere 5.1: VMware vCenter Certificate Automation Tool v1.0. For anyone that has followed my 15-part series on vSphere 5.1 installation, you will know the certificate portions are quite a challenge and a source of major headaches and hair loss.

The new tool is called vCenter Certificate Automation 1.0, and will replace the certificates for:

  • vCenter Server
  • vCenter Single Sign On
  • vCenter inventory service
  • vSphere web client
  • vCenter log browser
  • VMware Update Manager
  • vCenter Orchestrator

VMware has a KB article which goes into great detail about how to use the tool and the known issues. It’s critical you read the Know Issues section, as there’s a long list of issues to be aware of. One of the biggies to me is the unsupported case of registering VUM to vCenter using the FQDN. This is standard practice in all of my configurations, so for now v1.0 of this tool won’t be a complete solution. There are also some roll-back issues as well, so just to be safe I would make sure you have a complete backup your server and related databases, in case things go sideways.

It’s great to see VMware try and ease the pain they’ve created in the methodology they’ve employed to use SSL certificates. I hope in future versions that under the covers they do some major re-work of the SSL architecture to not require such complex and tedious steps or specialized tools to implement what I consider basic modern security. The Horizon View team got certificates “right” starting with 5.0.

You can find a four part series on using the tool in the real world here. I encourage everyone to check out that series, so you can get a feeling for how the process works.

VMware Publishes vSphere 5.1 SSL Implementation Guides

As people quickly found out when vSphere 5.1 was released, installation and configuration of vSphere 5.1 SSL certificates was very painful and no accurate VMware documentation existed. Through a lot of work and help from some of my readers, I published a process that was pretty successful but required some workarounds and wasn’t “official”. VMware is now making public a series of KB articles that are much more accurate and usable. I reviewed a draft version, and spotted a few areas that I gave VMware feedback on. You can find the KB articles here.

As I install the vSphere 5.1.0A update, and review all of the KB articles, I will update any of my posts as needed. Most seemed spot on, but they did have a different method for fixing the Web Client and Log Browser SSL issues. So that post will get a re-write once I can test the procedures. I should get everything updated over the weekend.

vSphere 5.1 SSL Install Guides

If you want to follow a whole series of blog posts on how to install vCenter 5.1, soup to nuts, then check out my posts here.

vCenter 5.1 SSL Pre-Staging Script

UPDATE 4/28/2013: VMware has released the vCenter Certificate Automation tool. This is a better tool for replacing the vCenter SSL certificates (post-install), and does not require pre-staging the certificates. Since this is an official tool, and does more than my pre-staging method, I strongly urge you to follow my refreshed vSphere 5.1 Update 1 instructions and use the VMware certificate tool instead of the script below.

Given the complexity and bugs with replacing the SSL certificates in vSphere 5.1, the method which seems to work pretty well is what I call vCenter 5.1 SSL pre-staging. In Part 2 of my vSphere 5.1 installation series, I show how to create the required SSL certificates. To make the installation a bit faster and less error prone, I wrote a super simple batch file that creates the required SSL directories and copies the certificates from Part 2 to the proper directories. You can then install the Inventory Service, vCenter, Web Client, and VUM with minimal fuss.

The batch file assumes the directory structure that I outlined in Part 2 is in place. Noteworthy is that the SSO service does not have a “default” directory for the SSL certificates, unlike the rest of the services. So I created one (see the first line in the batch file), which protects the SSO SSL certificates from getting messed with since configuration files point to their location. Thus using a “temp” location for the SSO SSL certificates is a bad idea, and will result in a broken install if/when those certificates are changed or deleted.

You can run the batch file after you complete Part 2, and before you proceed to any further sections. The SSO service still needs manual configuration for trusted SSL certs, but the rest of the services will automatically use the new certs.

After the batch file runs, you should see the rui.pfx, rui.key and rui.crt files in each of the SSL directories. You can proceed to Part 3 after you run the batch file.

mkdir c:\ProgramData\VMware\SingleSignOn\SSL
robocopy D:\Certs\SSO\ c:\ProgramData\VMware\SingleSignOn\SSL\ /XF rui.csr sso.cfg
copy D:\certs\Root64.cer C:\ProgramData\VMware\SingleSignOn\SSL\
mkdir "C:\ProgramData\VMware\Infrastructure\Inventory Service\ssl"
robocopy D:\Certs\Inventory\ "C:\ProgramData\VMware\Infrastructure\Inventory Service\ssl" /XF rui.csr inventory.cfg
mkdir "C:\ProgramData\VMware\VMware VirtualCenter\ssl"
robocopy D:\Certs\vCenter\ "C:\ProgramData\VMware\VMware VirtualCenter\ssl" /XF rui.csr vcenter.cfg
mkdir "C:\ProgramData\VMware\vSphere Web Client\ssl"
robocopy D:\Certs\WebClient\ "C:\ProgramData\VMware\vSphere web client\ssl" /XF rui.csr webclient.cfg
mkdir "C:\Program Files (x86)\VMware\Infrastructure\Update Manager\SSL"
robocopy D:\Certs\VUM\  "C:\Program Files (x86)\VMware\Infrastructure\Update Manager\SSL" /XF rui.csr vum.cfg
© 2017 - Sitemap