VMworld 2016: vSphere Encryption Deep Dive

External Threats

  • Nation states, profit motive, highly skilled, social engineering

Internal Threats

  • Snowden.
  • Admins who abuse privileges
  • Physical access to data

VMware’s Vision for security – Secure Access, Secure Infrastructure, Secure Data

VM Encryption Preview

  • Encryption managed via storage policies – Encryption done in ESXi kernel, uses AES-NI, and uses XTS-AES-256.
  • No modification within the guest. VM agnostic.
  • Policy driven. Full support of vMotion and vMotion is encrypted.
  • Uses an external KMS (KMIP compliant)
  • VMDKs are encrypted along with external files such as VMX, snapshots, etc.

Who manages VM encryption?

  • Security admin will manage your KMS and keys
  • Subset of vSphere admins will manage encryption within vSphere

vCenter RBAC has been enhanced for granular encryption control. For example, prevent admins from downloading encrypted VMDKs or opening a console to an encrypted VM.

Key Managers

  • KMIP 1.1 compliant key managers
  • Tested a variety such as Thales, HyTrust, etc.

Key Management Best Practices

  • KMS keys are pushed to all hosts for HA purposes
  • Multiple key managers are supported
  • Expired keys will not be used for new encryption operations. No deep re-encryption needed with new VM key. Shallow re-key operation.
  • No KMS means no booting of encrypted VMs
  • KMS needs to be as reliable as DNS. It must be highly available.

Core Dumps

  • Core dumps are encrypted with a host key
  • Logs are not encrypted
  • You can re-encrypt the core dump with a password (e.g. GSS support needs)
  • Always collect support bundle with a password
  • Uses OpenSSL for core re-keys

Backup, Restore and VM Best Practices

  • SAN mode backups are not supported (use hot-add).
  • No API changes for backup products
  • Backup proxy VM must be encrypted.
  • Backup service account needs cryptographer.directaccess permission
  • Backup data is not backed up encrypted
  • Have a policy in place to re-encrypt a restored VM
  • Backup solution should provide its own encryption solution
  • Don’t encrypt vCenter or your PSCs

Encrypted vMotion

  • 3 modes: Disabled, Opportunistic, Required
  • Configure vMotion encryption from vCenter GUI
  • One-time usage key for each vMotion
  • Set vMotion encryption via PowerShell as well


Ignite 2015: Encryption, Certificates and PKI

Session: BRK3130

Note: This was a great beginner level session for those not familiar with encryption, certificates or PKI. If you are in that boat, I would urge you to find the session video and watch the whole presentation. If you are a security professional and already know about these topics, then the content is probably too basic. I didn’t capture all the content below, but just took down some highlights what was covered.

Why am I here? Thanks to the NSA. Thanks to Edward Snowden. SharePoint, Lync, Exchange all  need to be secure.

Shows screens of RDP SSL warnings, and browser SSL warnings.

Are you still using passwords? Phishing and fraud, password fatigue, pass the hash attacks

IoT (Internet of things) is adding new concerns of authentication (connected cars, medical, industrial sensors)

Non-repudiation – Ability to bind a human to a digital document

Privacy – Hot topic over the last 2 years due to NSA and Snowden. Challenges are not new.

Encryption – Encryption at rest, in transit, challenges: weak algorithms

Encryption at rest – Bitlocker, EFS, SQL TDE

Encryption in transit – SSL/TLS, IPsec, Office 365 message encryption

Azure RMS – AD RMS for On-Premises. Protect documents from Birth to end of life. Protection regardless of location.

Speaker goes over symmetrical, asymmetrical encryption, hardware security modules (HSM) technologies such as AES and shows how they work.

What is hashing? Uniquely identify a stream of data. It’s a one way function.IMAG0425

Use the tool IIS Crypto to disable/enable and change the order that ciphers are use. FREE.

Good ideas: Remove RC4, reorder suites, Update to 2012 R2, research ECC vs. RSA

Talks about Certificate Authorities, certificates, and their basic properties. Also discusses path of trust, and where to find certificates in Windows.

CA Lifetime planning: End certs – 2 years, intermediate CA – 4 years, root CA – 8 years. Renew certificates when 50% of their life has expired.

S/MIME – For Email encryption and digital signatures

vSphere 5.5 Install Pt. 5: SSL Deep Dive

SSL certificateNow that you’ve read up on upgrade best practices and SSO enhancements in the first four installments, we can get to the fun stuff! Understanding vCenter 5.5 SSL certificates. Ok maybe not fun for all, but fun to me. Last year in my 5.1 install series I just plowed right into the minting process, but didn’t really explain the big SSL picture. Since SSL is probably the biggest point of confusion in vCenter 5.1 and later, this post will serve to illuminate one of my favorite subjects. After all I have to keep up my “SSL guy” nickname. Do not fear, this series will eventually get to actually mounting the vCenter ISO and doing a software install. But I hope these foundational pre-install articles enable you to better understand the product.

Blog Series

SQL 2012 AlwaysOn Failover Cluster for vCenter
vSphere 5.5 Install Pt. 1: Introduction
vSphere 5.5 Install Pt. 2: SSO 5.5 Reborn 
vSphere 5.5 Install Pt. 3: vCenter Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 4: ESXi 5.5 Upgrade Best Practices and Tips
vSphere 5.5 Install Pt. 5: SSL Deep Dive
vSphere 5.5 Install Pt. 6: SSL Certificate Template
vSphere 5.5 Install Pt. 7: Install SSO
vSphere 5.5 Install Pt. 8: Online SSL Minting
vSphere 5.5 Install Pt. 9: Offline SSL Minting
vSphere 5.5 Install Pt. 10: Update SSO Certificate
vSphere 5.5 Install Pt. 11: Install Web Client 
vSphere 5.5 Install Pt. 12: Configure SSO
vSphere 5.5 Install Pt. 13: Install Inventory Service
vSphere 5.5 Install Pt. 14: Create Databases
vSphere 5.5 Install Pt. 15: Install vCenter
vSphere 5.5 Install Pt. 16: vCenter SSL
vSphere 5.5 Install Pt. 17: Install VUM
vSphere 5.5 Install Pt. 18: VUM SSL
vSphere 5.5 Install Pt. 19: ESXi SSL Certificate

Permalink to this series: vexpert.me/Derek55
Permalink to the Toolkit script: vexpert.me/toolkit55

Who cares about SSL?

Why should you go through the headaches of replacing all the VMware self-signed certificates? What’s the risk of using untrusted certificates? What can happen if the SSL connection is compromised?

Hypervisors are likely the underpinnings of your business critical apps and intellectual property. If your hypervisor is compromised then it’s just a few short commands to access your critical business data. Unless you like your infrastructure being p0wn3d, then you don’t want your VMware infrastructure compromised. If you don’t use trusted certificates, and just click through all the VI client SSL warnings (you have clicked Ignore and trust this certificate many times…haven’t you?) then you won’t know that a man-in-the-middle attack has taken place.

A man-in-the-middle attack is where a third party intercepts your “secure” communications and relays data between you, the attacker’s device, and the end host (like an ESXi server). This can be accomplished by ARP spoofing, or other means. If it’s second nature to ignore and click through all VI certificate warnings, you will have no idea your credentials have been intercepted….in clear text. No fancy brute force decryption required. Just sit back, grab a coke, and enjoy cleartext flowing across your screen. An interesting article on attacking VMware is here.

The diagram below shows the man-in-the-middle attack, and the following graphic is the clear text credentials from the VI client. With those credentials the bad guys can now own your infrastructure.

10-2-2013 9-04-19 PM

10-2-2013 9-23-20 PM

If you use trusted SSL certificates and untrain your staff to ignore SSL warnings, then it’s significantly harder to spoof the communications.  Think of this this way: Would you enter your credit card details or social security number on a site that threw up constant SSL warnings? No. So why trust potentially billions of dollars of IP to untrusted certificates that your grandma could spoof. There’s a reason why financial institutions, Government and other high value targets require real certificates. Don’t think that just because you run Nana’s Cookie shop that the bad guys aren’t after you…they are.

To be clear, man-in-the-middle attacks and SSL certificate spoofing are not unique to VMware or because they did something wrong. It’s a problem across the industry with self-signed certificates. I hope you are sufficiently scared to take certificates seriously and replace them. Even if you bake cookies in your garage and sell them on the corner stand, I’d update your SSL certs.

Certificate Requirements

Last year with vCenter 5.1 there was some confusion around certificate requirements. Ok, A LOT of confusion. How many certs do you need? What CA template options? What is a certificate OU?  I was stumbling around in the dark last October since official guidance at that time was a wee bit skimpy. Thankfully that situation has been largely remedied by VMware. The list below is a combination of the vCenter Certificate automation tool requirements and of the certificates themselves:

  • Platform Support: Windows Server 2008 R2 SP1 or later, including Windows Server 2012 (not R2)
  • Private Key Algorithm: RSA with 1024 bits or higher (I use 2048). No fancy elliptical curve support.
  • Recommended Signature Algorithms: SHA256, SHA384, SHA512
  • NOT Recommended algorithms: MD2, MD5, SHA1
  • Path or certificate filename can’t contain: ^ (caret), % (percent), & (ampersand), ; (semicolon), ) (closing parenthesis)
  • Use PEM certificate format, with a header of —–BEGIN CERTIFICATE——
  • Must use a complete certificate chain
  • Use OpenSSL v0.9.8y (Do NOT use OpenSSL 1.x !!). Recommend 32-bit version.
  • Does NOT support wildcard certificates
  • Have the subject alternative name (SAN) field included in them
  • Have unique OrganizationalUnitNames for the components (which in turn creates a unique Distinguished Name (DN))
  • Key Usage: Include digitalSignature, keyEncipherment, dataEncipherment
  • Seven unique certificates: InventoryService, SSO, UpdateManager, vCenter, WebClient, LogBrowser, Orchestrator

As you can see, there are a number of very specific requirements to meet in order to have success with replacing your vCenter SSL certificates. These are essentially unchanged from vSphere 5.1, but are now documented. Thank you VMware! Official sources of the requirements are here and here, if you have a burning desire to read more…before you fall asleep.

One final requirement that can trip people up is the format of the RSA private key file (rui.key). VMware states that only a key file with the —–BEGIN RSA PRIVATE KEY—- string will be accepted. Newer versions of OpenSSL may format it differently, creating problems down the road.

RSA private key

Certificate Subject Properties

One of the areas of confusion is exactly what certificate properties are important for a successful trusted certificate implementation. If we look at a SSL certificate which we will be generating in an upcoming post, we see a number of properties that may be Greek to most people.

In the example below we see properties common to all our SSL certificates. For example, we can see the signature hash algorithm is the recommended SHA256. You can also see validity dates, which are important. The primary point of confusion, and what took a lot of detective work when 5.1 went GA, is what needed to be in the Subject field. The SSO service in 5.1 and carried through to 5.5, requires a unique DN (distinguished name) for each certificate. The most common way to make the DN unique is to modify the OU value for each certificate. You could modify another field, but I would not recommend it. For all practical purposes the OU must be unique.

As you can see in the first certificate the OU value is vCenterSSO. In the second certificate the OU is VMwareUpdateManager.  While you don’t have to use these exact strings, they must be unique within the SSO certificate store. I’m opting go with the ‘official’ OU descriptions. Yes, even if you do the simple install with all components on a single VM, every component on that server must have a different certificate. You can NOT use wildcard certificates, either.

Commercial CAs can also have problems minting these properties. Although I’m not sure why you’d want to spend the money on a public CA certificate for an internal service. But some people try it…and are usually disappointed with the results.

vCenter SSO certificate VMware Update manager certificate

Certificate Key Usage

There also has been some lively discussion around other certificate properties, and what is and is not required. Certificate key usage defines how the OS or application can use the certificate. For example, can you only use it to digitally sign an email? Or only for code signing? Or just to secure RDP traffic? Those ‘features’ are enabled by Key Usage and Enhanced Key Usage fields.

For the purposes of this series I’m using KB 2037432 as a reference. As you can see in the graphic below, which is a VMware provided certificate request template, you see various fields. Highlighted are the KeyUsage and extendedKeyUsage fields. Of some controversy are whether dataEncipherment and clientAuth options are really required. Those are not properties normally allowed if you use the Microsoft “web server” template. For the purposes of this series I’ll assume they are required and will show you how to create a custom CA template to mint them. YMMV, and possibly the “web server” template will work fine for you. VMware must have a good reason to include them, so I’m not going to argue. My cooler is stocked with bigger VMware fish that need to be fried.

VMware certificate request

If you look at the properties of a properly formatted certificate you will see I used the correct template which enabled the minting of a certificate with said key usage requirements.

SSL data encipherment SSL authentication

Subject Names

Another important certificate property is the subject name and the subject alternative name (SAN). The subject name (commonName) is the FQDN of your vCenter server (assuming all components are on a single VM). Even if you are using the simplified deployment model all seven certificates will have the same commonName. The subjectAltName are alternative names that the host is known by. For those of you familiar with Microsoft Lync or Exchange deployments, you will be familiar with SANs.

There’s a little debate on what needs go to into the SAN field. Specifically, do you include the IP address or not. In the official VMware certificate template request they include the IP (shown below). However, at VMworld they said not to include the IP. Authentication uses Kerberos, which requires hostnames (can’t use IPs). Since the certificate replacement process is challenging, having to re-issue certs if the IP changes is not remotely appealing. My stance is to NOT include IPs in the certificate.

An interesting note I found in KB2037432 was that the FQDN in the SAN field was moved to the last value to support SRM. Curious that the order of SAN fields would matter, but apparently it does for SRM. Sounds like a bug to me.

SSL subject alternative name

Certificate Signing Request (CSR)

In order for a CA to mint you a certificate you must submit what is called a CSR, or certificate signing request, to your CA. This request contains all of the properties in the request template that you see above. It also contains data which is tied to the private RSA key. In our case we will use OpenSSL to process configuration files for each of the seven certificates, then submit the CSR to the CA, where we can then download our freshly minted certificate. The CSR will be in Base64 format, meaning it’s a neatly formatted text file you can open with Notepad and peek at. Although you will need to be Neo to decode it on the fly.

Certificate Authority Hierarchy

In many environments you will have more than one certificate authority. Typically an enterprise would have an offline root CA that is tightly controlled and secured. You then have one or more online issuing CAs which are joined to the domain and issue your certificates. Or, you may have a simplistic model where a single online CA does everything. If you have two or more tiers of CA, then you have some extra work cutout for you. Why? vSphere needs information about all tiers so that it can properly validate the chain of trust all the way to the root. Subsequent installments will show you exactly what extra steps are needed.

Certificate Creation Process

The certificate creation process is fairly simple once you understand the process. First, OpenSSL creates a private RSA key, usually 2048 bits long. That is used in conjunction with the certificate configuration file by OpenSSL to generate a CSR. The CSR is submitted to the CA, and out pops an SSL certificate at the other end. Nifty!

SSL certificate process

PEM Certificates

The vCenter Certificate automation tool requires certificate files to be in a very specific format. If they are not in that format (PEM), then you will have show-stopping problems and wonder why everything is a smoking pile of ashes. PEM encoded files are a series of Base64 certificates in a specific order. First up the tool needs the public certificates for your CA(s). If you have a subordinate CA, then this file (chain.cer) needs to have both the root and subordinate CA BASE64 certificates concatenated together. Do not add any spaces or lines, as that will not work. An example is shown below.

PEM certificate

In addition to the root CAs, each of the seven services needs a chain.pem file that is a combination of your root CA(s) and the certificate for your service. Again, don’t add spaces or lines in the file or you will run into problems.

PEM certificate

Don’t worry about creating the files now or where they go. Coming up is a PoweShell script that does all of the work for you. But if you are troubleshooting, then understanding the file format should help. I know I got tripped up on it last year and embarrassingly opened a support ticket. But I probably wasn’t the only one. Now you won’t have to be one!


Now that you hopefully understand a bit more about the certificate requirements, you will be in a better position to mint all of our SSL certificates. SSL certificates are complicated, and even in vSphere 5.5 they remain largely unchanged. Paths where VMware stores the certs are somewhat different, but the big overhaul that was needed did not happen. There are some cool ‘under the covers’ changes to certificates that I haven’t seen publically talked about yet which I’ll cover in the future. The teaser is that it sets the foundation for better certificate management in the future.

Next up in Part 6 is how to configure your Microsoft CA to issue SSL certificates with all the required properties.