vSphere ESXi SSL mystery solved

For quite a while I’ve been trying to get SSL certificates uploaded to an ESXi 4.0 host which were issued by our internal Microsoft CA. Unfortunately I ran into issues, the last being that adding an ESXi 4.0 host to vCenter 4.0 with the certificate would die at 80%.

After additional testing, I now have a procedure which seems to work perfectly for ESXi 4.0 and 4.0 Update 1 hosts. But you must follow the steps exactly as written, or it may not work. It even works with a certificate from a Windows Server 2008 R2 CA using the new sha512ECDSA (elliptic curve digital signature althorithm with secure hash algorithm 512) NSA Suite-B certificates.

1. Download the Windows OpenSSL binaries, either 32-bit or 64-bit. Remember to install the Visual C++ binaries on prior to OpenSSL.

2. I create a directory called Certs in c:OpenSSL just to keep certificates separate.

3. Cd c:opensslcerts

4. c:opensslbinopenssl genrsa 2048 > rui.key

5. c:opensslbinopenssl req -new -key rui.key > rui.csr

6. At this point OpenSSL will prompt you for various parameters. Enter any information you wish, but make sure the Common Name is the FQDN of your ESX server (.e.g. Q100ESX01.contoso.net). Do not set a password.

7. Use NotePad and copy the contents of rui.csr to the clipboard.

8. Navigate to your Microsoft CA and select the option called something like “Submit a certificate request by using a base-64-encoded CMC….”

9. On the Saved Request screen paste the contents of the clipboard, and change the certificate template to Web Server.

10. Submit the request, then download the Base-64 encoded certificate (not the certificate chain). I saved the file as rui.cer into the c:OpenSSLCerts diretory.

11. Optional: Perform verification of the certificates per my blog post here.

12. c:opensslbinopenssl x509 -in rui.cer -out rui.crt

13. Open a VMware vSPhere CLI command prompt (if you don’t have RemoteCLI installed, download it here.

14. vifs.pl –server ESXhostname –put c:opensslcertsrui.key /host/ssl_key

15. vifs.pl –server ESXhostname –put c:opensslcertsrui.crt /host/ssl_cert

16. Reboot the ESXi host and wait five minutes after the ESXi console appears. Use a web browser and navigate to your ESXi host. In the address bar of your browser open the properties of the SSL certificate and verify it was issued by your CA and is not the self-signed certificate.

17. Add your ESXi host to vCenter, and it should NOT get stuck at 80% and fail.

If you run into problems, make sure on the ESXi console that the hostname is configured with a FQDN. From the ESXi console you can also view the management agent logs and look for any SSL related errors.

Next up is changing the vCenter server SSL certificates, as well as VUM. This was broken in 4.0, so hopefully Update 1 has solved these problems. Expect a blog update on this and a procedure, if I find one that works.

Print Friendly, PDF & Email

Related Posts

Notify of
Newest Most Voted
Inline Feedbacks
View all comments
August 19, 2011 12:05 pm

Yup, I can’t upload anymore either, it seems nobody has updated the post to help us who get this error. I noticed there was no /host directory on the vmware server so I created one and it still doesn’t work.

August 6, 2013 11:56 am

Your steps 4 and 5 can be combined into a single command: c:\openssl\bin\openssl req -new -newkey rsa:2048 -nodes -keyout rui.key -out rui.csr It is possible to add additional options to specify the answers to the name prompts, allowing almost complete automation (except the step where an authorized domain admin OKs the CSR, this obviously cannot be automated without compromising security). Now, what I have not yet found is a way to load the new certificate without rebooting the host, which can be really annoying for a routine annual certificate renewal, especially if you don’t have enough overcapacity to vMotion all… Read more »

March 18, 2014 4:22 am

I have the same problem that was describe before:
D:>vifs.pl –server host1 –username root –put rui.crt /host/ssl_cert
Enter password:
Error: File rui.crt can not be uploaded to ssl_cert.

I have check:
– path (/host/ssl_cert or /etc/vmware/ssl)
– the permissions

VMWare version: ESXi 4.1.0 latest version from VMWare

Can you tell me what I can try to upload the Certificate?

Thank you