vSphere 6.0 Install Pt. 15: VCSA vCenter Install

If you are a VCSA (vCenter server appliance) convert, and wish to use the vCenter server appliance, this post is for you! It assumes you already have an external VCSA PSC setup, per Part 10. So this post will walk you through a similar deployment process, but this time install vCenter instead of a PSC. If you’ve deployed a Windows vCenter, then skip this post.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install
vSphere 6.0 Install Pt. 15: VCSA vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Deploy VCSA vCenter

1. Download the VCSA ISO (yes ISO, not OVA) and mount it on a Windows VM.

2. Open the root of the ISO and click on the vcsa-setup.html file.

3. Since I’m assuming a fresh install, click on Install.2015-03-29_19-42-354. Accept the license agreement and click Next.

5. Enter the FQDN or IP address of the ESXi server which you want the PSC deployed on. Enter the associated credentials. Click next and wait for the verification to complete. You may get a warning about an untrusted SSL certificate. Accept it.

2015-03-29_19-48-33

6. On your DNS server configure A and PTR records for the vCenter’s address. This is critical!

7. Enter the FQDN of your appliance, and a complex password. If your password is not complex enough it will warn you and provide the complexity requirements.

2015-04-23_19-55-41

8. Select “Install vCenter Server” on the next screen, since we will be using our external PSC.2015-04-24_19-39-009. Now enter the FQDN of your PSC, and the SSO password.

2015-04-24_19-40-4910. Up next is VCSA sizing. In my case I selected Tiny, since this is a small home lab.

2015-04-24_19-42-56

11. Here you get to select your database. SQL is NOT an option, due to the lack of a supported ODBC binary for Linux. I’ll use the built-in Database.

2015-04-24_19-44-4112. Up next is configuring all the network settings. This is pretty self explanatory. Do take note of the time sync options. In a production environment I would suggest syncing to a trusted NTP source and not the ESXi host. Although you should have ESXi configured to sync with a trusted source as well. Not shown in the screenshot is the ability to enable SSH. Since I’m in a lab environment, I enabled SSH.

2015-04-24_19-47-14

13. Review all of the settings to ensure they are 100% correct. Click Next, and sit back for the installation to complete.

2015-04-24_19-49-06

Summary

Deploying vCenter in the form of a VCSA is easy peasy! I really like the VCSA for its ease of deployment, and self-contained nature. Clearly VMware has put a lot of development time into the VCSA, and it shows. Now that vCenter is installed, it’s time to replace more SSL certificates. That’s coming up next!

Critical VMware Security Patches Released

In case you missed it, VMware has released a number of product updates to address critical vulnerability in JRE. JRE is used in many products, so a wide range of products are affected. You can read the full bulletin here. The bulletin details which product version you need to be running to be patched. In many cases patches are “pending” such as vCenter 6.0, and SRM. So keep your eyes out for another VMware announcement when the patches become available. The full patch matrix seems to include products that didn’t made the affected products list. So carefully review the full security bulletin, as a majority (if not nearly all) VMware products are affected.

(Some) Affected Products:

Horizon View 6.x or 5.x
Horizon Workspace Portal Server  2.1 or 2.0
vCenter Operations Manager 5.8.x or 5.7.x
vCloud Automation Center 6.0.1
vSphere Replication prior to 5.8.0.2 or 5.6.0.3
vRealize Automation 6.2.x or 6.1.x
vRealize Code Stream 1.1 or 1.0
vRealize Hyperic 5.8.x, 5.7.x or 5.0.x
vSphere AppHA Prior to 1.1.x
vRealize Business Standard prior to 1.1.x or 1.0.x
NSX for Multi-Hypervisor  prior to 4.2.4
vRealize Configuration Manager 5.7.x or 5.6.x
vRealize Infrastructure 5.8, 5.7

Critical ESXi 5.5 Patch Released

Hot off the presses is a critical patch for ESXi 5.5, dated March 7, 2015. This addresses two VSAN patches:

  • Upgrading from VMware ESXi 5.5, Patch Release ESXi550-201502001 (Build 2456374) or VMware ESXi 5.5, Patch Release ESXi550-201501001 (Build 2403361) to vSphere 6.0 might result in VMware Virtual SAN (VSAN) Data Unavailability. See VMware Knowledge base article 2113024 for more details.
  • The Virtual SAN traces are not generated after you upgrade to VMware ESXi 5.5, Patch Release ESXi550-201502001 (Build 2456374) or VMware ESXi 5.5, Patch Release ESXi550-201501001 (Build 2403361).

You can read the full KB here. You can download the patch here.

Note: If you are Nutanix customer, this issue does not affect you so can safely ignore this update.

 

vSphere 6.0 Installation Pt. 14: Install Windows vCenter

Now that we are pretty far into this series, we can finally install our Windows vCenter. This will leverage our external PSC, for maximum scalability. Depending on your environment size, you may need to scale up the VM’s hardware specs for optimal performance. Consult VMware documentation for sizing guidance. In this exercise we will configure 2 vCPUs and 12GB of RAM, which is enough for a small environment.

If you would rather use the VCSA vCenter instead of a Windows vCenter, don’t fear, that will be coming up in a future blog installment. So if you don’t want a Windows vCenter, then hold on and soon enough I’ll have those instructions published.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install
vSphere 6.0 Install Pt. 15: VCSA vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Provision vCenter VM

Before we install vCenter, we need to provision the vCenter VM. Per VMware recommendations the VM needs at least 8GB of RAM for an embedded installation.  Don’t skimp on memory as performance will likely take a beating, depending on the number of hosts and VMs you are managing. I’d recommend 12GB minimum. Also keep in mind:

  • At least 2 vCPUs (hard minimum)
  • At least 12GB of RAM (8GB hard minimum)
  • At least 70GB D drive (more with VUM)
  • Use VMXNET3 NIC
  • Use any virtual hardware version. I recommend v10 or v11.
  • Recommend Windows Server 2012 R2
  • Enable hot add of memory/CPU (optional)
  • Fully patched OS (important)
  • Verify time sync between PSC and vCenter VM

If you want to use the web client on the vCenter server with IE, then you must install the Desktop Experience feature. Why? That’s the only way to get Flash player in IE with Windows Server 2012 and Windows Server 2012 R2. VMware really needs to dump the Flash interface and go HTML5. If you use a third party browser, make sure you get the very latest Flash player.

After you install the Desktop Experience make sure you patch it. Why? The stock Flash player version is not compatible with the web client and needs to be updated via Windows Update/WSUS/SCCM to the latest version.

10-8-2013 6-11-01 AM

If you will be using IE on the vCenter server you also need to turn off the IE enhanced security mode.

10-8-2013 5-40-17 PM

vCenter Install

When installing vCenter you have two primary options. The first is an embedded option, where it will install the PSC and all the vCenter components in one fell swoop. This is akin to the “Simple” install in vSphere 5.5. The second option lets you deploy the PSC separately from vCenter. If you only install the PSC, when you run the installer the next time you only have an uninstall option and can’t install the rest of the vCenter services. So for this install we will go ahead and do a full vCenter install, using the external PSC that we have previously installed. You can use either a Windows external PSC or a VCSA-based PSC for this. Choice is yours!

1. Launch the vSphere 6.0 autorun installer. On the main screen select vCenter Server for Windows.

2014-11-22_18-40-25

2. Accept the license agreement and pause on the Deployment Type screen. Select vCenter Server.

2015-03-22_14-32-35a

3. On the System Network Name screen verify the FQDN is that of your local server. It should be. Click Next. You may get a warning about IPv6, which you can ignore if you aren’t using it.

2015-01-25_19-03-08

4. Next up you need to point the installer towards your PSC and enter the SSO password you used during the installation process.

2015-03-22_14-41-50

5. When you click next you may get a warning about a time differential. It’s off by just a minute or two I would not worry about it. I saw a warning about a 63 second time delta. After any time warnings you will then get a certificate validation prompt. At this point you will also get a certificate pop-up. Even if you replaced your VMCA and PSC machine SSL certificates, you will see an untrusted certificate here. This is because the VMware Directory Service certificate is used for this authentication. If you followed along in my blog and replaced the VMdir certificate, then it will show a trusted certificate.

2015-04-03_18-07-42

6. Next you need to select your vCenter service account. I always use a Windows service account, so I input those credentials here. I also made sure the service account was in the local administrator’s group on the vCenter server. It will also need the “Log on as a service” right. To do that launch the “Local Security Policy” editor, navigate to “Local Policies” then “User Rights Assignment”.

2015-03-24_8-12-17

2015-03-24_8-06-027. Now you need to configure your database. For anything but a small home lab you should use an external database. If you do opt for the internal database, in vSphere 6.0 it is now vPostgres and NOT SQL Express. In the previous blog series article we configured SQL using my vCenter toolkit script. So now select that DSN from the drop down.

2015-03-24_12-13-43

8. Next up are the default port numbers, which you shouldn’t change.

2015-03-24_12-16-06

9. Now you can change the directory installation paths if you wish. I just took the defaults.

2015-03-24_12-16-48

10. Now you can review your configuration and make sure that everything is good to go. Click Install.

Summary

With the external PSC already installed, doing the vCenter install is a piece of cake. If you are in a small lab, then you don’t even need to fuss with setting up an external database like SQL. For production instances I would always use a SQL or Oracle database. Its best to leave the default installation paths, as VMware instructions for certificate replacement use the default paths. I just don’t see any big reason to stray from the defaults here. KISS principle applies. If you have to choose between using SQL or Oracle for the back end, I would lean towards SQL. The VMware Fling to convert a Windows vCenter to the VCSA currently only works with SQL, so should you ever want to change your deployment model SQL makes it easier.

Next up is installing the VCSA vCenter, which you can find here.

vSphere 6.0 Pt. 13: VMware Directory Svc Certificate

One of the lesser known SSL certificates in the vSphere 6.0 product is called the VMware Directory Service certificate. This is used by the built-in LDAP server for authentication and encryption. It’s most an internal use only certificate, and one that some customers may not worry about replacing. In fact, per VMware support, a lot of customers probably won’t replace this certificate. But, I’m a certificate whore, and wanted to be thorough in my coverage of vSphere 6.0. You will also see the certificate when you install vCenter 6.0 with an external PSC, and authenticate to the PSC. Even if you use the VMCA, the directory services certificate is not replaced by a trusted certificate.

In addition, the VMware certificate tool does not have a menu option to replace the VMware Directory Service certificate. But don’t fear, I’ve built it into my Toolkit script. What VMware doesn’t do, I do. So in this installment I will show you how to replace the VMdir certificate with either one trusted by your enterprise CA, or issued by the VMCA. The toolkit script will also automate the installation for you, on a Windows PSC. If you are using the VCSA, I’m sorry, but we have to use a manual process provided by VMware.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Mint VMDir Certificate (Online)

You should run the Toolkit script on your Windows external PSC, so you have all the files needed locally and it will also automate the installation. If you are using the VCSA PSC, then run this script from a Windows server that has PowerShell 3.0. Use this online procedure if your Microsoft CA will issue the subordinate certificate either with or without approval. This method also supports the VMCA.

If you are using a VMCA as a subordinate, then select menu option 3 to mint your certificates from the VMCA in step 1 below.

1. Run the Toolkit PowerShell script on your external PSC or a Windows server VM if using the VCSA PSC. Select the VMware Directory Service Certificate menu (option 3). Select the option to create a VMDir certificate with an online MS CA (option 1).2015-04-03_8-42-56

2. Enter the FQDN of the PSC, or press ENTER, if running from the PSC to accept the name. If no certificate approval is needed, the new VMDir certificate will be minted and downloaded.2015-04-07_11-03-09

 

If your MS CA is configured to require CA manager approval before issuing a certificate, you will see the following:

2015-04-02_8-06-50

Have the CA manager approve the request ID, then re-run my Toolkit script and select the “Resume a pending online request for VMDir certificate” (option 4). The script will show you the paths to the chained PEM file and the private key file. After the request is complete, all files are located in C:\Certs\VMDir.

Mint Machine Certificate (Offline)

Use this procedure if your issuing certificate authority is NOT a Microsoft online CA. It could be an offline Microsoft CA, or a non-MS CA as well.  This assumes you have the proper templates configured in your CA, per my Part 9 post.

1. Run the Toolkit script and in the VMware Directory Service menu (option 3) select the option to create a VMDir certificate with an offline or non-MS CA (option 2). The script will verify that you have downloaded the root chain certificates.

2. Because I was running this on the external PSC, I just pressed enter for the PSC name.2015-04-07_11-04-37

3. Navigate to C:\Certs\VMDir and upload the VMDir.csr file to your favorite CA and issue a certificate. Download the issued certificate in the base-64 format and save as VMDir.crt in the same folder. You MUST use this file name and it MUST be base-64 encoded. It should only contain the certificate, not a full chain.

4. Re-run the toolkit and from the menu select the option “Create VMDir PEM file from offline or non-Microsoft CA files” (option 5). No input is needed. This will properly create a PEM file with the full certificate chain.2015-04-07_11-06-54

 

Install VMDir Certificate (Windows PSC)

Note: For this procedure I am showing you how to use my Tookit script to install your VMDir certificate. VMware’s certificate tool does NOT support replacing the VMdir certificate, since not all customers feel the need to replace it. I feel the need. VMware did document the process in their vSphere 6.0 documentation, which is what I implemented in the script.

1. Re-run my Toolkit script and in the VMware Directory Service Menu (option 3) select the option “Install custom VMDir certificate on this computer” (option 6). 2015-04-03_8-44-00

2. Wait about 30 seconds, and the process will complete without any user input.

2015-04-02_21-04-48Install VMDir Certificate (VCSA)

By this point I’m assuming you have the BASH shell enabled, and know how to WinSCP and SSH into the VCSA. Those steps have been covered in pervious blog posts, so I’m not repeating them here.

1. Run my Toolkit script and on the main menu select VMware Directory Service Menu (Option 3). On the following menu select option 7 to rename the certificate files.

2. SSH into the VCSA and enter the following command:

/bin/service-control –stop VMWareDirectoryService

3. From the C:\certs\Machine directory copy the vmdircert.pem and vmdirkey.pem files to:

/usr/lib/vmware-vmdir/share/config/

4. Enter the following command:

/bin/service-control –start VMWareDirectoryService

Validate VMDir Certificate

In case you want to verify that the VMDir certificate actually was replaced and is using your trusted certificate, my toolkit can do that too!

1. Launch the Toolkit and from the main menu select VMware Directory Service menu (option 3). From there select Display VMDir Certificate (option 8).

2. OpenSSL is invoked to display in a somewhat unfriendly manner, the SSL certificate used for the LDAP services. Review the properties to ensure they contain those from your trusted CA.

Summary

While not a popular certificate to replace, replacing the VMware Directory Service certificate does its place. Since the Toolkit makes is so easy to do, I’d recommend replacing it as a matter of practice. This will eliminate a somewhat worrisome certificate validation pop-up during the vCenter installation process. Instead of seeing an untrusted certificate, you will see your freshly minted VMDir certificate.

Publishing vSphere 6.0 VMCA Signing Certificate

As mentioned in my vSphere 6.0 installation series, you can configure the new VMCA to be a subordinate to your enterprise CA. This is great, and opens up new certificate management options for organizations that wish to use trusted SSL certificates. However, you may run into a situation where this new root certificate is not published into active directory and thus your Windows computers will not trust it. This short post will show you how to publish the VMCA signing certificate to AD, which then gets pushed down to the domain computers.

You can see if you have this problem by going to your PSC/VMCA and double clicking on the C:\Certs\VMCA\root_signing_cert.crt and clicking on the Certificate Path tab. If there’s just one entry with a yellow warning, your VMCA is not trusted by that Windows computer. Note: This assumes you are using my Toolkit to create your VMCA certificates. That path is not native to the VMware installation. To correct this condition follow the procedure below.2015-04-04_17-04-13

1. On a domain controller launch the GPO management tool.

2. Find the appropriate group policy you wish to manage. In my simple lab I’m modifying the default domain policy.

3. Open Computer Configuration\Policies\Windows Settings\Security Settings\Public Key Policies\Trusted Publishers.

4. From your PSC/VMCA, go to C:\Certs\VMCA and copy the root_signing_cert.crt file to your domain controller.

5. Right click on Trusted Publishers and select Import. Run through the wizard and import your signing certificate.

6. On your PSC/VMCA open a command prompt and type “gpupdate /force” and wait a minute.

7. Double click on C:\Certs\VMCA\root_signing_cert.crt and you should now see the full certificate chain under the Certificate Path tab.

2015-04-04_17-07-31

Where is the Certificate?

If you are curious where the certificate got published, then open a blank MMC and add the certificates snap-in. Manage the computer account, then go down to Trusted Publishers. Open the Certificates folder and you should see the name of your PSC.2015-04-04_18-47-34

vSphere 6.0 Pt. 12: PSC Machine Certificate

Back in Part 11 of this series we configured the VMCA to be a subordinate CA to our enterprise CA. This ensures that all certificates which get used by vCenter components are automatically trusted. But as previously mentioned, not all organizations can use the VMCA. The US Federal Government would be a prime example, where there’s no way you can stand up your own subordinate CA.

So if you are one of the organizations that can NOT use the VMCA and need to use custom SSL certificate throughout, this post is for you. In this post we will replace the PSC’s machine SSL Certificate with a certificate issued by your enterprise CA, not the VMCA. If you followed Part 11 and are using the VMCA, skip this post.

Just like Part 11, I’ll go through the same process of using a Microsoft online CA, offline CA, and updating the certificates for both Windows and the VCSA. This should cover most scenarios that people have to deal with. If that’s not exactly what your scenario is, you can probably figure out what to do between VMware documentation and my Toolkit posts.

As always, download the latest version of my Toolkit script, as it is rapidly changing as I add more blog posts about SSL and work through issues. The download permalink is below. For this post you will need at least version 0.75 (April 2, 2015) or later to follow along.

Ironically, the VMware supplied certificate tool in it’s GA form has a bug when you replace the machine certificate with multiple intermediate CAs. You can find the KB here. So I’d recommend using my Toolkit script for a Windows PSC, as it does not have the bug and is easier anyway. :-) I am told VMware is working on an updated script, but I have no ETA on a release date. If you are using the VCSA you will need to use the workaround, which I cover in my post below.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Mint Machine Certificate (Online)

You should run the Toolkit script on your Windows external PSC, so you have all the files needed locally and it will also automate the installation. If you are using the VCSA PSC, then run this script from a Windows server that has PowerShell 3.0. Use this online procedure if your Microsoft CA will issue the subordinate certificate either with or without approval.

1. Run the Toolkit PowerShell script on your external PSC or a Windows server VM if using the VCSA PSC. Select the Machine SSL Certificate menu (option 4). Select the option to create a Machine SSL certificate with an online MS CA (option 1).

2. Enter the FQDN of the PSC, or press ENTER, if running from the PSC to accept the name. If no certificate approval is needed, the new machine certificate will be minted and downloaded.2015-04-02_8-02-13If your MS CA is configured to require CA manager approval before issuing a certificate, you will see the following:

2015-04-02_8-06-50

Have the CA manager approve the request ID, then re-run my Toolkit script and select the “Resume a pending online request for Machine SSL certificate” (option 4). The script will show you the paths to the chained PEM file and the private key file.

2015-04-02_8-18-27

After the request is complete, you will see the following files in the C:\Certs\Machine directory.

2015-04-02_8-20-02

You have now minted your Machine SSL certificate, but it is not yet installed. Read on further in this post on how to install it.

Mint Machine Certificate (Offline)

Use this procedure if your issuing certificate authority is NOT a Microsoft online CA. It could be an offline Microsoft CA, or a non-MS CA as well.  This assumes you have the proper templates configured in your CA, per my Part 9 post.

1. Run the Toolkit script and in the Machine SSL menu (option 4) select the option to create a Machine SSL certificate with an offline or non-MS CA (option 2). The script will verify that you have downloaded the root chain certificates.

2. Because I was running this on the external PSC, I just pressed enter for the PSC name.

2015-04-02_8-53-19

3. Navigate to C:\Certs\Machine and upload the machine_ssl.csr file to your favorite CA and issue a certificate. Download the issued certificate in the base-64 format and save as new_machine.crt in the same folder. You MUST use this file name and it MUST be base-64 encoded. It should only contain the certificate, not a full chain.

4. Re-run the toolkit and from the menu select the option “Create Machine SSL PEM file from offline or non-Microsoft CA files” (option 5). No input is needed. This will properly create a PEM file with the full certificate chain.

2015-04-02_9-10-18

Install Machine SSL Certificate (Windows PSC)

Note: For this procedure I am showing you how to use my Tookit script to install your Machine SSL certificate. VMware provides a Certificate Management tool that can perform the same steps. I show you how to use the VMware tool in the next section, when using the VCSA. The tool is the same on Windows and the VCSA. So if you feel more comfortable using the VMware tool to install the cert, skip down to that section. On Windows you can find the tool at C:\Program Files\VMware\vCenter Server\vmcad\Certificate-manager. My tool uses the manual method as documented in the vSphere 6.0 security guide, so the results are the same.

1. Re-run my Toolkit script and in the Machine SSL menu (option 4) select the option “Install custom machine SSL certificate on this computer” (option 6).2015-04-02_9-13-45

2. Sit back and wait while the script stops services, installs the new certificate, and restarts the services. Keep an eye on the process, as mid way through you will need to confirm the deletion of the existing machine certificate. Simply press Y.

2015-04-02_9-29-56

Install Machine Certificate (VCSA PSC)

Note: For this procedure I am showing you how to use the VMware Certificate Manager tool to install the VMCA signing certificate. This assumes you used my Toolkit to generate the certificate files. There’s a bug documented in this VMware KB about the tool failing with multiple intermediate CAs. I’ll include the workaround here, so you have a one stop shop for replacing your certificates.

1. If you haven’t already enabled BASH on your VCSA let’s do that now. Open a console into the VCSA. Press F2 to customize the system. Login. Arrow down to “Troubleshooting Mode Options” then enable BASH shell. Exit the VCSA console.

2. Open a SSH session to the VCSA and type the following:

shell

chsh -s “/bin/bash” root

Make sure you run the ‘chsh’ command from the ‘shell’ prompt and not the VMware restricted shell…it won’t recognize the chsh command. Thanks to William Lam’s blog post here for this step!

2. Download and install your favorite SCP client. I like WinSCP. Connect via SCP using the VCSA credentials.2015-04-01_14-40-003. Create a folder to put your SSL certificates. I like the ‘/root/ssl’ directory.

4. In WinSCP navigate to the C:\Certs\machine folder. Upload the new_machine.cer and ssl_key.priv files to the SSL directory on the VCSA. The other files in the machine folder are not needed, so don’t upload them. From the C:\certs folder upload the chain.cer AND the root64.cer files to the /root/ssl directory on the VCSA. Note that all the options begin with a double dash, not a single dash. Cut/paste may mangle the dashes and cause the command to fail. Best to manually type the whole command instead of cut/paste.

5. SSH into the VCSA and ensure you get a ‘shell’ prompt. This will be in red, and have the short name of the VCSA. Type the following command.

Windows :

“C:\Program Files\VMware\vCenter Server\vmafdd\dir-cli.exe” trustedcert publish –chain –cert c:\certs\chain.cer

VCSA:

/usr/lib/vmware-vmafd/bin/dir-cli  trustedcert publish –chain –cert /root/ssl/chain.cer

6. In the VCSA shell run the following command:

/usr/lib/vmware-vmca/bin/certificate-manager

6. Choose Option 1 from the main menu. Enter the SSO password as requested.

7. From the new menu select Option 2, Import custom certificates. Input the root certificate file names when prompted. Use /root/ssl/new_machine.cer for the first prompt and /root/ssl/ssl_key.priv for the second. For the third and final prompt enter /root/ssl/root64.cer.

2015-04-02_14-42-00a

 

8. After you enter all the certificate paths you will be prompted to continue. The whole replacement process takes less than two minutes.

Inspecting the Machine Certificate

Now that we have installed a new machine SSL certificate, we want to make sure it was issued by our enterprise CA and is trusted. This can easily be done via any browser of your choosing.

1. Launch your favorite browser and go to https://PSC-FQDN/websso/. Open the certificate properties for the SSL site.

2015-04-01_19-07-392. Click on the Certification Path, and verify that all of your enterprise CAs are listed. If you only see a single entry in this list, and not the full chain, that likely means your Windows computer does NOT trust the full chain. See your CA administrator for getting all of your enterprise CAs published through Active Directory.

2015-04-02_15-25-05

Solution Warning

A reader pointed out the SRM and other solutions may fail when replacing the machine certificate on vCenter or the PSC. If you find yourself in this situation, check out this VMware KB article for remediation.

Summary

When you aren’t using the VMware VMCA, you must mint and install a machine SSL certificate for the PSC from your enterprise CA. This certificate is used for all reverse proxy services, such as those accessed by HTTP. You can elect to either use my Toolkit script to install the machine cert, or the VMware tool. Either way, you end up with a trusted machine SSL certificate on your PSC.

vSphere 6.0 Install Pt. 11: VMCA as Subordinate

As mentioned in my vSphere 6.0 install part 3, VMware has introduced a new certificate option in this release. You can now make a built-in VMware certificate authority (VMCA) a subordinate to your enterprise CA. This is great news, as it will allow some automation of the certificate deployment process. However, there are certainly some regulated industries like the US Federal Government where they’d laugh you out of the office if you asked to stand up your own subordinate CA. So this model won’t be used by everyone.

So if you are one of those organizations that CAN use the built-in VMware VMCA, then follow along in this article to see how you make it a subordinate in your enterprise environment. Later in this series we will cover the manual replacement process so don’t worry…alternatives to the VMCA are covered.

As a reminder, I’ve written the vCenter Toolkit script which helps in the SSL deployment process. In this version of the script I’ve added some VMCA features. See the download link below. In this series I’m assuming a Windows Enterprise CA, but you can use a non-Windows CA with a little bit more work (manual submission and download).

Important: You MUST use Toolkit version 0.7 or later. Please download the latest Toolkit from the URL below.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Mint VMCA Certificate (Online)

You should run the Toolkit script on your Windows external PSC, so you have all the files needed locally and it will also automate the installation. If you are using the VCSA PSC, then run this script from a Windows server that has PowerShell 3.0. Use this online procedure if your Microsoft CA will issue the subordinate certificate either with or without approval. Before you run the Toolkit, always download the latest version as new versions can come out rapidly. You can get the latest from the permalink above.

1. Run the Toolkit PowerShell script on your external PSC or a Windows server VM if using the VCSA PSC. Select the VMCA menu (option 2). Select the option to create a VMCA signing certificate with an online MS CA (option 1).

2. Press Enter to accept the PSC name, and the script will take care of the rest. After it quickly runs you have a C:\Certs\VMCA directory with all of the needed files.2015-04-01_14-21-25

 

If your MS CA is configured to require CA manager approval before issuing a certificate, you will see the following:

2015-03-29_18-53-10

Have the CA manager approve the request ID, then re-run my Toolkit script and select the “Resume a pending online request for VMCA certificate” (option 3).

2015-03-30_19-07-35

After the request is complete, you will see the following files in the C:\Certs\VMCA directory. As you will notice, I’ve purposely used the same file names as the VMware documentation and tools.

2015-03-30_19-08-41

You have now minted your VMCA certificates, but they are not yet installed. Read on in the blog post to find out how to install them.

Mint VMCA Certificate (Offline)

Use this procedure if your issuing certificate authority is NOT a Microsoft online CA. It could be an offline Microsoft CA, or a non-MS CA as well. Don’t try this procedure with a public CA provider, as they won’t let you issue subordinate CA certificates. Only use this for internal enterprise CAs which you control and can issue subordinate certificate authority certificates with.

1. Run the Toolkit script and in the VMCA menu (option 2) select the option to create a VMCA signing certificate with an offline or non-MS CA (option 2). The script will verify that you have downloaded the root chain certificates.

2. Because I was running this on the external PSC, I just pressed enter for the PSC name.2015-03-29_12-00-023. Navigate to C:\Certs\ and upload the root_signing_cert.csr file to your favorite CA and issue a certificate. Download the issued certificate in the base-64 format and save as root_signing_cert.crt in the same folder. You MUST use this file name and it MUST be base-64 encoded. It should only contain the certificate, not a full chain.

4. Re-run the toolkit and from the menu select the option “Create VMCA PEM files from offline or non-Microsoft CA” (option 4). No input is needed. This will properly create a PEM file with the full certificate chain. 2015-03-30_19-10-59

Now that we have our VMCA certificates, we need to install them. I will cover both the Windows and VCSA installation process. Windows is easier, since I’ve built that functionality into my Toolkit. For the VCSA we will leverage the VMware certificate tool.

Install VMCA Certificate (Windows PSC)

Note: For this procedure I am showing you how to use my Tookit script to install your VMCA signing certificate. VMware provides a Certificate Management tool that can perform the same steps. I show you how to use the VMware tool in the next section, when using the VCSA. The tool is the same on Windows and the VCSA. So if you feel more comfortable using the VMware tool to install the cert, skip down to that section. On Windows you can find the tool at C:\Program Files\VMware\vCenter Server\vmcad\Certificate-manager. My tool uses the manual method as documented in the vSphere 6.0 security guide, so the results are the same.

1. Re-run my Toolkit script and in the VMCA menu (option 2) select the option “Install VMCA signing certificate on Windows PSC” (option 5).

2015-04-01_18-24-46

2. Sit back and wait while the script stops services, installs the new certificate, restarts the services, then lists the certificate properties. Keep an eye on the process, and verify a ‘success’ message half way through. I’ve added a pause and notification when you should see the ‘success’ message. The entire replacement process takes about two minutes. 2015-03-29_12-20-293. Once the script completed, it will show the properties of the new VMCA. Validate that these match the issued certificate.2015-03-29_12-21-47

4. To follow in the footsteps of the VMware Certificate tool that also replaces the Machine SSL certificate by a VMCA issued certificate, re-run the Toolkit and select the Machine SSL certificate menu (option 4). Then select Option 3, Create Machine SSL certificate with VMCA. Press enter to accept the default hostnames (assuming you are running this on the PSC/VMCA).

2015-04-01_18-33-54 5. Re-Run the toolkit and select the Machine SSL certificate menu again. This time select option 6. It will stop the PSC services, install the certificate, and then re-start the services. You will also get prompted mid way through if you want to delete the existing machine certificate. Answer “Y” here to proceed. 2015-04-01_18-39-20

Install VMCA Certificate (VCSA PSC)

Note: For this procedure I am showing you how to use the VMware Certificate Manager tool to install the VMCA signing certificate. This assumes you used my Toolkit to generate the certificate files.

1. If you haven’t already enabled BASH on your VCSA let’s do that now. Open a console into the VCSA. Press F2 to customize the system. Login. Arrow down to “Troubleshooting Mode Options” then enable BASH shell. Exit the VCSA console.

2. Open a SSH session to the VCSA and type the following:

shell

chsh -s “/bin/bash” root

Make sure you run the ‘chsh’ command from the ‘shell’ prompt and not the VMware restricted shell…it won’t recognize the chsh command. Thanks to William Lam’s blog post here for this step!

2. Download and install your favorite SCP client. I like WinSCP. Connect via SCP using the VCSA credentials.2015-04-01_14-40-003. Create a folder to put your SSL certificates. I like the ‘/root/ssl’ directory.

4. In WinSCP navigate to the C:\Certs\VMCA folder. Upload the root_signing_chain.cer and root_signing_cert.key files to the SSL directory on the VCSA. The other files are not needed, so don’t upload them.

5. SSH into the VCSA and ensure you get a ‘shell’ prompt. This will be in red, and have the short name of the VCSA. Type the following command:

/usr/lib/vmware-vmca/bin/certificate-manager

6. Choose Option 2 from the main menu. Enter the SSO password as requested.

7. From the new menu select Option 2, Import custom certificates. Input the root certificate file names when prompted. Use root_signing_chain.cer for the first prompt and root_signing_cert.key for the second.

2015-04-01_17-35-298. If you haven’t run the tool before, you will be prompted for a series of default values for the certool.cfg. Use the same values here as you did when setting up the Toolkit variables. When prompted, use the FQDN of the PSC for the ‘hostname’.

9. Wait a couple of minutes for the tool to run. After it has completed it has made the VMCA a subordinate to your enterprise CA, and also updated the machine SSL certificate to one issued by the new VMCA.

Inspecting the SubCA Cert

If you wish to look at the properties of the newly minted VMCA subordinate signing certificate which was installed, just double click on the C:\Certs\VMCA\root_signing_chain.cer file. Go to the Certification Path tab, and you can see the full certificate chain. If you do NOT see the full certificate chain, then follow my blog post here to enable your enterprise to trust the VMCA.

2015-03-29_19-10-30

You can also check out the Details tab, and go down to the “Basic Constraints” and “Key Usage” fields. There you can see this certificate can be used as a subordinate CA.

2015-03-29_19-12-08

Inspecting the Machine Certificate

Now that we have installed a new machine SSL certificate, we want to make sure it was issued by the VMCA and is trusted. This can easily be done via any browser of your choosing.

1. Launch your favorite browser and go to https://PSC-FQDN/websso/. Open the certificate properties for the SSL site.

2015-04-01_19-07-392. Click on the Certification Path, and verify that all of your enterprise CAs are listed, as well as the FQDN of your PSC. There should be a certificate listed under the FQDN of your PSC. If you only see a single entry in this list, and not the full chain, that likely means your Windows computer does NOT trust the VMCA subordinate CA. If that’s the case, I would recommend publishing the VMCA public certificate via Active Directory so that your entire domain will trust the VMCA. As you can see in my screenshot below I have three enterprise CAs and the VMCA.

2015-04-01_19-11-42

Solution Warning

A reader pointed out the SRM and other solutions may fail when replacing the machine certificate on vCenter or the PSC. If you find yourself in this situation, check out this VMware KB article for remediation.

Summary

Using my toolkit script, minting and updating the VMCA certificate is a breeze. My toolkit supports online Microsoft CAs, or any offline CA that you wish you use. Remember you can’t use a public CA for issuing your VMCA subordinate certificate. This must be an internal CA which you own, and which has been configured to issue certificates for the subordinate authority template.

Because we are using the VMCA for all of our certificates, we’ve also re-issued the PSC’s machine SSL certificate to one issued by the VMCA. This means any of the web services the PSC is providing are now using a VMCA issued certificate. In the next installment we will assume you can NOT use the VMCA for compliance reasons, and will replace the PSC’s machine SSL certificate with a trusted one. That is covered in Part 12. This will bring our two installations up to the same level. After that, we will replace the Directory Services certificate on the PSC.

Top vBloggers Announced!

Each year Eric Siebert from vSphere-Land.com has an annual voting for the top vBloggers in the world. Competition is fierce and back in 2013 I was honored with getting position #24, for which I was thrilled. Then came along more SSL woes with vSphere 5.5, and my vCenter Toolkit was born. In 2014 I then moved up to position #12, and couldn’t have been happier. A lot of other great Nutanix talent also rounded out the Top 25 last year. Now, in 2015 I’m proud to say that I broke into the top 10 at #7!

I want to thank everyone that voted. All of the Top 25 bloggers do amazing work, and I’ve learned a lot from all of their posts. Blogging takes a lot of time, and How-to’s are especially time consuming due to all the testing. For the full list of Top 25 bloggers, you can read Eric’s post here.

2015-03-31_11-02-57

vSphere 6.0 Install Pt. 10: Install VCSA PSC

New to my vSphere installation series is using the pre-packaged vCenter appliance (VCSA). Now that the VCSA is on par with the Windows vCenter server, I suspect more and more people will migrate to the appliance. So to that end, let’s install an external PSC using the VCSA. If you are using a Windows-based external PSC, then you can skip this blog post and go directly to Part 11 (VMCA as subordinate) when that gets published.

Blog Series

vSphere 6.0 Install Pt. 1: Introduction
vSphere 6.0 Install Pt. 2: Platform Services Controller
vSphere 6.0 Install Pt. 3: Certificate Management
vSphere 6.0 Install Pt. 4: vCenter Upgrade Best Practices
vSphere 6.0 Install Pt. 5: ESXi Upgrade Best Practices
vSphere 6.0 Install Pt. 6: Install Windows PSC
vSphere 6.0 Install Pt. 7: Config SQL DBs
vSphere 6.0 Install Pt. 8: Toolkit Configuration
vSphere 6.0 Install Pt. 9: SSL Templates
vSphere 6.0 Install Pt. 10: Install VCSA PSC
vSphere 6.0 Install Pt. 11: VMCA as Subordinate
vSphere 6.0 Install Pt. 12: PSC Machine Certificate
vSphere 6.0 Install Pt. 13: Directory Services Certificate
vSphere 6.0 Install Pt. 14: Windows vCenter Install

Permalink to this series: vexpert.me/Derek60
Permalink to my Toolkit script: vexpert.me/toolkit60

Deploy VCSA PSC

1. Download the VCSA ISO (yes ISO, not OVA) and mount it on a Windows VM.

2. Open the root of the ISO and click on the vcsa-setup.html file.

3. Since I’m assuming a fresh install, click on Install.2015-03-29_19-42-354. Accept the license agreement and click Next.

5. Enter the FQDN or IP address of the ESXi server which you want the PSC deployed on. Enter the associated credentials. Click next and wait for the verification to complete. You may get a warning about an untrusted SSL certificate. Accept it.

2015-03-29_19-48-33

6. On your DNS server configure A and PTR records for the PSC’s address. This is critical!

7. Enter the FQDN of your appliance, and a complex password. If your password is not complex enough it will warn you and provide the complexity requirements.

2015-03-29_19-51-28a8. Next up, select the PSC option and click Next.

2015-03-29_19-53-06

9. Now we get to configure SSO. Yippee! Since I’m assuming a new install, I’ll create a new SSO domain, enter a complex password, and SSO site name. Remember that you should NOT set your SSO domain name to the same as your Windows domain. You could use a sub-domain, such as sso.contoso.local. I’m sticking with vSphere.local.

2015-03-29_19-55-03

10. The appliance is automatically sizes for 2 vCPUs and 2GB of RAM. Not bad for a PSC. Click Next.

11. Next up is datastore selection. In my home lab I have datastores on my QNAP and VSAN. I’ll go with VSAN here.

2015-03-29_19-58-07

12. Now you get to configure your network settings. Everything here is self-explanatory. I used the public NTP servers for accurate time, and also enabled SSH (lower down on the screen).

2015-03-29_20-02-05a

13. On the summary screen review all of the details to ensure they are correct.

2015-03-29_20-04-18

14. Sit back for a few minutes and wait for your VCSA-based PSC to be installed!

2015-03-29_20-11-58

Summary

We walked through the manual process of deploying a VCSA-based PSC in your environment. The VMware wizard is very straight forward, and makes deploying the VCSA very easy. If you want to automate the deployment of the VCSA, check out William Lam’s awesome multi-part guide here. You can also check out an ‘official’ method of command line deployment here. Next up will be configuring the VMCA as a subordinate CA, which you can find here.

© 2015 - Sitemap