vCenter 5.1 U1 Installation: Part 7 (Install vCenter Server 5.1)

If you’ve made it this far, congrats! We are now ready to Install vCenter Server 5.1 Update 1! Yes in vSphere 5.1 there is A LOT of prerequisite work to do before you can start the vCenter Server installation. Part 6 of my series showed how to configure the vCenter and VUM SQL databases and the vCenter DSN. Now that all of the pre-reqs have been completed, we can now install vCenter Server 5.1 Update 1!

Before we get started, listed below are the other related articles in this series:

Part 1 (SSO Service)
Part 2 (Create vCenter SSL Certificates)
Part 3 (Install vCenter SSO SSL Certificate)
Part 4 (Inventory Service Install)
Part 5 (Inventory Service SSL Certificate)
Part 6 (Create vCenter and VUM Databases)
Part 8 (Install Web Client)
Part 9 (Optional SSO Configuration)
Part 10 (Create VUM DSN)
Part 11 (Install VUM)
Part 12 (VUM SSL Configuration)
Part 13 (VUM Configuration)
Part 14 (Web Client and Log Browser SSL)
Part 15 (ESXi Host SSL Certificate)

UPDATE 4/28/2013: I removed the SSL certificate pre-population steps, as using the VMware vCenter Certificate Automation tool is a much better option. You do that post-install, after all the components have been installed with self-signed certificates.

Install vCenter Server 5.1 Update 1

1. In Part 1 I created a service account that the SSO service used, and for the sake of simplicity I’ll use the same service account for the vCenter Server service. Login to your vCenter server as the service account. It should already have local admin rights on the vCenter server. Launch the vSphere 5.1 installer menu and select VMware vCenter Server and start the installation.

2. Select the appropriate language, read through all of the patents, EULA, and enter a license key if you have one.

3. On the Database Options screen you should select the second option then, if all went well, find your vCenter DSN from the drop-down menu.

4. Since we are using Windows authentication to the SQL server (more secure than SQL authentication) you can’t ender a database username or password.

5. You will likely see this warning message about the SQL database in full recovery mode, and that it may consume a lot of disk space without regular backups. This is normal and do NOT be alarmed. You ARE doing regular SQL backups right?

6. If you are running the installation as the vCenter service account (which you should be), then the account name will be pre-populated and you need to enter the appropriate password.
7. We don’t need to join an existing Linked mode group, so standalone is fine.

8. All of the default port numbers are fine, and for small environments we don’t need to increase the number of available ephemeral ports. If you will be powering on more than 2,000 VMs, then check the box.

9. JVM memory is an important configuration parameter, so carefully choose the right value. It doesn’t hurt to select a larger value, assuming you have adequate memory assigned to the vCenter VM.

10. New to vSphere 5.1 is the SSO service, so we need to input the master password used during the SSO installation process which I covered in part 1. The wizard will validate the password.

11. At this prompt you need to enter the group or user that will be recognized by the SSO service as the vCenter administrator. If you installed the SSO service in High Availability mode, then you will probably get an error “Wrong Input – either a command line argument is wrong….” if you try and use the “Administrators” group. So I would create an AD group that you want to use. Following my RBAC naming convention I specified the appropriate AD group. Use whatever group name you wish. The wizard will validate that it exists.

Note: If you get suck at this point in the installer, check out the reader feedback below. Ben Hicks and John have some great tips on possible solutions.

12. Next you should see the vCenter Inventory Service URL, which needs no modifications.
13. Change the installation path if you wish, but I left it the default value. Then click Install and wait for it to complete. Profile Driven install may take a loooong time to install…20 minutes or more. So be patient while the installer runs.

14. Per a VMware KB article you need to fix the ADAM SSL port registry type. To fix this issue navigate to:


Delete the Port SSL key and recreate it as a 32-bit DWORD with a decimal value of 636.   Note: Per reader feedback, if you are using Linked Mode, use a different port number (above 1025) for the Port SSL, otherwise there will be a conflict.

Assuming a successful installation, you can proceed to Part 8, where we install the vSphere Web Client.
Buy me a coffee

Related Posts

Notify of
Newest Most Voted
Inline Feedbacks
View all comments


Excellent to know! I’m trying a fresh install as well, each with different certs from my 2008 R2 CA. I’ll see how that goes!

Terafirma, yes now that you point that out, I found that in the VMware PKI Guide. I’ll amend my post.thanks for catching that!

September 18, 2012 7:51 pm

Got Web-Client going turned out to be just not waiting long enough for the service to start and compile its keystore. (me getting frustrated and impatient)

Only annoying thing is Web-Client can only have its cert updated by putting it in the ProgramData\vSphere Web Client\SSL\ folder before install as the register-sso scripts that VMware reference don’t actually exist!!

I must say VMware really dropped the ball on SSL certs this release bring on vCertManager from Michael Webster

All that is left now is Orchestrator.

September 22, 2012 8:37 am


thanks for your documnet. I try the best from your doc and it come to the vsphere server installation almost done the system come up the error 26002 setup fail to register VMware vCenter server to VMwar vCenter Inventory Service. I wonder you see this error before. need some help. Thank again!

September 22, 2012 10:38 am

Anonymous: Yes, I’ve run into the exact same problem and have an open case with VMware tech support. There is wide spread frustration in the community with trying to get SSL certs working. For now I’d skip the trusted SSL certificate generation and go with the vanilla install. As reliable solutions emerge I’ll update my posts with better instructions.

Anonymous, you can work around the 26002 setup failure by skipping Part 5, replacing the Inventory service SSL certificate. You can replace the SSO certificate and the install will continue as one would expect. I’ll keep researching the Inventory service SSL problem.

September 23, 2012 10:56 am

How do I change the vCenter SSL certificates?
Using custom SSL certificates in vCenter 5.1 is fucking pain.

October 7, 2012 10:55 am

Hi Derek, appreciate all your patience & time in writing these great posts! Just a quick question from my end: Our’s is a small environment running with Vcenter 4.1, SQL 2005 express & Six ESX 4.1 hosts.

Planning to perform inplace upgrade of vCenter 4.1 to 5.1 using Simple Install Option & SQL 2008 R2 Express (bundled along with vCenter). Do you think it should be straight forward or any hiccups which I need to work on before proceeding..? Thanks for your time!

Reply to  Ashok

I haven’t done in-place upgrades, and given the problems with 5.1, I would expect some hiccups. VMware is rumored to be releasing an update of some time this month to address the SSL problems. So personally I’d wait on the patch/update before upgrading. If your vCenter/SQL/Etc are on one VM, then just snapshot the VM and see how the upgrade goes. If it goes south, just unsnapshot.

October 12, 2012 6:28 am

Derek, how do you log into the machine with service account as that account was supposed to be just for services and shouldn’t be able to log-in? I can’t make it work…


Reply to  Anonymous

A service account is no different from a regular user account in AD. The service account must have local admin rights on the vCenter server. So once you put the service account in the local admins group on the vCenter server, you should be able to login with no problem.

October 15, 2012 12:05 am

i have added the administrator as a single user in step 11, and modified the _administrators_ group and added other AD accounts to it, but the users cannot login neither to the vSphere client nor to the web client.
how can i modify the users in the administrator (under SSO users and groups) role or create a new role with admin privileges and add the users to it.
please help and advice.

October 19, 2012 7:27 am

Thank you for your extensive instructions on how to install the new vCenter server 5.1. They have been very helpfull to me. Specially after one of my production vCenter servers crashed last weekend. With the exact same symptoms as the test vCenter server that crashed after replacing the SSL certificates. An empty VPX_ACCESS table and a vCenter server service that would start and immediately stop. Even though I didn’t touch the certificates, I just patched and rebooted it. I managed to fix it, by reinstalling the vCenter server service and changing the default admin group added and the default SSL… Read more »

November 11, 2012 8:42 pm

Hi Derek,

Great tutorial. I have a quick question. After finishing vcenter installation I couldn’t start the service. I’ve found the following error message:
error ‘Default’] [0] error:0906D06C:PEM routines:PEM_read_bio:no start line
error ‘Default’] [1] error:0906D06C:PEM routines:PEM_read_bio:no start line
error ‘Default’] [2] error:02001002:system library:fopen:No such file or directory
error ‘Default’] [3] error:2006D080:BIO routines:BIO_new_file:no such file
error ‘Default’] [4] error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib
error ‘Default’] Failed to initialize the SSL context: SSL Exception: error:0906D06C:PEM routines:PEM_read_bio:no start line

Do you have any ideas?

Reply to  Mark

@Mark: I would review the certificate files and ensure they have the proper start/end headers. Sounds like malformed certificates to me.

November 15, 2012 7:02 pm
Reply to  Mark

I had the exact same error and even opened a case with VMware, but I have managed to fix this myself. Following every guide I could find they kept saying to use Openssl-Win32. I tried several times to create the certificates and it kept giving me this error. I ended up humouring myself and installing OpenSSL-Win64 1.0.1c and low and behold, vCenter now happily accepts the pre-staged SSL Certificates. Give that a try, might work for you. I am still battling with the vSphere Web Client tho. I keep getting the yellow warning: Failed to verify the SSL certificate for… Read more »

November 20, 2012 12:20 am

I found the Problem. I had an OpenSSL error, so the .pfx file was empty.
After recreating the pfx file the webservices started successfully.

December 5, 2012 8:22 am

Looking for some help here, running thru your install guides…WHICH ARE FANTASTIC BTW…but am running into some issue that hopfully someone can help me out on. Installing on a fresh, brand new environment…vCenter / SSO etc does not exist so no upgrade…just a straight forward install from scratch environment. Here are my issues.1. SSO – As some earlier in the replies stated: Getting the following error when running “rsautil manage-oc-administrators -a list” “Error: Bean (PrimaryCommandTarget) initialization failure Invalid keystore format” But no resolution or a fix2. At the vCenter install. I am at the “vCenter Server administrator recongnized by… Read more »

December 11, 2012 7:00 am
Reply to  Dan Cruice

With regards to the “The user or group that you are trying to assign vCenter Server administrative privileges to does not exist”…error that I was getting. I read a post to try to use [email protected]…and surprise surprise, that worked for me.

December 5, 2012 6:35 pm


Thank you for your post. It is really helpful. Keep it up1


December 11, 2012 9:58 am

A lesson learned: I received the dreaded Error 26002 error during my first install even when using the newer version of the vCenter install media. In my case the issue seems to have been caused by having the Web Server Role installed on the server. I know you’re not supposed to have IIS on the box but I had the default web site stopped and disabled so I assumed it wouldn’t conflict. Apparently it conflicts anyway, because the only way I was able to successfully install vCenter was to remove the Web Server role completely from the server. It is… Read more »

December 13, 2012 5:40 am

About Error 26002. I got it as well although I followed all instructions in your great posts. After some hair polling it turns out that vCenter Server, unlike all other components, doesn’t like certificates that have any text before the start certificte marker. If you sign your certificate requests using openssl ca like I do then by default openssl adds the text form of the certificate before the encoded form in the resulting rui.crt file. To get around this either edit the certificate in notepad and remove all the text before the start certificte marker. Or better yet add the… Read more »

December 20, 2012 3:00 am

During the step 11 of the installation I have tried to use domain account, domain group, local group, local account but still I get the following error “The user or group that you are trying to assign vCenter Server administrative privileges to does not exist”. Is there any way to resolve the issue. VM where I try to install vCenter on is a part of windows domain

February 7, 2013 6:04 am
Reply to  Zubrania

I have just managed to resolved this issue. When trying to install vCenter server, I was getting the “does not exist error”. I tried everything to resolve this with regards to formatting of names / local vs domain users and groups but to no avail. The solution was to install the web client before vCenter. Once you have the web client installed, you login as your admin@system-domain user and go to “Sign-on and discovery” and then configuration. You need to add an identity source that corresponds to your domain. Put in a domain controller server with the format of ldap://fqdn… Read more »

Dave Benedict
May 22, 2013 11:13 am
Reply to  Ben Hicks

Hi and thanks so much for this information. It gives me way to troubleshoot that I would not have found on my own. I might add that regtool.cmd command in the latest release (4/25/2013) is fundamentally flawed. The output I get is: FilesVMwareInfrastructurejre"" was unexpected at this time. One of our java guys made a modification to the file with me. It needed two extra quotes. Now it runs but the sll handshake fails with the following error. com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certi ficate assertion not verified and thumbprint not matched Return code is: SslHandshakeFailed 1 I am still working the problem.… Read more »

December 28, 2012 8:52 am

Great and great tuto…

I meet a problem to register vCenter Server Administrator group. I made a group (netgus\APP_VCTR_All_Administrators) and I receive a message as what It don’t find my group in my Active Directory. I have lost a step may be, but I found on the Internet this article:

Have you a idea

December 28, 2012 5:13 pm

I found the problem…First, you should not use another language as English, even if VMware propose to you your language (french for me)Second, when I installed SSO, I received one error as “Error 29155.Identity source discovery error”. I had to install VMWare vSphere Web Client and added my AD identity source, but If your OS language is in french, you will receive some strange errors as : “illegal character in scheme name at index 0”. If you use the english language, you’ll have no problem. I have not been able to secure the channel between my databases and my vCenter… Read more »

January 23, 2013 9:48 pm

Hi, Derek, just would like to say that this blog is amazing, it is very detailed and thorough. Just wondering if anyone can shed some light here. I’m trying to upgrade from 5.0U1 to 5.1 on a different box. I saw the below from the official vmware Doc: “You can migrate an existing vCenter Server to a different machine during an upgrade to version 5.0, and then perform an in-place upgrade from version 5.0 to version 5.1. See the version 5.0 vSphere Upgrade documentation.” Has anyone tried a NON in-place upgrade to 5.1? If so, how did you manage to… Read more »

February 5, 2013 8:06 am


Thank a lot for this great article!

I realy could use some help here. When running the vCenter Servr install wizard I keep getting stuck at te “vCenter Inventory service information” window. Every time again I get the warning saying “Setup failed to validate VMware vCenter Inventory Service, error occured while talking…”. I really have no clue what could be the reason. Any assistance would be highly appreciated. Running installer v5.01b BTW. Cheers, B.

February 28, 2013 8:33 pm

@OP: This page should become page 8, and page 8 should become this page. The web client needs to be installed prior to the vCenter Server installation. For those of you stuck a step 11, user Ben Hicks’ suggestion is the key. First, run his console commands and let it fail.Next, close the vCenter server install, go to page 8 of this guide: <a href="… />and install the Web client. Once installed, log into the web client user your “admin@System-Domain” account. The web address should be: https://localhost:9443/vsphere-client/ (use localhost as the name).Click the administration tab on the left side of… Read more »

March 5, 2013 1:13 am
Reply to  John Ball

Hi John / Derek, The install tip came direct from VMware tech support. I think the issue came about (in my case) from an incorrect reverse look-up on the VMs IP address. During the installation – it throws up an error about not being able to correctly resolve and that it may cause problems. My guess is that it uses the PTR record to locate the local domain name and from there a domain controller. If the look-up works correctly, this information is populated and the web client installation is not needed. Either way, once this was all completed I… Read more »

July 16, 2013 10:43 pm
Reply to  John Ball

This saved me, just wanted to give you the props that these instructions worked for someone else too 🙂

Alex Shiroma
August 19, 2013 7:25 am
Reply to  John Ball

Has anyone run into an issue adding an AD Group as a member to the _Administrators_ Group in the last portion of John's instructions? I can search and add individual user accounts but I receive get the following error when searching

Error: exception during group search: (&(|groupType=-2147483640)(groupType=-2147483644)__(objectClass=group))

March 18, 2013 12:46 pm
March 21, 2013 11:08 am

Derek, I am looking for some advise concerning a ‘sort of’ upgrade. I am installing 5.1 on a clean system using your instructions but I want to connect it to an existing 4.1 database. Parts 1-6 have worked beautifully so I want to continue along this path. When I install vCenter Server and connect to the existing db, an upgrade dialog is presented as expected. Once the upgrade starts, however, a SQL exception is thrown. Our environment is small; 6 hosts and about 50 VMs so I am wondering if I should continue to troubleshoot this problem or just create… Read more »

April 25, 2013 4:48 pm

How do you manually replace the vCenter Orchestrator SSL certs?

April 29, 2013 4:43 pm
Reply to  Derek Seaman

Tried using the automation tool but I get this: [.] ERROR: The last certificate in the supplied certificate chain is not a self-
signed authority certificate. Append the authority certificate to the chain file.

The cert chain (.pem) contains the leaf, sub, and root certificates in that order. Any suggestions?

May 15, 2013 10:16 pm


I had used your 15 part doc earlier to create a VC5.1 installation with a MS CA assigned cert., Now I am doing a VC 5.1 Update1 install however I want to follow the manual process. I tried the VMware Tool and it got stuck half way making me start all over again. I see that on April 28th you updated the post and removed the pre-population of SSL Certs for Vcenter and VMware WebClient. Is it possible that I can access the older post somewhere with the steps.



June 6, 2013 3:11 pm

The issue I'm having is I can install vCenter Server and put in all of data points. It's start to install and when it get to the part where VMWare installs Orchestrator it just hangs. If I go to add remove programs Orchestrator is installed but the install program is hung. I have to CTL+ALT+Delete it to kill the process. I've tried uninstalling everything and reinstalling it, rebooting, older version installs, and all the same. I will say this though. One time when I install 5.0 before getting 5.1 It hung on Orchestator and I kill the install. After that… Read more »

June 22, 2013 8:57 am
Reply to  Mike

My vCenter 5.1 Update 1 hung up at "Installing Orchestrator" as well. I followed this VMware KB Article and it allowed the installer to complete.

June 6, 2013 11:53 pm

Here is a note for anyone that is configuring a multisite or HA SSO database – we did this for linked mode , but there are a couple other reasons you may do this as well – make sure that you do not install the vmware Vcenter server as the network server – you will be locked out of Vcenter as the only user that will be granted rights – make sure that you are logged in as the user that the service will run under and when prompted make sure you choose to run the service as that user… Read more »

June 26, 2013 3:07 pm

Hi Derek – thanks for great guide. Killing me getting this installed though. Trying the VCenter install jdbc connection is not working during the install: [2013-06-26 23:49:50 INFO] Invoking testdbprops "C:UsersTEMPWU~1.000AppDataLocalTemp{A4400513-2688-45A9-8439-CA991F4E4106}VM2D39.tmp" Testing DB connection from C:UsersTEMPWU~1.000AppDataLocalTemp{A4400513-2688-45A9-8439-CA991F4E4106}VM2D39.tmp: [2013-06-26 23:49:50 INFO] Config name=dummy [2013-06-26 23:49:50 INFO] Property file=null [2013-06-26 23:49:50 INFO] Loaded url from props=<not set> [2013-06-26 23:49:50 INFO] Overrides= [2013-06-26 23:49:50 INFO] prop:dbtype [2013-06-26 23:49:50 INFO] prop:url [2013-06-26 23:49:50 INFO] prop:driver Error: SQL Server returned an incomplete response. The connection has been closed. [2013-06-26 23:50:04 SEVERE] Error in invocation of testdbprops SQL Server returned an incomplete response. The connection has… Read more »

Mike Evans
August 10, 2013 8:47 pm

FYI, step 14 has been resolved in vCenter 5.1 Update 1a

August 18, 2013 2:55 pm

Hey Derek, I wanted to follow up with you. Since VMware's release of the SSL tool, the suggestions I mentioned in this posted about installing the client, making the necessary SSO changes, then installing vCenter Server was not needed in my case.

I guess some things changed in the back end of this vCenter release to no longer warrant configuring SSO through the web client then installing vCenter.

Thanks for keeping everybody in the loop with VMware's releases!