Decommissioning a vCenter PSC and VCSA

Posted by & filed under Virtualization, VMWare.

I recently needed to decommission a VCSA and external PSC. Following the VMWare KB 2106736 I proceeded to decomission the servers usign the cmsso utility.

 

Decommission vCenter — connected to the PSC it is registered with and:

root@vcenter-sb-psc [ ~ ]# cmsso-util unregister --node-pnid vcenter-sb.redacted.lan --username administrator@vsphere.local
Password:
2017-11-01T18:20:15.806Z   Running command: ['/usr/lib/vmware-vmafd/bin/dir-cli', 'service', 'list', '--login', 'administrator@vsphere.local']
2017-11-01T18:20:15.863Z   Done running command
Success

Now to connect to the PSC that will be staying online and decommission the other PSC

root@vcenter-psc [ ~ ]# cmsso-util unregister --node-pnid vcenter-sb --username administrator@vsphere.local
Password:
Could not find a host id which maps to vcenter-sb in Component Manager
Failed!!!

I proceeded to use vdcrepadmin to check out the replication partners which is only vcenter-sb-psc.redacted.lan from my PSC that will be staying online:

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# ./vdcrepadmin -f showpartners -h vcenter-psc.redacted.lan -u administrator
password:
ldap://vcenter-sb-psc.redacted.lan

Then I checked the actual servers:

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# ./vdcrepadmin -f showservers -h vcenter-psc.redacted.lan -u administrator
password:
cn=vcenter-psc.redacted.lan,cn=Servers,cn=redactedfl,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=vcenter-sb-psc.redacted.lan,cn=Servers,cn=redactedsb,cn=Sites,cn=Configuration,dc=vsphere,dc=local

We see both PSC’s as expected. Finally I removed the PSC that is to be decommissioned:

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# ./vdcleavefed -h vcenter-sb-psc.redacted.lan -u administrator
password:
vdcleavefd offline for server vcenter-sb-psc.redacted.lan
Leave federation cleanup failed. Error[1] - Operations error

Error again. Some googling led me to techbrainblog’s excellent page on using these utilities and also the solutions to some common but cryptic errors. Very useful. The solution to this error in particular is to simply shut down the old PSC. It needs to be offline before the command is ran.

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# ./vdcleavefed -h vcenter-sb-psc.redacted.lan -u administrator
password:
vdcleavefd offline for server vcenter-sb-psc.redacted.lan
 vcenter-sb-psc.tnsc.lan server cleanup performed.

Good to go!

Ref: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2106736

Ref: https://techbrainblog.com/2015/10/02/issues-and-errors-when-decommissioning-the-vcenter-server-or-a-platform-services-controller-vcsa-6-0/

Recovering from a failed platform services controller installation – vSphere 6.5

Posted by & filed under Virtualization, VMWare.

I have used to below commands to recover from a failed PSC deployment. When trying to redeploy after the failed deployment, I encountered the error:

“Failed to run vdcpromo”

Following the below steps on the current PSC resolved the error and I was then able to successfully restart the PSC deployment.

Also, protip to avoid having to keep redeploying the appliance, take a snapshot right after phase 1 completes. Then you can simply restore the snap and access your vm via the web interface to try again.

login as: root

VMware vCenter Server Appliance 6.5.0.10000

Type: vCenter Server with an embedded Platform Services Controller

Using keyboard-interactive authentication.
Password:
Last login: Wed Sep 20 15:34:18 2017 from 10.110.0.181
Connected to service

    * List APIs: "help api list"
    * List Plugins: "help pi list"
    * Launch BASH: "shell"

Command> shell
Shell access is granted to root
root@vcenter [ ~ ]# cd /usr/lib/vmware-vmdir/bin
root@vcenter [ /usr/lib/vmware-vmdir/bin ]# ./vdcleavefed -h vcenter-psc.redacted.lan -u Administrator
password:
vdcleavefd offline for server vcenter-psc.redacted.lan
 vcenter-psc.redacted.lan server cleanup performed.
root@vcenter [ /usr/lib/vmware-vmdir/bin ]#

 

docs.vmware.com/en/VMware-vSphere/6.5/co…

Additional info: I also ran into this when trying to deploy an additional PSC that had a failed installation, but got a completely different error (see below). Going to Administration -> System Configuration in the flash vSphere web client also displays the failed PSC. Login to the live PSC and use the above commands to cleanup, then restart the new PSC deployment. Refreshing the System Configuration page once the vdcleavefed command was ran confirms the cleanup is complete and the failed install is no longer listed.

The error I received when deploying this PSC was:

Could not connect to VMware Directory Service via LDAP. Verify VMware Directory Service is running on the appropriate system and is reachable from this host.

Removing the failed deployment via vdcleavefed did not resolve the issue.

I decided to test LDAP connectivity to the PSC from the failed PSC deployment. I SSH’d into the box and did the following:

root@localhost [ /usr/lib/vmware-vmdir/bin ]# ./vdcadmintool


==================
Please select:
0. exit
1. Test LDAP connectivity
2. Force start replication cycle
3. Reset account password
4. Set log level and mask
5. Set vmdir state
6. Get vmdir state
7. Get vmdir log level and mask
==================

1
Please enter LDAP server host: vcenter-psc.redacted.lan
Please enter LDAP server port: 389
Please enter LDAP server SSL port: 11712
Please enter LDAP Bind DN: cn=Administrator,cn=Users,dc=vsphere,dc=local
Please enter LDAP Bind UPN: Administrator@vsphere.local
Please enter LDAP Bind password:

ldap://vcenter-psc.redacted.lan:389 (ANONYMOUS) bind succeeded.

++++++++++++++++++++ ldaps://vcenter-psc.redacted.lan:11712 SSL bind failed. (-1)(Can't contact LDAP server)

ldap://vcenter-psc.redacted.lan:389 SRP bind succeeded.

++++++++++++++++++++ ldap://vcenter-psc.redacted.lan:389 GSSAPI bind failed. (9100)(Unknown (extension) error)

Edit: Additional semi-related data

Get machine’s guid

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost

Get machine’s pnid (machine/host name?)

root@vcenter-psc [ /usr/lib/vmware-vmdir/bin ]# /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost

Get services in the directory

root@vcenter-psc [ ~ ]# /usr/lib/vmware-vmafd/bin/dir-cli service list