Restart hostd - "The real way"
Johnny Zhang
Anyone work with ESX server would know sometimes you need to restart hostd service on the ESX server to "refresh" information on ESX. Normally we do this by typing "service mgmt-vmware restart"
However, sometimes this is not good enough. Every time you restart hostd it will map to 4 files under /var/lib/vmware/hostd/stats.
If any of those file got corrupted by any reason, restart hostd will not help. Every time when hostd restart called, the service will check if those files exist, if not it will create them before starting the service. So we can remove those files to make sure hostd service starts from scrach "rm -rf /var/lib/vmware/hostd/stats/*" Make sure you are in the right direct tory when you run the rm command especially with the -rf switch
Labels: , , 0 comments | edit post
VMware ESX server logs - HA (1)
Johnny Zhang
There are many useful log files on a ESX server to help you find what cause a problem. However, there may be to many of them how do you know which one to look at? I will base on my own experiences to explain what I think is important. I will start with HA logs. HA is configured on vCenter. However, once configured it will function without vCenter. It will build it's own HA domain (This domain always use the name 'vmware'). If you have more than 5 servers in a HA cluster, there will be 5 primaries and the rest will be secondary. The log file location after ESX 3.5 U2 is at /var/log/vmware/aam directory
The most impportant log would be "vmware_server_name.log". You can find many HA related info or errors from here.
Even with 5 primaries, there will be only one cluster manager which is the one holding all the rules. You can find which one is the cluster manager by

less vmware_server_name.log | grep -i "VMWareClusterManager submitted to run on node "| uniq
You can see how HA changed the cluster manager from host to host, and the last record is the current manager.
You can also find out if there was a isolation event happened by
less vmware_bs-bcs-h132.log | grep -i ISOLATED
When HA detect heartbeat failure on one host, it will try to find out if this is a agent failure or host failure. The isolation event will only trigger when there is a host failure. HA does this by ping the host after the heartbeat is gone
less vmware_bs-bcs-h132.log | grep -i "Ping Node results:"
When the host replys the ping it will mark as ALIVE. When the host is not responding it will mark as DEAD and HA event will trigger. (More to come in the future)
Labels: , , 1 comments | edit post
Download log file via vifs
Johnny Zhang
After ESX 3.5 U2, all version of ESX supports remote Cli. The vifs tools in remote Cli is useful when you need to get log files from a remote host.

C:\Program Files\VMware\VMware VI Remote CLI\bin>vifs.pl -D /host --server bs-bcs-h132.bsl.vmware.com --username root

Now once you know what is there, you might want to get some of those files. let's say i want to get hostd.log from this server, and put the file under my c:/ directory, I can use:

C:\Program Files\VMware\VMware VI Remote CLI\bin>vifs.pl -g /host/hostd.log c:/ --server bs-bcs-h132.bsl.vmware.com --username root
Labels: , 0 comments | edit post