Johnny Zhang
I was working with one of the BCS customer on a case. The customer was originally planing to extend a Windows 2003 VM's OS disk to a larger size. However, after extend the VMDK files with VI client, and extend the NTFS inside Windows (Since this was a Windows 2003 guest, the customer needs to use a helper VM to do the job) after failed to power on the VM, the customer found a shocking fact: there was a snapshot. Then the customer tried to shrink the VMDK file back to the original size... and number of other things was also tried. So when we start to work on this issue, the VMDK file was broken in many different ways alone with the snapshot. To make the thing worse the last and only backup was created right after the extension of the VMDK file.

The end story was, we did recover all the data on the VMDK, the VMDK was not good enough to power on as a boot disk, but that was good enough to save the day. Here was how we did it:

First, list the size of the flat file by using "ls -l" in the VM directory
Keep in mind the number we need is the size of the flat file. In my test set up it's 8589934592. We need this number later. Now we need open the VMDK file (The pointer). In my case snap_test.vmdk:
Note, we need the CID and the RW number.
The RW number is the flat disk size divide by 512.
in my case 8589934592 / 512 = 16777216. If this number is wrong, you will not able to power on the VM. (So first we fix this part).

Now let's open the snapshot VMDK file. snap-test-000001.vmdk

In this file the "parentCID" should be the same as the snap-test.vmdk's CID. In my case 4cc5f033. If this is wrong, then the VM will not power on. (We fixed this one as well). The RW size would be also wrong, since it will only record the original size. Change this to the same size as the parent RW (16777216).
Now we need a helper VM to attach this disk as a second VMDK file (Not boot disk). When the OS boots up it will automatically run check disk against the VMDK and auto fix the errors. Now you should able to mount the disk inside the helper VM and copy over the data.
Johnny Zhang
In vCenter 4.0 we now have some easy to used monitor tools for ESX server performance. One of the chart is the memory usage
This usages is NOT the active memory that is currently used by the ESX server or the VM. So when you see very high usage, don't be panic.
In fact, this chart shows the total "Consumed Host Memory" by all the VMs running on the host.
As we can see from the graph the "Consumed Host memory" is 2277.00MB and the "Active Guest Memory" is only 122.00MB. Why a big different?

This is by design, the "Consumed Host Memory" shows the highest memory used by that VM.
OS such like Windows will touch all the memory assigned to it during start up, so most likely you will see the consumed memory is close or little more then the memory assigned to the VM (Overhead to power on the VM is also added here). ESX server will NOT take back the memory if there is no resource contention on the server. The reason is, once the VM need more memory it can access them faster than send the request to vmkernel again.
If you want to know the real time memory usage you can look into the advanced chart to find it out.
Johnny Zhang
Note: This is tip is based on communications between engineering and myself. In most cases there is no need to make any change. This is just for you to know. (This also only based on VI3, not sure if there are changes on vSphere)

Sometimes we see ESX hosts disconnected from vCenter. from the vpxa log files:

['App' 7644 error] [VpxdVmomi] Got vmacore exception: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

This error is common in an environment which hosts are under load.

There could be multiple causes for this kind of connection request failure:

1. Network glitch
2. proxy server on hostd is not able to respond the request due to stress.
3. hostd process is busy or crashed at the time the request arrives.

Now, vCenter has a host connection pool, it will store idle connections between ESX server and vCenter. So when there is a need to connect the server, vCenter will not need to create the connections all over again (Which in some cases we will hit the above issues). The default connection size is set at 20. you can increase this (for example to 50)

You need to add to vpxd.cfg files:


Personally, I believe this will help for a large set-up by reducing the connection requests. please note, the active connection is not put into this pool, so in most cases 20 is good enough.
Labels: 0 comments | edit post