Since most of you are upgrading form VI 3 to vSphere, I'd like to share with you on this issue I worked with one of the BCS customers.
I received this call when customer failed to add their ESX 3.5 U4 host to their new vCenter 4.0 server. The error message they hit was “A general system error occurred” (Always nice to see a message does not give any direction). So for the first step, we tested viclient connect directly to ESX server and that worked for us. Normally, that means hostd is working (This is actually the key issue). So we focused on vpxa.
from the vpxa logs:
[2010-05-20 11:29:01.793 server_name error 'App'] [VpxaHalCnxHostagent] caught InvalidProperty::Exception - filter creati
on failed, temptorary versioning workaround
[2010-05-20 11:29:01.820 server_name verbose 'App'] [VpxVmomi] Invoke error: vpxapi.VpxaService.login session: 5238dc5c-bd8b-fcdd-f450-ab23a826a413 Throw: vmodl.fault.SystemError
So we tried many things and nothing worked... I then want to see what hostd report during that time
Here is the hostd log:
[2010-05-20 13:04:36.281 'App' 52411312 error] AdapterServer caught unexpected exception: vector::reserve
[2010-05-20 13:04:36.281 'App' 52411312 trivia] Completing task haTask vim.event.EventManager.createColle
ctor-389 as 3
[2010-05-20 13:04:36.281 'PropertyProvider' 52411312 trivia] RecordOp 2: info, haTask--vim.event.EventMana
ger.createCollector-389
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Activation [N5Vmomi10ActivationE:0x9fd9780] : Invoke done
[createCollector] on [vim.event.EventManager:ha-eventmgr]
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Throw vmodl.fault.SystemError
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Result:
(vmodl.fault.SystemError) {
dynamicType =,
reason = "",
msg = ""
}
[2010-05-20 13:04:36.325 'Proxysvc Req00153' 18353072 trivia] Client HTTP stream read error
[2010-05-20 13:04:36.325 'Proxysvc Req00153' 18353072 trivia] Closed
The error message "AdapterServer caught unexpected exception: vector::reserve" lead me to a problem report. However, the report had no vCenter involved. I asked customer to bring up the ESX server via viclient, click on any virtual machine, then click on "event" tab, sure enough, it hits “A general system error occurred” message, just like the problem report. The good news is patch 19 fixed this issue. We installed patch 19 ( ESX350-200912401-BG - PATCH ) and able to add those ESX server to the new vCenter 4.0.
If you already have patch 19 installed, you should not have this issue. If you don't, then put this on the side, it would be handy in case you need it.
I received this call when customer failed to add their ESX 3.5 U4 host to their new vCenter 4.0 server. The error message they hit was “A general system error occurred” (Always nice to see a message does not give any direction). So for the first step, we tested viclient connect directly to ESX server and that worked for us. Normally, that means hostd is working (This is actually the key issue). So we focused on vpxa.
from the vpxa logs:
[2010-05-20 11:29:01.793 server_name error 'App'] [VpxaHalCnxHostagent] caught InvalidProperty::Exception - filter creati
on failed, temptorary versioning workaround
[2010-05-20 11:29:01.820 server_name verbose 'App'] [VpxVmomi] Invoke error: vpxapi.VpxaService.login session: 5238dc5c-bd8b-fcdd-f450-ab23a826a413 Throw: vmodl.fault.SystemError
So we tried many things and nothing worked... I then want to see what hostd report during that time
Here is the hostd log:
[2010-05-20 13:04:36.281 'App' 52411312 error] AdapterServer caught unexpected exception: vector::reserve
[2010-05-20 13:04:36.281 'App' 52411312 trivia] Completing task haTask vim.event.EventManager.createColle
ctor-389 as 3
[2010-05-20 13:04:36.281 'PropertyProvider' 52411312 trivia] RecordOp 2: info, haTask--vim.event.EventMana
ger.createCollector-389
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Activation [N5Vmomi10ActivationE:0x9fd9780] : Invoke done
[createCollector] on [vim.event.EventManager:ha-eventmgr]
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Throw vmodl.fault.SystemError
[2010-05-20 13:04:36.281 'Vmomi' 52411312 info] Result:
(vmodl.fault.SystemError) {
dynamicType =
reason = "",
msg = ""
}
[2010-05-20 13:04:36.325 'Proxysvc Req00153' 18353072 trivia] Client HTTP stream read error
[2010-05-20 13:04:36.325 'Proxysvc Req00153' 18353072 trivia] Closed
The error message "AdapterServer caught unexpected exception: vector::reserve" lead me to a problem report. However, the report had no vCenter involved. I asked customer to bring up the ESX server via viclient, click on any virtual machine, then click on "event" tab, sure enough, it hits “A general system error occurred” message, just like the problem report. The good news is patch 19 fixed this issue. We installed patch 19 ( ESX350-200912401-BG - PATCH ) and able to add those ESX server to the new vCenter 4.0.
If you already have patch 19 installed, you should not have this issue. If you don't, then put this on the side, it would be handy in case you need it.
