Have you ever seen ESXi syslog events that have a the hostname field set to localhost no matter how the ESXi host or the syslog server is configured? Are the ESXi hosts experiencing this issue using AutoDeploy? If so, this article is for you!
Example
ESXi logs should look similar to the following:
2014-01-27T14:08:33.364Z esx03.matrix Vpxa: [FFF06B70 verbose 'VpxaHalCnxHostagent' opID=WFU-ed3fb256] [WaitForUpdatesDone] Starting next WaitForUpdates() call to hostd
The format of ESXi logs typically follows: <timestamp> <hostname> <appname>: <message>. The <hostname> field should match the hostname configured on the ESXi host. Sometimes, ESXi logs look like the following:
2014-01-27T14:08:33.364Z localhost Vpxa: [FFF06B70 verbose 'VpxaHalCnxHostagent' opID=WFU-ed3fb256] [WaitForUpdatesDone] Starting next WaitForUpdates() call to hostd
The difference here is that the <hostname> field is set to localhost. If the hostname of your ESXi host is not localhost you may be wondering why the ESXi logs are labeled as localhost. The next question to ask yourself is are you using AutoDeploy? If the answer to this question is yes then you may have found the issue. Turns out certain version of ESXi have a bug where AutoDeploy configures syslog before configuring the hostname of the ESXi host. The result is that syslog believes the hostname of the ESXi host is localhost.
Impacted version of ESXi
- ESXi 5.0
- ESXi 5.1
Fixed versions of ESXi
The good news is that every version of ESXi has a fix out at this time:
- ESXi 5.0 update 3
ESXi hosts that are booted in stateless mode appear with name localhost in the syslog file
When a stateless ESXi hosts is rebooted and the host is configured to obtain DNS configuration and host name from a DHCP server, the syslog file displays the host’s name as localhost instead of the host name obtained from the DHCP server. As a result, for a remote syslog collector, all ESXi hosts appear to have the same host name.
This issue is resolved in this release. - ESXi 5.1 patch 2 – while the bug appears unrelated, I have confirmed this does address the issue found in stateless ESXi configurations.
PR998848: Upon reboot, ESXi 5.1 hosts configured to obtain DNS configuration and host name from a DHCP server displays its host name as localhost in syslog rather than displaying the host name obtained from the DHCP server.
As a result, for a remote syslog collector, all ESXi hosts appear to be the same, with the same host name.
Workaround
If you are not running one of these ESXi releases there is still a workaround. Once the ESXi host comes online you can restart the syslog process on the host (see http://kb.vmware.com/kb/2003127 for directions or if using Log Insight use vSphere integration). This workaround addresses the issue as the hostname has already been set on the ESXi host so syslog can now reference the proper hostname when sending syslog events. One important caveat to this workaround is that it only applies while the host remains online. If/When the host redeployed is powered off or restarted the issue will arise again as the AutoDeploy configuration is applied.
© 2014, Steve Flanders. All rights reserved.
