A long time ago, I talked about an internal error message I received on ESXi. The workaround was to reboot the ESXi host, which is not the best outcome in my opinion. Recently, I hit this issue again, but this time specific to trying to configure remote syslog. I saw this issue will configuring vSphere integration on Log Insight and would like to walk through the steps I took to address the issue.
I started by entering the information for a vCenter Server instance into the vSphere integration section of my Log Insight instance. I selected the checkbox to configure the ESXi hosts and select save. When the operation finished, I was presented with the following error message:
Now, you may think this issue is Log Insight related. As such, you may go check the vSphere integration log on the virtual appliance and see a message like the following:
[com.vmware.loginsight.vsphere.config.VimVsphereConfigurer] [Error while setting syslog option: javax.xml.ws.soap.SOAPFaultException - A general system error occurred: Internal errorstr=A general system error occurred: Internal error; cod=ServerFaultCode; act=null; det[0]=[SystemErrorFault: null]javax.xml.ws.soap.SOAPFaultException: A general system error occurred: Internal error"
While this may look like a Log Insight issue, the problem is actually with ESXi. If you go to ESXi, you can run an esxcli command to check the current configuration of remote syslog:
# esxcli system syslog config get Default Network Retry Timeout: 180 Local Log Output: /scratch/log Local Log Output Is Configured: false Local Log Output Is Persistent: true Local Logging Default Rotation Size: 1024 Local Logging Default Rotations: 8 Log To Unique Subdirectory: false Remote Host: <none>
Hmm, but it just looks like Log Insight failed to configure the ESXi host. Well, looks can be deceiving. Have a look at the hidden syslog log file on the ESXi host for more information:
# tail /var/log/.vmsyslogd.err 2014-05-08T11:00:42.075Z vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: loginsight.interopad.com 2014-05-08T11:00:42.075Z vmsyslog : ERROR ] Exiting with parse failure 2014-05-08T11:20:24.174Z vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: loginsight.interopad.com 2014-05-08T11:20:24.175Z vmsyslog : ERROR ] Exiting with parse failure 2014-05-16T10:05:27.939Z vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: loginsight.interopad.com 2014-05-16T10:05:27.940Z vmsyslog : ERROR ] Exiting with parse failure 2014-05-16T11:31:42.960Z vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: loginsight.interopad.com 2014-05-16T11:31:42.960Z vmsyslog : ERROR ] Exiting with parse failure 2014-05-16T11:56:50.774Z vmsyslog.config : ERROR ] Config error: Failed to resolve remote host: loginsight.interopad.com 2014-05-16T11:56:50.775Z vmsyslog : ERROR ] Exiting with parse failure
OK, so appears either the FQDN is incorrect, DNS is misconfigured on the ESXi host, or DNS is misconfigured in the environment. Why didn’t Log Insight just say that?! Log Insight can only return to you what it receives from the vSphere API. In this case, an internal error is thrown. In short, you are looking at a limitation of ESXi. For more troubleshooting of syslog on ESXi, see this post.
© 2014 – 2021, Steve Flanders. All rights reserved.
Thanks much Steve – you helped me fix LogInsight in my home lab today!
Glad this post helped!
That’s hilarious. Could’ve saved so much time if LI just reported it as a DNS error D:
Thanks for this blog!
Hey Dave — Thanks for the comment. Unfortunately, the problem may not always be DNS so LI cannot make that assumption. What would be great is if ESXi reported this error properly via its API — then LI could display what ESXi presented.
Thanks a lot! this really helped me sort out the issue.
Great to hear!
Just wanted to throw this out there – I had similar output: 2017-08-14T17:44:44.004Z vmsyslog.config : ERROR ] Config error: Fail ed to resolve remote host: conwaylogs1.conway.local. Thing is, that DNS is no longer available – the zone doesn’t even exist. Turns out syslog was configured in the host from a previously LI deploy and the new LI instance will not overwrite what’s there. If you clear that field out through vCenter then all is well.
Thanks for sharing.