Skip to content

Tag: syslog

Configure Remote Syslog on VMware Products

Many VMware products offer remote syslog functionality, but some do not include all the logs that you may care about. In addition, some VMware products do not support remote syslog (e.g. VMware products like vCenter Server that run on Windows as Windows does not natively support syslog – more on this in a later post). If you are looking to collect logs from VMware products it is important to understand where the log files are located. Below you will find the appropriate log locations for many VMware products. A big thanks to my colleague Michael White for putting together this list and in particular the vCAC information!

Configuring Remote Syslog on vSphere Using vCenter Log Insight

Now that you have vCenter Log Insight deployed and configured, you need to configure sources to send their log messages to Log Insight. If you enabled vCenter Server integration then you should start to receive log messages, but this is only the beginning of configuring remote syslog sources.

IMPORTANT: Configuring vCenter Server integration does not configure vCenter Server or ESX/ESXi hosts connected to vCenter Server to send remote syslog to Log Insight. For more information continue reading!

Let’s start by going over what Log Insight supports today and what mechanisms it has to configure vSphere devices for you.

Automating VCSA Configuration

If you have deployed the VCSA a couple of dozen times like me then you quickly realize that it is necessary to script the initial configuration of the device. I would highly recommend taking a look at William Lam’s blog for some great setup scripts including:

Something that I noticed was missing from William’s scripts was the ability to configure application layer services such as NTP and Syslog. As such, I put together a couple quick scripts shared below.

Corosync: unknown error

For environments where the use of a load balancer is not feasible because of say cost or complexity I often use Corosync to provide similar functionality. Corosync is a piece of software that allows for clustering of an application to provide high availability. One issue I have often experienced with the tool is that the error messages are not descriptive making troubleshooting difficult.
As an example, I have used Corosync to cluster syslog servers in the past. In one such environment I had a pair of syslog servers in an active-standby configuration with a VIP. While the VIP came up as expected the syslog server reported an unknown error as shown below.

[email protected]:/home/test$ sudo crm status
Last updated: Wed Jan 23 00:00:47 2013
Last change: Tue Jan 22 23:49:38 2013 via cibadmin on log01
Stack: openais
Current DC: log01 - partition WITHOUT quorum
Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, 2 expected votes
4 Resources configured.
Online: [ log01 ]
OFFLINE: [ log02 ]
Resource Group: log_svr
vip (ocf::heartbeat:IPaddr2): Started log01
Failed actions:
log_svc:0_start_0 (node=log01, call=6, rc=1, status=complete): unknown error

So what was causing the error and how can you clear it up?

Configuring syslog on ESXi

I was assigned an interesting problem a few weeks back. A customer had requested that all ESXi servers have syslog configured in order to troubleshoot a potential bug in ESXi. A technician was assigned the case and configured all ESXi hosts to point to the syslog server on the standard port. The problem was the logs were not being seen on the syslog server. I was asked to figure out why the configuration was not working as expected.
In our particular case, all hosts pointed to a syslog VIP, which was responsible for load balancing syslog requests to a pool of syslog servers. Initially, I checked the load balancer to see if the syslog traffic was making it to the VIP. As it turned out, it was. After confirming the load balancer was working as expected, I began to suspect the configuration on the syslog servers. The only thing that I could think of which would prevent the syslog server from accepting syslog messages from the ESXi hosts was ACLs. Looking at the syslog configuration, I confirmed that the ACLs were set to allow traffic from the VMkernel VLAN configured for management traffic.
Why were the syslog messages not being received?