Now that you have vCenter Log Insight deployed and configured, you need to configure sources to send their log messages to Log Insight. If you enabled vCenter Server integration then you should start to receive log messages, but this is only the beginning of configuring remote syslog sources.

IMPORTANT: Configuring vCenter Server integration does not configure vCenter Server or ESX/ESXi hosts connected to vCenter Server to send remote syslog to Log Insight. For more information continue reading!

Let’s start by going over what Log Insight supports today and what mechanisms it has to configure vSphere devices for you.

Overview

Today, vCenter Log Insight only supports ingestion of new unstructured data via the syslog protocol. The previous sentence has many important words I would like to elaborate on:

“today” – who knows what the future holds!
“new” – while vCenter Log Insight only supports ingesting new unstructured data via the syslog protocol, it does support ingesting previously ingested and since archived data via a command line import function. While the command line import function can be used to ingest any unstructured data, it is important to note that using the import function is currently only supported for previously ingested and since archived data (more on this in a later post).
“unstructured data” – vCenter Log Insight is capable of ingesting more than just syslog messages, but the only supported way to ingest new data is via the syslog protocol.
“syslog protocol” – more specifically, the following ports and protocols are supported (same as ESXi):
- 514/udp
- 514/tcp
- 1514/tcp (ssl)

The actual configuring of remote syslog sources is independent of the Log Insight product, but Log Insight does provide mechanisms to configure some vSphere products today. These include:

vCenter Server integration – this allows for the collection of events, tasks, and alarms from the specified vCenter Server instances. Please note, this does not include logs from vCenter Server or any ESX/ESXi hosts connected to vCenter Server.
configure-esxi – this command line script is capable of configuring ESXi hosts 4.x and later connected to a vCenter Server instance to send logs to Log Insight

Let’s cover these in more detail.

vCenter Server integration

vCenter Server integration allows for the collection of events, tasks, and alarms for ingestion by Log Insight. Events, tasks, and alarms contain critical information about what is going on in your virtual environment and any issues that you should be aware of. What you may not know is that events, tasks, and alarms are not in vCenter Server logs today. Instead, events, tasks, and alarms are stored in the vCenter Server database. One of the primary reasons for wanting to extract events, tasks, and alarms from the vCenter Server database is for retention and compliance reasons. Depending on the activity on your vCenter Server instance, events, tasks, and alarms could be rotated out in a matter of days or less.
So, how does vCenter Server integration work on Log Insight? You will find a section to configure VMware integration during the initial configuration wizard

or on the Administration section in the web UI

In both areas, you have the option to configure one or more vCenter Server instances. Simply supply the hostname and credentials for a user with read-only privileges and Log Insight will take care of the rest. Be sure to test the connection before saving the changes.

If in the Administration section of the UI then after saving you must restart the Log Insight process

Remember that restarting the Log Insight process will result in certain version of ESXi 5.x to stop sending remote syslog

There are a few things you should know about vCenter Server integration:

Only supported on vCenter Server 5.1.x – connections to other versions may be possible, but polling may not work as expected
Log Insight connects to vCenter Server over the vCenter Server API every 2.5 minutes polling for new events, tasks, and alarms (no changes are made to vCenter Server by Log Insight)
vCenter Server integration does not configure the collection of vCenter Server logs (more on this in a later post)
The read-only user must be defined on the root vCenter Server object with the propagate checkbox selected

configure-esxi

configure-esxi is a command line tool provided on the Log Insight virtual appliance. The tool allows for the configuration of remote log hosts on ESXi hosts 4.x and later that are connected to a vCenter Server instance. To use the tool, log into the virtual appliance and run:

loginsight:~ # configure-esxi --help
/opt/vmware/bin/configure-esxi -u '<username>' [-p '<password>'] -s <vcserver> [-h <esxi>[,<esxi>...]] [-t <syslog> [-f] | -x <syslog> | -r | -q]
	--help                            - Print a usage message briefly summarizing these command-line options, then exit.
	-u, --username <username>         - vCenter Server username. When entering your username please note that special characters may be interrupted by
                                            the terminal. As such, it is recommended to enclose your username in single quotes.
                                            Examples: 'testuser', 'domain\testuser', 'testuser@domain'
	-p, --password <password>         - vCenter Server password. When entering your password please note that special characters may be interrupted by
                                            the terminal resulting in an incorrect password notification. As such, it is recommended to enclose your
                                            password in single quotes. If your password contains single quotes then they must be escaped with a back
                                            slash. For example, if your password is /John Doe's Pa$word/ not including the forward slashes then it should
                                            be entered as /'John Doe\'s Pa$word'/.
	-s, --server   <vcserver>         - vCenter Server hostname.
	-h, --host     <esxi>[,<esxi>...] - List of ESXi hostname separate by comma (no space). Used if you want to target specific ESXi hosts
                                            within a vCenter Server instance. Can only be used in conjunction with -s. Please note that <esxi> must match exactly what
                                            is seen in <vcserver>. For example, if <vcserver> shows esx01.local then passing esx01 will not work. In addition,
                                            configure-esxi is only supported on ESXi 4.x and greater.
	-t, --target   <syslog>           - Configure a syslog target where <syslog> = <protocol>://<hostname>:<port> | <hostname>. Please note for ESXi
                                            4.x if using <hostname> then udp://<hostname>:514 will always be set as it is the only supported configuration.
	-x, --remove   <syslog>           - Remove a syslog target where <syslog> = <protocol>://<hostname>:<port> | <hostname>. Please note that <syslog>
                                            must match exactly what -q returns. For example, if -q shows udp://loginsight01.local:514 then passing
                                            loginsight01.local will not work.
	-r, --reload                      - Reload syslog configuration. This may be required for certain versions for ESXi 5.x that stop logging to remote
                                            destinations if a remote destination becomes unavailable. For more information see http://kb.vmware.com/kb/2003127. Please
                                            note this flag only works for ESXi 5.x.
	-q, --query                       - Return current syslog settings.
	-f, --force                       - Force overwriting of ESXi 4.x settings. ESXi 4.x only allows a single remote syslog destination to be
                                            specified. If you wish to overwrite the existing entry then you must specify this flag. Can only be used in
                                            conjunction with -t.

Next, specify the appropriate flags depending on the operation you wish to perform. In general, I would recommend the following order of operations:

Order of operations

1. Query for current loghosts on ESXi hosts

loginsight:~ # configure-esxi -u root -s 192.168.1.8 -q
Password:
Querying remote log settings of hosts on vCenter Server: 192.168.1.8
NOTE: configure-esxi will work methodically but not quickly. Large inventories
or slow network connections will further slow down execution. Please be patient.
Attempting to connect to vCenter Server 192.168.1.8
Host: nesxi51-01.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "tcp://192.168.1.24:514"
Host: nesxi51-02.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "udp://192.168.1.250:514,udp://192.168.1.24:514"
Host: nesxi41-01.matrix, VMware ESXi 4.1.0, 800380
Syslog.global.logHost: "udp://192.168.1.24:514"
Host: nesx41-01.matrix, VMware ESX 4.1.0, 800380
*** WARNING: Skipping host nesx41-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: nesxi40-01.matrix, VMware ESXi 4.0.0, 504850
Syslog.global.logHost: "udp://192.168.1.24:514"
Host: nesx40-01.matrix, VMware ESX 4.0.0, 504850
*** WARNING: Skipping host nesx40-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: esx03.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "192.168.1.24,192.168.1.249"
Host: nesxi50-01.matrix, VMware ESXi 5.0.0, 469512
Syslog.global.logHost: "192.168.1.24"
Host: 192.168.1.10, VMware ESXi 5.1.0, 799733
No remote syslog target set on host 192.168.1.10

2. Add loghosts as appropriate

loginsight:~ # configure-esxi -u root -s 192.168.1.8 -t 192.168.1.23
Password:
Configuring hosts connected to 192.168.1.8 to send logs to 192.168.1.23
NOTE: configure-esxi will work methodically but not quickly. Large inventories
or slow network connections will further slow down execution. Please be patient.
Attempting to connect to vCenter Server 192.168.1.8
Host: nesxi51-01.matrix, VMware ESXi 5.1.0, 799733
*** WARNING: VMware ESXi 5.1.0, 799733 has a known issue where if a remote syslog destination becomes
*** unavailable the host will stop sending remote syslog messages until the syslog server is
*** restarted. Please note that tcp/ssl also experience this behavior.
*** See http://kb.vmware.com/kb/2003127 for more information.
Old Syslog.global.logHost: "tcp://192.168.1.24:514"
New Syslog.global.logHost: "tcp://192.168.1.24:514,192.168.1.23"
Changes successful.
Host: nesxi51-02.matrix, VMware ESXi 5.1.0, 799733
*** WARNING: VMware ESXi 5.1.0, 799733 has a known issue where if a remote syslog destination becomes
*** unavailable the host will stop sending remote syslog messages until the syslog server is
*** restarted. Please note that tcp/ssl also experience this behavior.
*** See http://kb.vmware.com/kb/2003127 for more information.
Old Syslog.global.logHost: "udp://192.168.1.250:514,udp://192.168.1.24:514"
New Syslog.global.logHost: "udp://192.168.1.250:514,udp://192.168.1.24:514,192.168.1.23"
Changes successful.
Host: nesxi41-01.matrix, VMware ESXi 4.1.0, 800380
*** WARNING: Skipping ESXi 4.x host nesxi41-01.matrix
*** ESXi 4.x only supports a single remote syslog target.
*** Specify -f to force overwriting of the existing configuration
Host: nesx41-01.matrix, VMware ESX 4.1.0, 800380
*** WARNING: Skipping host nesx41-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: nesxi40-01.matrix, VMware ESXi 4.0.0, 504850
*** WARNING: Skipping ESXi 4.x host nesxi40-01.matrix
*** ESXi 4.x only supports a single remote syslog target.
*** Specify -f to force overwriting of the existing configuration
Host: nesx40-01.matrix, VMware ESX 4.0.0, 504850
*** WARNING: Skipping host nesx40-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: esx03.matrix, VMware ESXi 5.1.0, 799733
*** WARNING: VMware ESXi 5.1.0, 799733 has a known issue where if a remote syslog destination becomes
*** unavailable the host will stop sending remote syslog messages until the syslog server is
*** restarted. Please note that tcp/ssl also experience this behavior.
*** See http://kb.vmware.com/kb/2003127 for more information.
Old Syslog.global.logHost: "192.168.1.24,192.168.1.249"
New Syslog.global.logHost: "192.168.1.24,192.168.1.249,192.168.1.23"
Changes successful.
Host: nesxi50-01.matrix, VMware ESXi 5.0.0, 469512
*** WARNING: VMware ESXi 5.0.0, 469512 has a known issue where if a remote syslog destination becomes
*** unavailable the host will stop sending remote syslog messages until the syslog server is
*** restarted. Please note that tcp/ssl also experience this behavior.
*** See http://kb.vmware.com/kb/2003127 for more information.
Old Syslog.global.logHost: "192.168.1.24"
New Syslog.global.logHost: "192.168.1.24,192.168.1.23"
Changes successful.
Host: 192.168.1.10, VMware ESXi 5.1.0, 799733
*** WARNING: VMware ESXi 5.1.0, 799733 has a known issue where if a remote syslog destination becomes
*** unavailable the host will stop sending remote syslog messages until the syslog server is
*** restarted. Please note that tcp/ssl also experience this behavior.
*** See http://kb.vmware.com/kb/2003127 for more information.
Old Syslog.global.logHost: ""
New Syslog.global.logHost: "192.168.1.23"
Changes successful.

3. Query for current loghosts on ESXi hosts

loginsight:~ # configure-esxi -u root -s 192.168.1.8 -q
Password:
Querying remote log settings of hosts on vCenter Server: 192.168.1.8
NOTE: configure-esxi will work methodically but not quickly. Large inventories
or slow network connections will further slow down execution. Please be patient.
Attempting to connect to vCenter Server 192.168.1.8
Host: nesxi51-01.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "tcp://192.168.1.24:514,192.168.1.23"
Host: nesxi51-02.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "udp://192.168.1.250:514,udp://192.168.1.24:514,192.168.1.23"
Host: nesxi41-01.matrix, VMware ESXi 4.1.0, 800380
Syslog.global.logHost: "udp://192.168.1.24:514"
Host: nesx41-01.matrix, VMware ESX 4.1.0, 800380
*** WARNING: Skipping host nesx41-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: nesxi40-01.matrix, VMware ESXi 4.0.0, 504850
Syslog.global.logHost: "udp://192.168.1.24:514"
Host: nesx40-01.matrix, VMware ESX 4.0.0, 504850
*** WARNING: Skipping host nesx40-01.matrix
*** ESX does not support remote configuration of syslog via an API. Consider upgrading to ESXi.
*** See http://kb.vmware.com/kb/1005030 for manual configuration.
Host: esx03.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "192.168.1.24,192.168.1.249,192.168.1.23"
Host: nesxi50-01.matrix, VMware ESXi 5.0.0, 469512
Syslog.global.logHost: "192.168.1.24,192.168.1.23"
Host: 192.168.1.10, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "192.168.1.23"

Other options

You can specify the -h flag to restrict the ESXi hosts on a vCenter Server instance to perform operations against

NOTE: It is not possible to restrict operations on a vCenter Server object like datacenter, cluster, or folder today.

loginsight:~ # configure-esxi -u root -s 192.168.1.8 -h nesxi51-01.matrix,nesxi50-01.matrix -q
Password:
Querying remote log settings of hosts on vCenter Server: 192.168.1.8
NOTE: configure-esxi will work methodically but not quickly. Large inventories
or slow network connections will further slow down execution. Please be patient.
Attempting to connect to vCenter Server 192.168.1.8
Host: nesxi51-01.matrix, VMware ESXi 5.1.0, 799733
Syslog.global.logHost: "tcp://192.168.1.24:514,192.168.1.23"
Host: nesxi50-01.matrix, VMware ESXi 5.0.0, 469512
Syslog.global.logHost: "192.168.1.24,192.168.1.23"

You can reload the syslog configuration on an ESXi hosts (important due to http://kb.vmware.com/kb/2003127)

loginsight:~ # configure-esxi -u root -s 192.168.1.8 -h nesxi51-01.matrix,nesxi50-01.matrix -r
Password:
Reloading syslog on ESXi 5.x hosts connected to 192.168.1.8
NOTE: configure-esxi will work methodically but not quickly. Large inventories
or slow network connections will further slow down execution. Please be patient.
Attempting to connect to vCenter Server 192.168.1.8
Host: nesxi51-01.matrix, VMware ESXi 5.1.0, 799733
Reloading the syslog server on host nesxi51-01.matrix
Changes successful.
Host: nesxi50-01.matrix, VMware ESXi 5.0.0, 469512
Reloading the syslog server on host nesxi50-01.matrix
Changes successful.

Important information

Only supports configuring ESXi connected to a vCenter Server instance
- ESX is not supported as ESX does not support configuring of syslog via an API
- OK is ESX and ESXi hosts reside on the same vCenter Server instance
- Directly configuring an ESXi host with configure-esxi is possible, but not supported today
Only supports ESXi version 4.x and newer
Only supports ESXi standard syslog ports (514/tcp, 514/udp, 1514/ssl)
Does not configure VMs to send remote syslog to Log Insight
Put usernames and passwords with special characters in single quotes. Run configure-esxi –help for more information.
ESXi version 4.x only supports a single remote destination
- configure-esxi has a -f flag to forcefully override an existing loghost configured on an ESXi 4.x host
- The -f flag of configure-esxi has no effect on ESXi 5.x hosts
ESXi 5.x has known issues with remote logging for both TCP and UDP (see http://kb.vmware.com/kb/2003127)
- TCP – ESXi 5.0.x and 5.1.x
- UDP – ESXi 5.0 and 5.0.1

2 comments on “Configuring Remote Syslog on vSphere Using vCenter Log Insight”

Yuval Tenenbaum

June 23, 2013 at 7:47 pm

Q- What is the value of running “configure-esxi -u root -s 192.168.1.8 -q” to Query the remote log settings of hosts on vCenter Server if we then run
configure-esxi -u root -s 192.168.1.8 -t 192.168.1.23 which Configures hosts connected to 192.168.1.8 to send logs to 192.168.1.23 regardless if they are already connect or not? Is it just FYI kind of thing?

Steve Flanders

Post Author June 24, 2013 at 4:49 pm

Hey Yuval – Great question! Yes, it is an FYI thing. One thing we have seen is that users run the command and expect configure-esxi to override whatever is currently set on the ESXi host, but this does not happen. By checking before and after the user should not be surprised by the results and if they did mean to remove the old entry they can then do so. I hope this helps!

Configuring Remote Syslog on vSphere Using vCenter Log Insight

Overview

vCenter Server integration

configure-esxi

Order of operations

1. Query for current loghosts on ESXi hosts

2. Add loghosts as appropriate

3. Query for current loghosts on ESXi hosts

Other options

Important information

Related

Steve Flanders

2 comments on “Configuring Remote Syslog on vSphere Using vCenter Log Insight”

Leave a Reply Cancel reply

Overview

vCenter Server integration

configure-esxi

Order of operations

1. Query for current loghosts on ESXi hosts

2. Add loghosts as appropriate

3. Query for current loghosts on ESXi hosts

Other options

Important information

Share this:

Related

Steve Flanders

2 comments on “Configuring Remote Syslog on vSphere Using vCenter Log Insight”

Leave a Reply Cancel reply