Log Insight Agent: Building a Configuration for Product X

A covered how to configure the free Log Insight Windows agent when the Log Insight 2.0 beta was released. Since the agent’s announcement, the demand for the agent and the appropriate configuration has increased daily. The most common request I have received to date is, “how do I monitor logs for product X using the Log Insight Windows agent?” In this post, I would like to take a deeper look at the filelog configuration option of the agent and how to approach building configuration sections.

Documentation

The official Log Insight documentation covers how to configure file collection here.

Basics

The Log Insight Windows agent works based off of a configuration file. The configuration file is made up of a variety of different sections, each denoted by a section header in square brackets, and each section defines some parameters to adhere by. The agent supports both client-side and server-side configurations. The client-side configuration is stored in:

C:\ProgramData\VMware\Log Insight Agent\liagent.ini

NOTE: that C:\ProgramData is typically a hidden directory so be sure to show hide files and folders beforehand) while the server-side is stored in the agent’s database.

The first section of the configuration file is the [server] section, which must be defined and can only be controlled on the client-side. Next is a [storage] section followed by three default [winlog] sections to collect the default Windows event viewer logs. On either the client-side or the server-side the [storage] and [winlog] sections can be modified. In addition, new [winlog] and [filelog] sections can be created. Since this post is about monitoring log files, the [filelog] section will be discussed in depth.

Format

The [filelog] configuration section uses the following format:

[filelog|<uniquename>]
directory=<directory>
include=<globfilename1>;<globfilename2>
exclude=<globfilename3>;<globfilename4>
event_marker=<regex>
charset=<UTF-8|UTF-16LE|UTF-16BE>
tag={"<tag_name1>":"<tag value 1>","<tag_name2>":"<tag value 2>" }
enabled=<yes|no>

Important notes

• The directory option does not support globs. This means a unique section will need to be created for every directory.
• By default, include is set to * or all files in the directory. It is important to note that the include option is case sensitive.
• Exclude is powerful for temporary files (e.g. *.swp on VIM)
• Event_marker is a Perl-based regex used to properly handle multiline messages. It is highly recommended that you configure this.
• Charset defaults to UTF-8.
• Tag is optional and gives you the ability of adding static fields outside of an event during ingestion (similar to the source field on syslog messages).
• Enabled defaults to yes and is a way to keep a configuration section without running it.
• If a specified directory or include does not exist when the agent is started, the directory or include is ignored even if added after the agent is started. If added after the agent is started, the agent must be restarted to start collecting logs from the directory or include.
• The agent only collects changes to files found in the include and will not send already existing messages found in the include.

Real Examples

Based on the above format, a [filelog] section could look as simple as:

[filelog|myLog]
directory=C:\Temp\myLog

Or it could be more complex:

[filelog|myLogs]
directory=C:\Temp\myLogs
include=myLog-*.log
exclude=myLog-error-*.log;myLog-warning-*.log
event_marker=^\d{4}-\d{2}-\d{2}[A-Z]\d{2}:\d{2}:\d{2}\.\d{3}

Product example

As you can see, the configuration is very straightforward and once it is created it remains mostly static. The question becomes, how do you go about building a configuration for product X? For the purposes of demonstration, I will use VMware vCAC. There are multiple Windows components for vCAC and each has its own log file. How many log files are we talking about? About six files for components and then one file per agent where most environments have two or more agents. When looking at the different vCAC logs you will notice that agent logs are different from the rest of the vCAC logs. The basic formats of each are defined below.

Agents

• The include is the standard agent include
• No need to exclude
• Marker needs to follow pattern: [7/11/2014 4:04:44 PM]

Everything else

• The include is always *All.log or Repository.log
• No need to exclude
• Marker needs to follow pattern: [UTC:2014-03-23 15:33:42

Building a filelog configuration

Knowing the directories and files you want to monitor as well as the standard format of the logs, all you need to do is build the regex for the event_marker and you are ready to create a configuration.

Configuration

IMPORTANT: Below is a sample configuration for vCAC. Please note that this configuration may need to be modified for your environment. For example, you may have selected a different drive to install vCAC on. In addition, the agent directory can be specified during installation so you may have other/different directories that need to be monitored. The below configuration can be applied on the client-side or the server-side and it does not matter if you have a distributed vCAC installation or not. Remember to restart the Log Insight Windows Agent service if applying the configuration client-side.

; ======
; Agents
; ======
; You may have more or less agents and the agent directory is specified during installation
; Modify this section as applicable to your environment
[filelog|vcac-agent-vsphere]
directory=C:\Program Files (x86)\VMware\vCAC\Agents\vsphereagent\logs\
event_marker=\[\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{1,2}:\d{1,2}
[filelog|vcac-agent-vcns]
directory=C:\Program Files (x86)\VMware\vCAC\Agents\vcns\logs\
event_marker=\[\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{1,2}:\d{1,2}
[filelog|vcac-agent-nsx]
directory=C:\Program Files (x86)\VMware\vCAC\Agents\nsx\logs\
event_marker=\[\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{1,2}:\d{1,2}
[filelog|vcac-agent-vdiagent]
directory=C:\Program Files (x86)\VMware\vCAC\Agents\VDIagent\logs\
event_marker=\[\d{1,2}/\d{1,2}/\d{4} \d{1,2}:\d{1,2}:\d{1,2}
; =======
; Servers
; =======
; Legacy location for vCAC 5.x - will not work with content pack
[filelog|vcac-demor]
directory=C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\DEMOR\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
; Legacy location for vCAC 5.x - will not work with content pack
[filelog|vcac-demwr]
directory=C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\DEMWR\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
[filelog|vcac-dem]
directory=C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\dem\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
[filelog|vcac-deo]
directory=C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\deo\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
[filelog|vcac-server]
directory=C:\Program Files (x86)\VMware\vCAC\Server\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
[filelog|vcac-mm]
directory=C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Web\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}
[filelog|vcac-website]
directory=C:\Program Files (x86)\VMware\vCAC\Server\Website\Logs\
include=*All.log;Respository.log
event_marker=\[\w\w\w:\d{4}-\d{2}-\d{2}

Summary

As you can see, the steps to creating a Log Insight agent configuration for monitoring logs on a filesystem is very straightforward:

• Locate the directories and files you want to monitor
• Determine what files NOT to monitor
• Determine what a new event looks like in the log file
• Build a configuration section

In my next post, I will provide some additional sample configurations for common Windows applications.

© 2014, Steve Flanders. All rights reserved.