Generating Log Insight Agent Configurations with liagentify

I often get questions around Log Insight agent configuration. While the options are clearly documented, it can be time-consuming and error prone to construct lengthy agent configuration files. In this post, I will introduce a CLI utility I wrote to make this process easier and less error prone.

li-agent

The Log Insight agent configuration is made up of sections that monitor specific logging aspects on the client device. There are several important things to note about these configuration sections:

  • Each configuration section must have a unique name.
  • Today, the agent supports two types of event monitoring: winlog and filelog.
  • Winlog requires exactly one channel to monitor
  • Filelog requires a single directory to monitor (no globs)
  • By default, winlog and filelog will collect everything in the configured channel/directory
  • Winlog offers the option to limit what is collected
  • Winlog and filelog offer the ability to pass tags
  • Filelog supports three different character sets today: UTF-8, UTF-16LE, UTF-16BE

While the generate “rules” of the agent syntax are pretty easy to pick up, you may notice that multiple configuration sections may be needed to collect all of the events you care about on a client device. For example, with filelog, every directory you want to monitor events in needs its own configuration section. If you are running vSphere 6 and use the VCSA, you will notice that 28 different directories need to be monitored to collect all of the VMware specific log messages.

To make it easier to generate these configuration files, I have written a script that takes the inputs supported by both winlog and filelog today and spits out a configuration section. For example:

Now with a little CLI magic, you can easily generate configuration files for multiple directories.

Note: This magic assumes that options such as include, exclude, event_marker, charset and tags are identical

For example, let’s say I have directories /var/log/a and /var/log/b from which I wish to collect only .log files with an event_marker of ^\d. I can either write a for loop with the directories included around liagentify:

Or I can write the directories to a file and use a for loop:

As you can see, the script makes it very easy to generate agent configurations.

Important: The script to does not properly validate all inputs today. You should check the agent logs after applying the configuration to ensure you entered the correct values. For example, all winlog section and all filelog section names must be unique, however this script will allow the same name to be used more than once.

And with that, here is the script:

© 2015, Steve Flanders. All rights reserved.

Leave a Reply