Log Insight Agent : Excluding Events with Blacklist

As you probably know, the Log Insight agent allows you to explicitly include or exclude:

• filelog: log files within a directory
• winlog: Windows event viewer channels

One question I received at VMworld was if you could explicitly include or exclude:

• filelog: certain log messages from within a log file
• winlog: certain events from within Windows event viewer channels

In this post, I will provide the answer. Read on to learn more!

Background

The short answer is: YES! This is done through the use of blacklist or whitelist options in either filelog (Linux or Windows) or winlog (Windows) configuration sections for either the cfapi (default) or syslog protocols. These options are defined as follows:

• blacklist = <filter_expression>
• whitelist = <filter_expression>

So what is a filter expression? It is made up of:

<field> <operator> <value> [<and|or|&|\||^|~> <filter> <operator> <value> …]

Let’s walk through each of these parameters.

Field

A field is either an automatic field handled by the agent (I cover these fields in this blog post) or a field created via a parser.
Note: Automatic fields are really only applicable to winlog configuration sections. The reason for this is because with filelog, only client specific automatic fields exist. Leveraging a client specific field would be equivalent to explicitly including or excluding entire files and not a subset of events within a file. This means for filelog configuration sections, parsers must be used to leverage blacklist and whitelist options — more on this in a second.

IMPORTANT: You can only blacklist/whitelist based on a field. You can not blacklist/whitelist on text. Given you can create fields with parsers, you can blacklist/whitelist events within any filelog/winlog section.

Operator

The primary operators are the double equal sign (==) and the not equal sign (!=), however every field also supports <, <=, >, >=. In the latter case, the agent attempts to convert the field to an integer to perform the operation and if that fails then it performs the operation based on a string match.

IMPORTANT: You MUST use a double equal sign. The reason for this is because options specified within configuration sections use a single equal sign.

Value

When using automatic fields, the value is the field value without quotations. When using parser fields, the value is the text value wrapped in double quotations.

Blacklist

A blacklist allows you to exclude events that match the defined filter expression. Let me provide a few examples.

Automatic Field

[winlog|blacklist]
channel=Security
blacklist= EventID == 4688 or EventID == 5447

Parsed Field

[filelog|blacklist2]
directory=/tmp/blacklist
parser=blacklist-parser
blacklist= field == "bad"
[parser|blacklist-parser]
base_parser=clf
format=hello %{field}i world

Here is a sample log file that can be used against the above configuration:

$ cat blacklist.log
hello good world
hello bad world
hello good world

If you test with the same log file, you will see the “hello good world” event twice in Log Insight, but you will not see the “hello bad world” event (since it is blacklisted).

Whitelist

A whitelist allows you to include only events that match the defined filter expression. Let me provide a few examples.

Automatic Field

[winlog|blacklist]
channel=Security
blacklist= level > WINLOG_LEVEL_SUCCESS and level < WINLOG_LEVEL_INFO

Parsed Field

[filelog|whitelist]
directory=/tmp/whitelist
parser=whitelist-parser
whitelist= field == "good"
[parser|whitelist-parser]
base_parser=clf
format=hello %{field}i world

Here is a sample log file that can be used against the above configuration:

$ cat whitelist.log
hello good world
hello bad world
hello good world

If you test with the same log file, you will see the “hello good world” event twice in Log Insight, but you will not see the “hello bad world” event (since only “hello good world” is whitelisted).

Blacklist and Whitelist

Of course you can use both blacklist and whitelist in the same configuration section. Let me provide an example.

[filelog|apache]
directory=/tmp/apache
parser=apache-parser
whitelist= server_name == "sflanders.net"
blacklist= remote_host == "127.0.0.1"

Use Cases

I am sure you can think of use cases for the blacklist or whitelist options, but I figured I would share the one given to me during VMworld. The use case was web servers being load balanced from an active-active pair of load balancers. These load balancers were configured to do health checks against the web servers. These health checks generated frequent log messages about the healthy state of the web servers. The user was interested in removing this “noise”. In this case, all you need to do is introduce a parser to get a field from such health check messages and then blacklist that field.

Summary

As you can see, the Log Insight agent configuration is extremely powerful and flexible depending on your needs. Not only can it handle multiple files within a directory, it can also exclude individual events within a file.

© 2016, Steve Flanders. All rights reserved.