Log Insight 3.0 Agents: Automatic Parser

To wrap up on the parsers, I would like to cover the automatic parser. Read to learn how it works!
li-agent

How the Parser Works

The automatic parser makes it possible to parse both the timestamp as well as KVPs using the default options of both parsers.

Basic Example

No matter what the logs look like, you could use the following:

[filelog|some_logs]
directory=/var/log/test
parser=auto

You can also define a dedicated parser section — though this does not provide any additional value:

[filelog|some_logs]
directory=/var/log/test
parser=myauto
[parser|myauto]
base_parser=auto

Note you cannot change the default options for either the timestamp nor the KVP parser included in the auto parser. This means you cannot set the format of the timestamp parser or the delimiter of the KVP parser — and the KVP parser defaults to fields=*. If you need to change either than you should use the two parser separately instead of via the auto parser. As you may remember, use of the KVP parser with fields=* is not recommended unless it actually matches the fields you care about so use of this parser should be done selectively.

Summary

The auto parser makes it quick and easy to get the capabilities of the timestamp and KVP parsers. The auto parser only allows the default options of both the timestamp and KVP parsers. Note the auto parser should not be turned on for all logs as it may match events you do not intend — like logs that contain SSL certificates. Do you use the auto parser today?

© 2015, Steve Flanders. All rights reserved.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back To Top