I have not talked about Log Insight querying in a long time primarily because it has not changed. One thing I touched about in my query building series was suffix searching, or put another way searching for a keyword that ends with a fixed keyword. In this post, I would like to show how you can do suffix queries in Log Insight. Read on to learn more!
What is a suffix query and when do you need it?
A suffix query is when you know what a keyword ends with, but you either do not know what a keyword starts with or it starts with multiple different things and you wish to query over all of them. For example, let’s say I name my ESXi hosts in the following format: <location>-esx<number> and I have names such as:
- us01-esx001
- eu01-esx011
In this case, I know that “esx” followed by three numbers exist. I may even know the possible location values today, but they could change in the future. What I would like to do is query for everything that contains “esx”. Log Insight only allows this if “esx” starts the keyword, which is not true in this case.
When do you NOT need a suffix query?
- When you know the prefix(s) for a keyword
- If the only dynamic part of the keyword is case sensitivity (Log Insight is case-insensitive)
How to construct a suffix query
WARNING: Suffix queries in Log Insight can be extremely slow and inefficient. Pay attention to the best practices below, test performance of suffix queries and limit or eliminate suffix queries whenever you can.
To construct a suffix query, you need to use the “matches regex” operator. Given the regex queries are the slowest queries in Log Insight, it is recommended that you provide additional context (i.e. keywords) that reduce the number of events that are returned. For example, if I only need to query for hostnames that suffix with “esx” and contain the appname “hostd” then I should add a filter for appname contains hostd to the suffix query.
Note: The less results the non-regex queries return the better as this reduces the number of events for which the regex needs to be run against. In addition to additional context, reducing the time range of the query can also help.
To create a suffix query, use a regex in front of the keywords you know. For example: \w\w\d\d\-esx\n\n\n or \S+esx\n\n\n or .*esx\n\n\n. The more specific the regex the better, so the first option would be the recommendation.
© 2017 – 2021, Steve Flanders. All rights reserved.
