In my last post, I covered the new query API available in Log Insight 3.3. If you read the post then you can see there is a lot of information and while some examples were provided, it may not be easy to understand the basics of the API. In this post, I would like to focus on the basics as well as use cases, caveats, and best practices. Read on to learn more!
Use Cases
- Programmatically query data out of Log Insight
- NOT to query data out to perform functions Log Insight already provides in another tool (e.g. UCOUNT)
- NOT to query data out to perform functions Log Insight already provides in order to alert or remediate issues
- NOT to mass export data from Log Insight
- NOT to forward events to another tool
In short, whenever you can use Log Insight’s own capabilities you should.
Basics
- You must authenticate first
- The query API requires at least a role with user privileges
- All query API requests must be made over https
- Two headers must be passed with every query API request
- Content-type: application/json
- Authorization: Bearer <sessionID>
- All query API requests are GETs with no body sent
- Two query API URLs are available
- /api/v1/events
- /api/v1/aggregated-events
- The query API defaults to simple and fast queries:
- Events: up to 100 events from the last minute, with a 30 seconds timeout
- Aggregated events: as above, with 5 second time bins and the COUNT aggregation function
- If a time range different from one minute is desired than a timestamp in epoch ms must be passed (e.g. …/timestamp/1461529204894/…)
- To change the timeout pass the timeout option (e.g. …&timeout=300&…)
- Everything must be URL-encoded (e.g. for “/api/v1/foo/> 10” the actual URL must be “/api/v1/events/foo/%3E%2010” or “/api/v1/events/foo/%3E+10”)
- The query API has the same limits as the UI in terms of number of results returned
Caveats
- The query API does not support group-by or order-by clauses as of 3.3
- The query API does not support content pack fields as of 3.3
- The query API does not allow for the same filter (e.g. text) to be ANDed together (OR only) as of 3.3
- The query API does not allow for two different filters (e.g. text and hostname) to be ORed together (AND only) as of 3.3
Best Practices
- Limit the number of concurrent queries issues via the query API
- Ensure any query API code closes connections when done (as this will cancel queries that are still running)
- Do not worry about batching (it is better to run a single query then multiple concurrent queries)
- Just like the UI, always use the smallest time range possible
- Be careful with the timeout parameter especially when issuing concurrent queries
© 2016, Steve Flanders. All rights reserved.
