Log Insight Query API: Use Cases, Basics, Caveats, and Best Practices

In my last post, I covered the new query API available in Log Insight 3.3. If you read the post then you can see there is a lot of information and while some examples were provided, it may not be easy to understand the basics of the API. In this post, I would like to focus on the basics as well as use cases, caveats, and best practices. Read on to learn more!

li-logo

Use Cases

  • Programmatically query data out of Log Insight
  • NOT to query data out to perform functions Log Insight already provides in another tool (e.g. UCOUNT)
  • NOT to query data out to perform functions Log Insight already provides in order to alert or remediate issues
  • NOT to mass export data from Log Insight
  • NOT to forward events to another tool

In short, whenever you can use Log Insight’s own capabilities you should.

Basics

  • You must authenticate first
  • The query API requires at least a role with user privileges
  • All query API requests must be made over https
  • Two headers must be passed with every query API request
    • Content-type: application/json
    • Authorization: Bearer <sessionID>
  • All query API requests are GETs with no body sent
  • Two query API URLs are available
    • /api/v1/events
    • /api/v1/aggregated-events
  • The query API defaults to simple and fast queries:
    • Events: up to 100 events from the last minute, with a 30 seconds timeout
    • Aggregated events:  as above, with 5 second time bins and the COUNT aggregation function
  • If a time range different from one minute is desired than a timestamp in epoch ms must be passed (e.g. …/timestamp/1461529204894/…)
  • To change the timeout pass the timeout option (e.g. …&timeout=300&…)
  • Everything must be URL-encoded (e.g. for “/api/v1/foo/> 10” the actual URL must be “/api/v1/events/foo/%3E%2010” or “/api/v1/events/foo/%3E+10”)
  • The query API has the same limits as the UI in terms of number of results returned

Caveats

  • The query API does not support group-by or order-by clauses as of 3.3
  • The query API does not support content pack fields as of 3.3
  • The query API does not allow for the same filter (e.g. text) to be ANDed together (OR only) as of 3.3
  • The query API does not allow for two different filters (e.g. text and hostname) to be ORed together (AND only) as of 3.3

Best Practices

  • Limit the number of concurrent queries issues via the query API
  • Ensure any query API code closes connections when done (as this will cancel queries that are still running)
  • Do not worry about batching (it is better to run a single query then multiple concurrent queries)
  • Just like the UI, always use the smallest time range possible
  • Be careful with the timeout parameter especially when issuing concurrent queries

© 2016, Steve Flanders. All rights reserved.

Leave a Reply