Skip to content

Month: July 2014

Log Insight: Importing Existing Logs

One question I get over and over again is can you / how do you import existing logs into Log Insight? The common use-cases are:

  • Support bundle – someone has a support bundle and wants to analyze the logs
  • RCA – an existing set of logs exist and analysis to determine the root cause of an issue is desired
  • Analysis – a log analysis tool does / did not exist and analysis of previous logs is desired

So, how to you import existing logs into Log Insight?

Log Insight 2.0: Service Types

Log Insight 2.0 introduced the ability to scale-out in addition to scale-up. With this change, a Log Insight instance is capable of playing different roles or service types. These include:

  • Standalone (available in 1.x)
  • Master (new in 2.0)
  • Worker (new in 2.0)

NOTE: Service types should not be confused with node types. Every Log Insight instance (or node) is capable of running any service it is matter of what service is enabled on the node.
I would like to discuss the differences between these service types and some important considerations to keep in mind.

Time in Log Insight: Events + Timestamps + Queries

Next up in the blog series about time in Log Insight I would like to answer the question, what time is used when you issue a query to Log Insight? You might say the time the originator of the message indicated in the message itself, but this is not the case. Read on to learn more!
NOTE: This is a rather long post. If you plan on skimming, be sure to fully read and understand the Queries section below.

Log Insight: Using the Ingestion API

As I am sure you know, Log Insight 2.0 features an ingestion API, which makes it possible to ingest information without use of the syslog protocol. The API uses a JSON string to send events to Log Insight and also supports the ability to pass fields during ingestion time. An example of a JSON message would be:

{"messages":[{"text":"Hello Log Insight"}]}

Depending on your operating system, you have a variety tools to send API events like the above. For example:

Depending on the method you choose and the format in which you pass the information you will get one of the following return codes:

  • 200 OK
  • 400 Bad Request
  • 500 Internal Server Error
  • 503 Service Unavailable

Unless you receive 200 OK something is wrong that needs to be corrected. If you get 503 Service Unavailable then the issue is either server-side or network related. The 400 and 500 error codes point to a client-side error. The question becomes, how do you fix client-side errors?

Log Insight 2.0 Configuration Minimums

In addition to configuration maximums, it is important to understand configuration minimums. Again, most of this information is in the official documentation, but is not consolidated in a central location… until now!
UPDATE: Configuration minimums have been added to the Log Insight documentation starting with 3.0. This information may be out of date — please use the official documentation!

Log Insight Configuration Maximums

I have received many requests for Log Insight configuration maximums. While most of the relevant information is available in the official documentation, there is no central place to reference… until now!
UPDATE: Configuration maximums have been added to the Log Insight documentation starting with 3.0. This information may be out of date — please use the official documentation!