Log Insight Cluster: Data Not Balanced Across Nodes

In Log Insight 2.0 a scale-out feature was introduced. The best practice when using scale-out is to configure an external load balancer in front of the cluster and send all ingestion traffic (i.e. syslog and ingestion API) to the load balancer instead of directly to a node. The reason for this best practice is to assist with balancing data across nodes and also to provide ingestion high availability. In this post, I would like to discuss why data may not be balanced across a Log Insight cluster whether a load balancer is used or not.

unbalanced_scale

Continue reading