When architecting a syslog solution, it is important to understand the requirements both from a business and a product perspective. I would like to discuss the different remote syslog architectures that are possible when using vCenter Log Insight.
Problem Statement
For Log Insight, the problem statement is as follows. Every VMware product generates log messages. Every physical device (e.g. compute, network, storage) generates log messages. Every VM, including operating system and application, generates log messages. An environment generates a lot of log messages a day and the amount of logs increase when issues arise. Troubleshooting and finding root cause of issues is challenging unless logs can be aggregated and queried over.
Remote Syslog
Remote syslog is used for a variety of reasons including:
- Aggregation
- Querying
- Correlation
- Retention
- Security/Compliance/Auditing
In terms of Log Insight, the solution looks something like this:
When architecting a remote syslog solution, it is important to understand the requirements of the solution. This will include things such as:
- Number of devices sending events
- Amount of events generated
- Amount of retention necessary of events
- Who needs access to what events
- Security considerations
- Operating (e.g. backup, disaster recovery, performance)
In terms of syslog architecture, the two most common approaches are:
- Client-Server
- Hub-and-Spoke
I would like to cover each of these architectures as they apply to Log Insight.
Client-Server
In this architecture, every device for which you would like to collect events is configured to send events directly to one or more Log Insight instances.
Pros
- Simple architecture
- Possible to have a single pane of glass
- Only dependent on syslog agent, network connectivity, and Log Insight
Cons
- Every device needs to be configured to send logs to Log Insight
- In a greenfield environment, this is true for any architecture
- If automation and/or configuration management is being used this is a non-issue
- Pro: Log Insight handles configuration of ESXi hosts today
- Client side changes need to be made on every device
Other Considerations
- Log Insight listens on udp/514, tcp/514, and tcp(ssl)/1514 today
- Log Insight supports a maximum of 750 directly connected devices today
- Log Insight does not support relaying messages to other systems today
Summary
This option is great for the following types of environments:
- Greenfield, no syslog to date
- Leverage automation and/or configuration management
- Less than 750 devices sending remote syslog
Hub-and-Spoke
In this architecture, every device for which you would like to collect events is configured to send events to a syslog aggregator and the syslog aggregator is configured to forward events to one or more Log Insight instances.
Syslog Aggregator
A syslog aggregator is a syslog server that may process and/or store syslog events as well as forward events to another syslog server. Syslog aggregators are often used in multiple datacenter environments to reduce the likelihood of dropping events. Some examples of syslog servers that can be used as syslog aggregators are Syslog-NG and Rsyslog. I will cover syslog aggregators, including configuration, in a later post.
Pros
- Flexibility and control of what events are forwarded, the format they are forward in and where they are forwarded to
- Client side changes only need to be made to syslog aggregators instead of every device in an environment
Cons
- More complex architecture
- One or more third-party syslog aggregators need to be setup and configured
Other Considerations
- Log Insight can support up to 7,500 events per second in this configuration
- Log Insight cannot be used as a syslog aggregator today
Summary
This option is great for the following types of environments:
- Leveraging remote syslog today
- Leverage automation and/or configuration management
- Close to or more than 750 devices sending remote syslog
© 2013 – 2021, Steve Flanders. All rights reserved.
