As you may know, there are several syslog RFCs. RFC 6587 covers, amongst other things, something called octet-framing. Previous versions of Log Insight only supported non-transparent-framing. Log Insight 4.0 adds support for octet-framing. Read on to learn more!
The Problem: Non-transparent-framing
Up until Log Insight 4.0, Log Insight supported non-transparent-framing of syslog messages sent over TCP. You can read all the details about non-transparent-framing in RFC 6587 section 3.4.2, but in short messages end with a TRAILER character. Also, a transport receiver can assume that non-transparent-framing is used if a syslog frame starts with the ASCII character “<” (%d60). Non-transparent-framing use to be the standard, but has been known to cause problems. The most notable issue is the TRAILER character is not escaped within the message.
IMPORTANT: This problem has to do with syslog messages sent over TCP only. If the message is not sent over syslog or is not sent over TCP then this issue is not applicable.
The Solution: Octet-framing
To mitigate the issues discovered with non-transparent-framing, octet-framing was introduced. You can read all the details about non-transparent-framing in RFC 6587 section 3.4.1, but in short messages start with a message length digit. A transport receive can assume that octet-framing is used if a syslog frame starts with a digit.
IMPORTANT: This problem has to do with syslog messages sent over TCP only. If the message is not sent over syslog or is not sent over TCP then this issue is not applicable.
Log Insight < 4.0 + Octet-framing
It turns out that prior to Log Insight 4.0, octet-framing was not supported. An example of this can actually be seen with Synology DSM logs. If you configure Synology DSM to send over to Log Insight over TCP with RFC 5424 format then here is an example of how some of the messages appeared prior to Log Insight 4.0:
Notice how multiple messages are being treated as a single event.
Note: This issue is not isolated to RFC 5424 and applies to RFC 3164 as well. The important part is TCP.
Log Insight >= 4.0 + Octet-framing
The solution was simply to add support for octet-framing. No configuration changes necessary — simply upgrade to Log Insight 4.0 and octet-framing works. Here is the same example of Synology DSM messages in Log Insight 4.0:
IMPORTANT: Log Insight does not claim support for any RFCs at this time. It should work for RFC 3164 and RFC 5424 except section 5.1 now that octet-framing in RFC 6587 is supported. No such claim can be made for RFC 6587 (except octet-framing) RFC 5425, or RFC 5426.
While I am showing off the Synology content pack — just a heads-up that it was updated the end of last year to support DSM 6.0. Be sure to check it out!
© 2017, Steve Flanders. All rights reserved.
