Recently I talked about the new webhook feature in Log Insight 3.3. One thing I mentioned in that post is that without native support, you need to write shims to translate events into a format that the remote destination understands. In this post, I will demonstrate a sample shim as well as provide the code so you can play with it yourself. Read on to learn more!
Run a web server
To start, you will need a web server that you can configure Log Insight to point to. You can either leverage an existing one, stand a new one up and write some glue to point to your shim or you can just build it into your shim. The example shim includes a web server to make it really easy to deploy and configure.
Ability to test
In order to understand the Log Insight format and perform ingestion testing with your shim, it is helpful to have a test URL that will just dump the payload it receives. In the case of Log Insight webhooks, the body is the only part of the payload we really care about. In addition to handling the incoming payload from Log Insight, you will also need to see the transformation of the data as it works its way through the shim and of course the output of the translation which represents the payload that will be sent to the destination. All this means is that proper logging should be added to the shim.
Supported destinations
Finally, you need to decide which destinations you wish to support. For this example shim, the following destinations are supported:
- Socialcast
- Slack
- PagerDuty
Of course you are welcome to add destinations, which should be easy given the test URL, existing logging, and example destinations above. If you do add destinations, please be sure to share!
The shim
First off, I would like to thank Alan Castonguay for all of his work on the shim as well. He started this project, I added to it and we collaborated toward the end.
Once you download the shim, you will need to adjust the variables at the top. By default, the shim uses port 5001 for the web server, but this can be changed as desired. Depending on the destinations you wish to enable, you may need to enter the appropriate URL variables defined (if you attempt to leverage a destination that is not configured, the shim will let you know). With the variables defined, you just fire up the shim and then you are ready to send traffic to it.
With it running, go to http://<IP>:<PORT> in a browser for important information about the shim. I have also included it below.
This is a demo shim for a VMware vRealize Log Insight 3.3 or newer webhook.
IMPORTANT: It is NOT SUPPORTED to run this shim on the Log Insight virtual appliance!!
This shim features translations for:
- Socialcast (NOTE: payload must be under 1MB or webhook will fail)
- /endpoint/socialcast — Requires specifying a SOCIALCASTURL and sends up to 10 events
- /endpoint/socialcast/NUMRESULTS — Requires specifying a SOCIALCASTURL and passing the NUMRESULTS to be forwarded
- Slack (/endpoint/slack)
- /endpoint/slack — Requires specifying a SLACKURL and sends up to 10 events
- /endpoint/slack/NUMRESULTS — Requires specifying a SLACKURL and passing the NUMRESULTS to be forwarded
- PagerDuty (NOTE: SERVICEKEY can be email address or integration key for service)
- /endpoint/pagerduty/ — Requires passing a SERVICEKEY in the URL
- Test
- /endpoint/test — Does not require anything and just dumps the body of the POST event
IMPORTANT: This code is current in Beta status. Known issues:
- This shim is very trusting; does not guarantee the notification source identity or require any shared secret
- This shim does not convert HTML encoding so results may not appear on the destination in the desired output
- This shim does not retry to send an event, it is fire and forget
- No tests to ensure this shim respects the APIs it is calling
- This shim has only partial exception handling and logging
As you can see, you must specify which URL to leverage and each has different options. Please try it out and let us know what you think. The latest version can be found here.
© 2016, Steve Flanders. All rights reserved.
