Many requests have come in for a dormant host alert in Log Insight. I am happy to announce that the feature is available in Log Insight 4.6. Read on to learn more!
What is a Dormant Host?
The request is as follows — If I have a host that is logging to Log Insight and then it stops logging to Log Insight for some period of time, then I would like to receive an alert. This means that any host who has logged to Log Insight is no longer considered a dormant host.
How to Configure Alert
Dormant host alerts are configured from the Hosts (/admin/hosts) page:
As you can see, there is an “Inactive hosts notification” checkbox that can be selected:
Upon selecting it, you will notice it defaults to alerting based on devices that have not logged in the last day. This can, of course, be adjusted as needed down to the last 10 minutes. Like with event-based email alerts, you will notice that the inactive host notification time range dictates the frequency at which the alert is triggered (for the last day, it is every hour).
By default, if you enable inactive hosts notification, it applies to all hosts logging to Log Insight. You can choose the whitelist hosts with the whitelist checkbox:
If you select the “i” icon, it will tell you it takes a comma-separated list of hosts. Note the hostnames provided must be complete. Partial and glob hostnames are not allowed at this time. Whitelisting helps dynamic environments like dev/test or when you only care about certain hosts going offline.
Finally, you will notice after selecting the inactive host notification checkbox that next to the filter option, you get a “show only inactive” checkbox:
As the name implies, if you select it, then you will only see inactive hosts in the table.
IMPORTANT: You must hit save after enabling the “inactive host notification” checkbox before the “show only active” checkbox will do anything.
What do you get?
The notification is a system notification, so it depends on what you have configured under the Alerts section of the General (/admin/general) page.
NOTE: The first time you enable this feature, you will receive an email with all currently dormant hosts.
A sample email looks like the following:
NOTE: It is not uncommon to see weird hostnames. Remember, LI follows the syslog RFC so whatever word is where the hostname should be will be treated as the hostname.
Summary
You asked, Log Insight listened! Now you know when hosts are no longer reporting to Log Insight. This is a really cool feature! What do you think?
© 2018 – 2021, Steve Flanders. All rights reserved.
Again: Thank you for this cool article, Steve.
I just activated the alert and found orphaned servers that logged once. How can they be deleted from Log Insight to clean up the domant servers list?
E.g. we had an orphaned DNS entry for an IP address of an existing ESXi host. Hence, I see this orphaned server name in the list.
Or I see several of our ESXi hosts that reported with their IP addresses instead of their names more than two monts ago (I don’t know how this happened). How to purge them?
My favorite is the hostname called “last” in the /admin/hosts list. What’s that? I do not know.
The first time you enable the feature it will notify you of all dormant hosts — it will not notify you again unless they become un-dormant and then dormant again (I will update the article). Regarding random names like “last”, remember LI follows the syslog RFC so this means you received an event where the word “last” was where the hostname was supposed to be.
Thank you. I got the reduced dormant server list over time. And concerning “last”: Yes, it seems that this was a malformed event. What else …
finally, I like it!
Webook Shim to auto remediate the dormant host! 🙂
Totally possible since system notifications can be sent via webhook!
Hi Steve,
Does the agents/hosts are cleaned up from the UI console automatically if not active for a specific duration? For example: I have installed the Log Insight agent on a server and now it has been decommissioned, so would Log Insight automatically clean up the inactive agent from the UI console?
As I recall, the entry remains in the table until the retention period has expired. So if you have 30 days of retention then you would need to wait at least 30 days after the last event for the given entry was received.