While 95% of what you need to do on a Log Insight instance can be done from the UI, a few things, such as advanced configuration changes, required access to the CLI. In Log Insight 3.0 advanced configuration changes must now be done in a different way. Read on to learn more!
Examples
You might be wondering what types of advanced configuration changes I am referring to. One example is configuring specific Active Directory domain controllers for Log Insight to poll. While connecting to an Active Directory domain as well as adding user and groups can be done from the UI, specifying domain controllers can only be done from the CLI as of Log Insight 2.5. This example and others can be found on http://kb.vmware.com.
New in 3.0
In 3.0, using the CLI for advanced configuration changes IS NO LONGER SUPPORTED!! Instead, the UI must be used. To access the advanced configuration section within the UI you must have a role of View Admin (read-only) or Super Admin (read-write) and must manually specify a URL of https://<loginsight>/internal/config. Upon doing so you will see something similar to:
Now, you can simply add your desired advanced configuration — as specified in a KB article — within the text box and then select Save. Note that playing with these configuration options is NOT SUPPORTED UNLESS explicitly stated in a VMware KB article — a very clear warning is listed at the top of the page.
Summary
In Log Insight 3.0, advanced configuration changes must be performed from the UI — CLI changes are not supported and may have undesired results.. Making advanced configuration changes requires a role of Super Admin and a manual URL of /internal/config. Advanced configuration changes are only supported when mentioned in official VMware sites such as KB articles.
Please note that all existing KB articles have been updated to reflect the changes required in Log Insight 3.0. If you already made changes via the UI in a version of Log Insight before 3.0 and then upgrade to 3.0 your changes will be preserved and you do not need to go the UI to make any additional changes.
© 2015, Steve Flanders. All rights reserved.
Hi,
I have Log Insight 3.0 windows agent installed on my machine. I’m facing difficulty in parsing the logs which are not structured with the same delimiter(like Tab, single space, semicolon etc) across the log line. Please refer below example where fields are separated by a single Tab character or single space or the combination of space characters and Tab characters. I’m aware that I can extract these fields from the Log Insight Server user interface, but I need this at agent side.
By using new Client-Side parsing feature in 3.0, how can I extract fields before agent sends the logs to Log Insight Server.
Following is my log sample.
2015/10/01 07:01:55.326 HOSTNAME a8d89e97-a45e-4073-8811-4d67e563ad3d [0000000-0000000] (42) Debug TestNamespace.TestClass Hello World! This is my first Log Insight Test
How can I extract the fields from the above log line? I tried using csv parser but didn’t succeed. Can I pass the delimiter as “\s+” so that it matches any space between the fields? I tried it but it is not working may be regex is not allowed here, even though if it accepts “\s+” this will fail at the last part of the message as logmessage can contain spaces.
[filelog|MyTest]
enabled=yes
directory=F:\Logs
event_marker=^\d{2}\/\d{2}\/\d{4}\s\d{2}:\d{2}:\d{2}|^\d{4}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2}\.\d{3}
tags={“Provider1” : “MyTest”}
parser=myparser
[parser|myparser]
base_parser = csv
fields = timestamp, host, activityid, deviceid, threadid, loglevel, logger, logmesage
delimiter = ” ”
field_decoder={“timestamp”: “tsp_parser”}
[parser|tsp_parser]
; timestamp is a built-in parser
base_parser=timestamp
; “format” is an option of timestamp parser
format=%Y/%m/%d %H:%M:%S
Can any parser provide me an option to specify the format(similar to timestamp format) of my log message or format which accepts regex for matching log message?
Or is there any way to extracts fields from these kind of logs. Any help is highly appreciated.
Thanks in advance!
Hey Mohan — Thanks for the comment! CSV requires matching fields across events if you have more or less fields defined then are actually in the event then the CSV parser will fail. I plan to do posts on the agent parsers soon. In the meantime, take a look at the CLF parser as it should allow you to do what you want. For example:
[parser|myparser]
base_parser=clf
format=%t %i %{uuid}i [%{thread}i] (%{code}i) %M
Thanks Steve for the quick reply. I’ll try using CSV parser and will get back to you if I have any further queries.
Hey Steve,
I tried CLF parser, but I’m still facing issue with my log lines format.
I’m using the following format which is separated by Tab character, but it doesn’t work always because the gap between “threadid” to “loglevel” and “loglevel” to “logger” is not constant, it wont be tab character only because of the number of characters varies and extra spaces are added.
format=%t %i %{activityid}i [%{userid_deviceid}i] (%{threadid}i) %{loglevel}i %{logger}i %M
Example:
2015/10/01 07:01:55.326 HOSTNAME a8d89e97-a45e-4073-8811-4d67e563ad3d [0000000-0000000] (42) Debug TestNamespace.TestClass Hello World! This is my first Log Insight Test
2015/10/01 07:01:55.326 HOSTNAME a8d89e97-a45e-4073-8811-4d67e563ad3d [0000000-0000000] (2) Info TestNamespace.TestClass Hello World! This is my first Log Insight Test
Difference “(42) Debug” and “(2) Info”
Can the format variable accept regex to match spaces and tab characters something similar to the following format?
format=%t\s+%i\s+%{activityid}i\s+[%{userid_deviceid}i]\s+(%{threadid}i)\s+%{loglevel}i\s+%{logger}i\s+%M
Please let me know you inputs.
Regex is not supported today — you could use nested parsers to handle the two cases — see the documentation for an example
Hi Steve,
Sorry for posting my question here, I tried to post this question on below community forum, but i was not able to login to website due some issue.
https://communities.vmware.com/community/vmtn/vcenter/vcenter-log-insight?src=vmw_so_vex_sflan_191
Regards,
Mohan G
Just FYI, you could also try https://loginsight.vmware.com