Heads Up! Log Insight Integration Issue When Upgrading To vR Ops 6.1

Any issue has been discovered where upgraded instances of vR Ops from 6.0.x to 6.1 that were integrated with Log Insight no longer display Log Insight notification events. Read on to learn more about the issue and how to fix it.

vrops-256

Issue

After upgrading a vR Ops instance that was integrated with Log Insight from vR Ops 6.0.x to 6.1 Log Insight notification events no longer appear. Please note this only impacts upgraded vR Ops instances and only vR Ops instances where Log Insight integration was configured before the vR Ops upgrade.

Based not seeing Log Insight notification events in vR Ops, you can also confirm the issue, but checking some Log Insight log files. First, check the /storage/var/loginsight/alert.log file to confirm alerts are actually being fired. You will need to search based on the name of the alert you configured in the Log Insight UI. The output is in JSON format is may be quite long:

Then check the /storage/var/loginsight/vcenter_operations.log which will have 500 errors like the following:

Workaround

VMware is aware of the issue and working on publishing a KB. In the meantime, I wanted to walk through the workaround with screenshots.

0. WARNING

Do not remove integration between Log Insight and vR Ops from the Log Insight Administration section. If you do then you will lose all historic notification events sent to vR Ops.

1. Get the object ID of the vRealizeOpsMgrAPI adapter instance

One way to do this is to look it up under https://<vrops>/ui/index.action#/administration/inventory-explorer:

li-vrops-ie1

Unfortunately, by default the ID is hidden and it may be hard to determine how to find the required information:

li-vrops-ie2

Also no copy option exists once the information is found:

li-vrops-ie3

The other way to do this is with a REST API call — you will need REST for the next step so I recommend this approach. First, you need to find out the API commands. I learned you can do this directly from your vR Ops instance via a URL:

li-vrops-api

Next you need to make the API call. I use the Advanced REST Client plugin for Chrome:

li-vrops-idYou will be prompted to authenticate and then you should get a 200 OK Status. The result of the command will be at the bottom of the screen:

li-vrops-id2

2. Delete the vRalizeOpsMgrAPI adapter instance

This MUST be done with a REST API call — you cannot delete from the vR Ops UI and should not attempt to delete anything from the vR Ops UI. Again we can use the browser to perform the required action using the information from the previous step:

li-vrops-id-deleteThis time you should receive a 204 No Content Status and no additional output. You can confirm the action worked by checking the vR Ops instance again. Please note it is not sufficient to simply hit the refresh button in the upper right-hand corner, doing so results in:

li-vrops-ie4Note that the line item is gone from the List, but still appears on the expanded navigation. I prefer to go to a different option — say Object Relationships — and then come back to Inventory Explorer. Then the page should look like this:

li-vrops-ie5

3. Restart the collector service on all vR Ops nodes

This needs to be done from the CLI of each vR Ops node. By default, SSH is disabled so you will need to enable it on the console of the VM by logging in and running /etc/init.d/sshd start. Then the command is:

Again, please note this MUST BE DONE ON EVERY VR OPS NODE!

4. You are done!

Log Insight notification events should now be received by vR Ops. One note is that it can take several minutes for vR Ops to properly recognize the change, Log Insight alerts will take 5 or more minutes to make it to vR Ops depending on alert threshold configuration, and vR Ops could take 5 or more minutes to process the alert sent by Log Insight. Long story short, be patient!

One final note is that the vRealizeOpsMgrAPI adapter instance will not appear until a success Log Insight notification event is processed. Once it is, you will see a new adapter instance with a new ID:

li-vrops-ie6

While I am on the topic, one of the improvements in vR Ops 6.1 is that Log Insight notification events now show with the properly set Criticality:

li-vrops-notification

A special shoot out to Pavan and everyone else who helped put together this workaround!

© 2015, Steve Flanders. All rights reserved.

2 thoughts on “Heads Up! Log Insight Integration Issue When Upgrading To vR Ops 6.1

  1. Moussa= says:

    I would juste take the time to thank you for all your precious articles !
    Thank you very much ! your blog helps me every day 🙂

Leave a Reply