vRA Remote Logging

My post on vCAC logging has been quite popular since its release. With VMware’s release of new and updated management products at the end of 2014, some changes to vCAC, now vRA, exist. In order to avoid confusion by attempting to update the older post, I decided it was time for a new post. Also, with the release of the Log Insight Linux agent, it is a good time to show end-to-end remote logging for vRA when leveraging the Log Insight agents.

Unfortunately, vRA still does not support setting a remote syslog destination to forward all vRA logs within the GUI yet. Like last time, I would like to cover where all the log files are located and more importantly how you can forward them to a remote syslog destination like Log Insight.

vRA Product Icon Mac_0

Log Locations

Let me start by laying out all the different components and the log locations:

  • vRA VA + vRCS
    • /var/log/vcac/catalina.out
    • /var/log/vco/app-server/catalina.out
    • /var/log/apache2/access_log
    • /var/log/apache2/error_log
    • /var/log/apache2/ssl_request_log
    • /storage/artifactory/home/logs/artifactory.log
    • /storage/artifactory/home/logs/access.log
    • /storage/artifactory/home/logs/request.log
    • /storage/artifactory/home/logs/import.export.log
  • vRA Windows
    • C:\Program Files (x86)\VMware\vCAC\Agents\<PLUGIN>\logs\<FILE>
      • <PLUGIN> examples: vSphereAgent, nsx, VC50, VC55Agent, VDIAgent, vCNS
      • <FILE> examples: vSphereAgent, EpiPowerShellAgent, VdiPowerShellAgent
      • Note: <PLUGIN> is based on the name of the agent given during installation
    • C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\<DEM>\Logs\*_All
      • Note: <DEM> is based on the name of the DEM given during installation (defaults to ‘DEM’)
    • C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\<DEO>\Logs\*_All
      • Note: <DEO> is based on the name of the DEO given during installation (defaults to ‘DEO’)
    • C:\Program Files (x86)\VMware\vCAC\Server\Logs\All
    • C:\Program Files (x86)\VMware\vCAC\Server\ConfigTool\Log\vCACConfiguration-<date>
    • C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Data\Logs\<nothing today>
    • C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Web\Logs\Repository
    • C:\Program Files (x86)\VMware\vCAC\Server\Website\Logs\Web_Admin_All
    • C:\Program Files (x86)\VMware\vCAC\Web API\Logs\<nothing today>
  • SSO
    • /var/log/vmware/sso/catalina.out
    • /var/log/vmware/sso/ssoAdminServer.log
    • /var/log/vmware/sso/vmware-identity-sts-perf.log
    • /var/log/vmware/sso/vmware-identity-sts.log
    • /var/log/vmware/sso/vmware-sts-idmd-perf.log
    • /var/log/vmware/sso/vmware-sts-idmd.err
    • /var/log/vmware/sso/vmware-sts-idmd.log
  • VRO
    • /var/log/vco/app-server/catalina.out
  • APPD
    • /home/darwin/tcserver/darwin/logs/catalina.out
  • VRB
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/catalina.out
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/auditFile.log
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/itfm-external-api.log
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/itfm-reflib-update.log
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/itfm-vc-dc.log
    • /usr/local/tcserver/vfabric-tc-server-standard/tcinstance1/logs/itfm.log
  • VCS
    • /var/log/vmware/vpx/vpxd.log
    • /var/log/vmware/vpx/vws.log
    • /var/log/vmware/vpx/vmware-vpxd.log
    • /var/log/vmware/vpx/inventoryservice/ds.log
    • /var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log
    • /var/log/vmware/sso/ssoAdminServer.log
    • /var/log/vmware/sso/vmware-identity-sts.log
    • /var/log/vmware/sso/vmware-sts-idmd-perf.log
    • /var/log/vmware/sso/vmware-sts-idmd.log

Wow, that is a lot of log files! In order to forward these log files to a remote syslog destination like Log Insight, you need to configure a syslog agent on each device. In order to save everyone a lot of time, I have put together the configurations necessary based on the syslog agent installed in the VA for each vCAC component. Enjoy!

Log Insight Server-Side Configuration

Before I break down the configuration for each component of vRA and its dependencies, it is worth mentioning that if you are running Log Insight and install the Log Insight Windows/Linux agent on all vRA components and other VMware dependencies, then you can more easily configure remote logging centrally on the Log Insight server-side under Administration > Agents and the agents will automatically collect the logs applicable to them (i.e. they will ignore any configuration that is not applicable). This makes agent configuration much easier than the manual steps provided in the sections that follow.

Important: In order to push server-side configuration to Log Insight agents, you must use the ingestion API with the agent. If you use the syslog protocol with the agents then the below configuration will not work.

Unfortunately, the Windows part of vRA for remote log collection is a little tricky (this is true whether you use this central approach or the more manual approach in the following sections). Below you will find two configuration sections, the first is static configuration information that can be copied and pasted as-is. The second is dynamic configuration, which depends on what installed and what you named it. For the second configuration section, you will need to replace anything that looks like <THIS>.

Important: For the Windows components, you must use the Log Insight agent and the ingestion API if you want the vRA content pack for Log Insight to work properly. If you do not use the Log Insight agent or if you do not use the ingestion API with the Log Insight agent then some of the queries in the vRA content pack will return no results.

  • Static – copy and paste the below information as-is. While some of it may not apply to your environment the agent will properly monitor what is applicable. Do not forgot to see the Dynamic bullet below this section as it needs to be configured as well!

    Important: The use of the ingestion API with the Log Insight agent is highly recommended for the Linux components and required for the Windows component. Also, the use of asterisk (*) in the include directory is correct and should not be changed. The asterisk is a glob and means one or more characters. Using this over the entire filename is recommended as the filename may change over team (e.g. rebranding from vCloud to vRealize).

  •  Dynamic – do NOT copy and paste the information below without changing anything with <brackets>. Note the below information depends on what you have configured in your environment. For example, if you installed 10 agents, you will need to add 10 agent configuration sections like the example below.

    Important: The use of the ingestion API with the Log Insight agent is highly recommended for the Linux components and required for the Windows component. Also, the use of asterisk (*) in the include directory is correct and should not be changed. The asterisk is a glob and means one or more characters. Using this over the entire filename is recommended as the filename may change over team (e.g. rebranding from vCloud to vRealize).

vRA VA + vRCS

Log Insight Linux Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Rsyslog

vRA Windows

Important: For the Windows components, you must use the Log Insight agent and the ingestion API if you want the vRA content pack for Log Insight to work properly. If you do not use the Log Insight agent or if you do not use the ingestion API with the Log Insight agent then some of the queries in the vRA content pack will return no results. Also, whether you are running IAAS in an all-in-one or distributed model the below configuration can be used. Any log files that do not exist will be ignored.

Log Insight Windows Agent

The recommended way to collect logs from the vRA Windows components is using the Log Insight Windows agent. Unfortunately, the Windows part of vRA for remote log collection is a little tricky. Below you will find two configuration sections, the first is static configuration information that can be copied and pasted as-is. The second is dynamic configuration, which depends on what installed and what you named it. For the second configuration section, you will need to replace anything that looks like <THIS>.

  • Static

  •  Dynamic

SSO

Log Insight Windows Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Log Insight Linux Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Syslog-NG

VRO

vRO is the one component where using an agent is not recommended. Instead, one should edit the log4j configuration to enable remote syslog. This requires two changes within /etc/vco/app-server/log4j.xml (note vRO may also be installed on the vRA VA so be sure to configure it there as well if applicable):

  1. Replace <LOGINSIGHT>
  2. Change:

    To:

Finally, restart VRO: /etc/init.d/vco-server restart

APPD

Log Insight Linux Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Syslog-NG

VRB

Log Insight Linux Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Syslog-NG

VCS

Log Insight Linux Agent

Important: The use of the ingestion API with the Log Insight agent is highly recommended.

Syslog-NG

© 2015, Steve Flanders. All rights reserved.

6 thoughts on “vRA Remote Logging

  1. Dan Y says:

    Steve, you feel this still applies well to 6.1 or 6.2 of vRA? The current content pack points to 6.0 and I’m seeing some discrepancies on the queries on the dashboard.

  2. Ben says:

    Hello,

    For vRO, I think the default seemed to be that the facility isn’t set either, so as well as replacing you also need to set on the line above it.

    Also, it didn’t seem that a restart of vRO was required, it just seemed to kick into life! Thanks for the guide.

  3. Venkat says:

    Steve,
    Excellent article, thanks for your time and efforts for detailed write-up. Is there any possibility to get the samples configs ? Some of the screenshots are chopped out in the screenshots.

    • Hey Venkat — Thanks for the comment. The vRA content pack in LI includes the agent group configurations, I would recommend using that. If you are looking for the non-liagent configuration, you should see button in the code blocks on the blog post that can be used to see the cut off content. I hope this helps.

Leave a Reply