Skip to content

The OpenCensus Service

In my last post, I talked about OpenCensus. In this post, I would like to focus on the OpenCensus Service. Read on to learn more!

OpenCensus Agent

As the name implies, the Agent is meant to sit as close to the application as possible. It can be added as a binary, sidecar, or daemonset. It should receive all metrics and traces from the application and/or on the host it is running on. It can then send the data it received to one or more destinations.

It was created so that client libraries could send to a destination and not receive errors or back pressure. The idea is that you always deploy the OpenCensus Agent and then when you add instrumentation to your application you always point it to the Agent. The Agent will receive it, but will not forward it anywhere unless it is configured to do so.

It should be noted that the Agent does basically no buffering or retry today. It expects that the destination(s) it sends to is/are highly available. While the agent is capable of sending directly to one or more backends, the best practice is to send to the OpenCensus Collector.

Configuration

By default, the Agent listens for incoming traffic on 55678 (OpenCensus format). If you need the agent to receive in other formats you can specify them via the agent configuration. For example:

By default, the Agent does not send to any destination. Destinations can be specified by configuring one or more exporters. The typical exporter configuration would be to the OpenCensus Collector. For example:

OpenCensus Collector

As the name implies, the Collector is meant to sit between the application and the destination(s), but as close to the application as possible (e.g. same datacenter or region). It can be added as a binary or container, but should run as a standalone service. It should receive all metrics and traces from the application, typically via the OpenCensus Agent. It can then perform operations against the data before sending it to one or more destinations.

Some of the reasons why the Collector were created include:

  • Ability to buffer and retry data at production scale
  • Limit egress points
  • Intelligent (tail-based) sampling
  • Span annotations (e.g. datacenter or region)
  • Tag redaction

Note with intelligent sampling, the Collector should be as close to application as possible, but will not be in the same datacenter/region unless all application calls ONLY happen in the same datacenter/region. This is because the Collector must receive all spans for a given trace in order to make an intelligent sampling decision (more on intelligent sampling in a future post).

Configuration

It should be noted that the Collector is based on the Agent code base. The receivers and exporters are identical to the Agent and as such the configuration is mostly the same as well. Like the Agent, the Collector listens on port 55678 by default and does not export by default. The main differences in configuration with the agent are around additionally supported features.

The exporting configuration can be different as the Collector supports bounded-buffer retry as well as batching through its queued-exporter. For example:

In addition, the Collector offers configuration outside of receivers and exporters. For example, you can add Collector-level tags to spans:

As well as configuration for intelligent sampling:

Finally, the Collector also offers command-line flags to enable functionality (i.e. without manually editing or supplying the configuration). The parameters available today are:

Best Practices

You can use the Agent without the Collector and the Collector without the Agent. You can also use the client libraries without the Agent or the Collector. With that said, the best practice is to configure client libraries (whether OpenCensus or not) to point to the OpenCensus Agent, the OpenCensus Agent to point to the OpenCensus Collector, and the OpenCensus Collector to point to the backends of your choice.

Why is this the best practice? Lots of reasons:

  • You want to configure the client libraries once and leave them – by pointing to the agent this is possible without needing to worry about errors or back pressure
  • You want an agent that is light weight and only does work when configured to export to a destination
  • You want a collector that is capable of doing advanced functionality on top of the data and also is capable of ensuring your data reaches its destination in a secure manner

In addition, it is recommended you use the OpenCensus Service whether you use the OpenCensus client libraries or not. Why? Lots of reasons:

  • Supports the most popular receivers and destinations today
  • Is extensible and additional receivers/exporters can easily be added
  • Is the only open-source software to support intelligent sampling today
  • Works for metrics and traces
  • Is vendor-agnostic

Summary

The OpenCensus Service provides a lot of capabilities related to the collection of metric and trace data. It offers enterprise-grade features including scale-out, queuing, retry, and redaction support while being completely open-source and vendor agnostic. Whether you use the OpenCensus client libraries or not, the OpenCensus Service provides a reliable, performant, and heavily tested foundation for observability data collection.

© 2019, Steve Flanders. All rights reserved.

Published inCloud Native

Be First to Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *