The OpenCensus Service
Published by Steve Flanders on
A common question I get asked is, “we have an urgent need to implement tracing, and we can’t wait for the OpenTelemetry release in September. What’s the recommendation for people in my shoes?” Throughout last week’s conversations at KubeCon EU, and those that have come up this week, starting with OpenCensus has been preferred due to the inclusion of an implementation.
Continue reading to learn more about how the OpenCensus Service can be leveraged today no matter what client instrumentation you use.
The OpenCensus project consists of an API specification, client libraries, and a collection mechanism known as the OpenCensus Service. The OpenCensus Service was specifically created to address the issue of how to get observability information to the backend or backends of your choice. Some of the issues that the OpenCensus Service tackles include:
- Transforming data into and out of different formats
- Buffering and retries
- Batching and caching
- Security and egress management
- Push and pull based data collection
- Dynamic configuration
- Data formatting and sampling
- Multi-destination support
The OpenCensus Service is made up of two components: an Agent and Collector. Depending on requirements, you may choose to use either, both, or neither. The most important concepts to understand in the OpenCensus Service are the notion of:
- Receivers - how you get data into the system
- Exporters - how you get data out of the system
The OpenCensus Service supports a broad set of open-source as well as commercial receivers and exporters. In addition, the pluggable architecture makes it so anyone can easily contribute a new receiver or exporter.
As the name implies, the OpenCensus Agent is meant to sit as close to the application as possible. It can be added as a binary, sidecar, or daemonset. It is designed to receive all metrics and traces from the application and/or host it is running on. It can then send the data it receives to one or more destinations.
The OpenCensus Agent was created to:
- Reduce support requirements to a single exporter for client instrumentation
- Eliminate the need to reconfigure client instrumentation (i.e. send locally)
- Automatically collect host/application metrics
- Offload logic such as batching, buffering and retries
The Collector is designed to sit between the application and destination, ideally as close to the application as possible (e.g. same datacenter or region). The Collector runs as a standalone service and can be added via a simple binary or container. Metrics and traces can be received via the Opencensus Agent or directly via client instrumentation. The Collector can then perform operations against the data before reliably sending it to one or more destinations.
The OpenCensus Collector was created to:
- Buffer and retry data at production scale
- Limit egress points
- Intelligently sample (i.e. tail-based)
- Allow span annotations (e.g. datacenter or region)
- Allow tag redaction or modification
Although client libraries can be used without the Agent or the Collector, the best practice is to configure them as follows:
- Configure client libraries (whether OpenCensus or not) to point locally to the OpenCensus Agent
- Configure the OpenCensus Agent to point to the closest OpenCensus Collector
- Configure the OpenCensus Collector to point to the backend(s) of your choice
Why is this the best practice? Lots of reasons:
- Client libraries only need to be configured once – by pointing to the Agent this is possible without needing to worry about errors or back pressure
- The Agent is lightweight, with minimal overhead that only takes effect once configured to export to a destination
- Without a Collector, any additional operational functionality during ingest including secure channel transmission would have to be handled via custom instrumentation
Users that choose to not use the OpenCensus client libraries will still find value in the OpenCensus service due to:
- Support for the most popular receivers and exporters today
- Extensibility, specifically in ability to easily add receivers/exporters
- Support for intelligent sampling in an open source project
- Ability to ingest both metrics and traces
- Portability across backends without the need to change client instrumentation
The OpenCensus Service provides a lot of capabilities related to the collection of metric and trace data. It offers enterprise-grade features including scale-out, queuing, retry, and redaction support while being completely open-source and vendor-agnostic. Whether you use the OpenCensus client libraries or not, the OpenCensus Service provides a reliable, performant, and heavily tested foundation for the collection of observability data.
The OpenCensus Service will be moving in its current form to the OpenTelemetry Service over the next couple of months. Other open-source projects, such as Jaeger, are planning to move to the OpenTelmetry Service in the future – you can read more about this in Yuri’s post. If you are interested in learning more about the OpenCensus Service then check out the documentation or join the conversation on Gitter.