The OpenTelemetry Collector and Exporters
The trace and metric data that your service or its dependencies emit are of limited use unless you can actually collect that data somewhere for analysis and alerting. The OpenTelemetry component responsible for batching and transporting telemetry data to a backend system is known as an exporter.
To understand how OpenTelemetry’s exporter model works, it is useful to understand a little bit about how instrumentation is typically integrated into service code. Generally, instrumentation can be done at three different points: at the service level, at its library dependencies, and at its platform dependencies. Integrating at the service level is fairly straightforward, involving declaring a dependency in your code on the appropriate OpenTelemetry package and deploying it with your code. Library dependencies are similar, except that libraries will generally only declare a dependency on the OpenTelemetry API. Platform dependencies are a more unusual case. When I say ‘platform dependency’, what I mean are the pieces of software you run to provide services to your service, things like Envoy and Istio. These will deploy their own copy of OpenTelemetry, independent of your actions, but will also emit trace context that your service will find useful.
The exporter interface is implemented by the OpenTelemetry SDKs, and uses a simple plug-in model that allows for telemetry data to be translated to the particular format a backend system requires, as well as the transmission of data to said backend. Exporters can be composed and chained together, allowing for common functionality (like tagging data before export, or providing a queue to ensure consistent performance) to be shared across multiple protocols.
While the exporter model is very convenient, there are instances when you don’t have the ability to actually redeploy a service in order to add a new exporter. In some organizations, there is a disconnect between the people writing the instrumented code and the people running the observability platform, which can impair the velocity of rolling out these types of changes. In addition, some teams may prefer to abstract the entire exporter model out from their code, and into a separate service. This is where the OpenTelemetry Collector comes in.
The Collector is a separate process that is designed to be a ‘sink’ for telemetry data emitted by many processes, which can then export that data to backend systems. The Collector has two different deployment strategies – either running as an agent alongside a service, or as a remote application. In general, using both is recommended: the agent would be deployed with your service and run as a separate process or in a sidecar; meanwhile, the collector would be deployed separately, as its own application running in a container or virtual machine. Each agent would forward telemetry data to the Collector, which could then export it to a backend analytics system like Lightstep.
Regardless of how you choose to instrument or deploy OpenTelemetry, exporters provide powerful options for reporting telemetry data. You can directly export from your service, you can proxy through the Collector, or you can aggregate into standalone collectors – or even a mix of these.