tracing – Structured Observability Framework for Rust

Overview

Tracing is an observability framework designed for Rust applications, providing a system for instrumenting programs to collect and analyze diagnostic information. It is particularly well-suited for asynchronous Rust programs and those within the Tokio ecosystem, where understanding execution flow and potential bottlenecks can be complex. The framework distinguishes itself from traditional logging by introducing the concept of "spans," which represent periods of execution, and "events," which are points in time associated with diagnostic data. This structured approach allows for more granular and contextualized observability data, making it easier to diagnose issues in complex systems, including microservices and distributed architectures.

The core tracing crate provides the foundational APIs for instrumenting code, allowing developers to define spans and record events with associated key-value data. This data can include anything from function arguments and return values to network request IDs and database query details. The flexibility of attaching arbitrary data to spans and events enables developers to capture the specific context relevant to their application's logic.

For processing and exporting this diagnostic data, the tracing-subscriber crate offers a modular set of tools. Subscribers are responsible for collecting the data emitted by the tracing API and determining how it should be filtered, formatted, and ultimately stored or transmitted. This modularity means developers can configure tracing to output to various destinations, such as the console, log files, or external observability platforms. The design allows for dynamic filtering of spans and events based on criteria such as verbosity level, module path, or custom predicates, helping to manage the volume of diagnostic output without re-compiling the application.

Tracing is particularly beneficial for applications requiring detailed insights into their runtime behavior, such as web servers, data processing pipelines, and long-running services. Its integration with OpenTelemetry via the tracing-opentelemetry crate further extends its utility, enabling Rust applications to participate in distributed tracing systems. This compatibility allows for end-to-end visibility across services written in different languages, facilitating the diagnosis of performance issues and errors that span multiple components in a distributed system. The framework's emphasis on structured data ensures that the collected information is machine-readable and amenable to analysis by observability tools, moving beyond simple human-readable log messages.

Key features

• Structured logging: Capture diagnostic information as structured key-value pairs, making data easier to parse, query, and analyze programmatically.
• Spans and events: Instrument code with "spans" to represent contextual periods of execution and "events" for specific points in time, providing a hierarchical view of application flow.
• Contextual data: Attach arbitrary key-value data to spans and events, allowing for detailed context relevant to specific execution paths or operations.
• Dynamic filtering: Configure subscribers to filter diagnostic data based on verbosity levels, module paths, or custom logic at runtime, reducing noise in logs.
• Extensible subscriber architecture: Utilize the tracing-subscriber crate to direct diagnostic output to various destinations, including the console, files, or remote observability services.
• OpenTelemetry integration: Integrate with the OpenTelemetry ecosystem via tracing-opentelemetry for distributed tracing, enabling end-to-end visibility across polyglot microservices.
• Asynchronous Rust support: Designed with first-class support for asynchronous Rust programs, providing clear insights into concurrent operations and task execution.

Pricing

Tracing is an open-source project distributed under the Apache 2.0 and MIT licenses. There are no direct costs associated with using the core tracing crates or the broader tracing ecosystem.

Common integrations

• Tokio: Seamlessly integrates with the Tokio runtime for robust asynchronous application observability.
• OpenTelemetry: Exports traces and spans in OpenTelemetry format via the tracing-opentelemetry crate, allowing compatibility with various APM and observability platforms.
• Log (Rust): Can act as a backend for the standard log crate, redirecting traditional log messages into the tracing system.
• Fmt (console output): The tracing-subscriber crate includes formatters for console output, making it easy to view structured logs during development.
• JSON logging: Subscribers can be configured to output JSON-formatted logs, suitable for ingestion by log aggregation systems like Elasticsearch or Splunk.

Alternatives

• log: The defacto standard logging facade for Rust, providing a simple macro-based interface for emitting log messages.
• slog: A structured, contextual, and composable logging library for Rust, offering more advanced features than the basic log crate.
• OpenTelemetry: A broader set of APIs, SDKs, and tools for generating, collecting, and exporting telemetry data (metrics, logs, and traces) vendor-agnostically.

Getting started

To begin using tracing in a Rust project, add the necessary crates to your Cargo.toml. A basic setup involves the tracing crate for instrumentation and tracing-subscriber to process the emitted data. This example demonstrates how to create a simple span and emit an event.

# Cargo.toml

[dependencies]
tracing = "0.1"
tracing-subscriber = "0.3"

Then, set up a basic subscriber in your main function and add some instrumentation:

// src/main.rs

use tracing::{info, span, Level};
use tracing_subscriber;

fn main() {
    // Initialize the default subscriber. This will print tracing events to stdout.
    tracing_subscriber::fmt::init();

    // Create a span that represents a function's execution.
    let root_span = span!(Level::INFO, "my_application_run", version = "1.0.0");
    let _enter = root_span.enter(); // Enter the span context

    info!("Application started up.");

    // Call a function that also has instrumentation
    do_work(42);

    info!("Application finished.");
}

#[tracing::instrument(level = "info", skip(value), fields(input = value))]
fn do_work(value: u32) -> String {
    // Inside this function, a new span named "do_work" will be active.
    // The 'input' field will be automatically added to the span.
    info!("Performing some task.");

    let result = format!("Processed: {}", value);

    // Emit an event with additional data.
    tracing::event!(Level::DEBUG, output = %result, "Task completed.");

    result
}

When you run this program (cargo run), you will see formatted output in your console showing the spans entering and exiting, along with the events and their associated structured data. The #[tracing::instrument] attribute automatically creates a span for the function it annotates, making instrumentation concise.