Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

otel-kit

otel-kit is the repository for three Rust crates that cover different observability boundaries:

  • otel-init initializes OpenTelemetry providers and tracing subscribers;
  • tracing-otel provides shared HTTP tracing utilities and an opinionated logging/bootstrap facade;
  • axum-otel adapts the shared HTTP span behavior to Axum and tower-http::TraceLayer.

Choose the smallest crate that owns the behavior your application needs. An Axum application can use axum-otel with its own subscriber, while an application that wants the repository’s complete logging setup can combine it with tracing-otel’s logger or env feature. Provider-only users can depend directly on otel-init.

The workspace packages are currently version 0.33.1. Cargo.toml declares Rust 1.92.0 as the minimum supported Rust version. The repository toolchain may pin a newer compiler for local development.

Where to start

  • Getting started — install the right crates and build a minimal application.
  • Crates — choose the smallest crate for each observability responsibility.
  • Cargo features — keep optional dependencies and capabilities explicit.
  • Comparison with axum-tracing-opentelemetry — choose between focused request middleware and this workspace’s broader observability setup.
  • OTLP export and lifecycle — configure exporters and shut providers down safely.
  • HTTP span attributes — understand emitted fields before writing dashboards or alerts.

The complete type-level references are on docs.rs for tracing-otel, docs.rs for otel-init, and docs.rs for axum-otel.

Getting started

The workspace packages are version 0.33.1. tracing-otel has no default features, so applications must enable the capability they use.

Application logging

Enable logger for code-based configuration:

[dependencies]
tracing-otel = { version = "0.33.1", features = ["logger"] }
anyhow = "1"
tracing = "0.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
use tracing_otel::{LogFormat, Logger};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let _guard = Logger::new("inventory-api")
        .with_format(LogFormat::Json)
        .with_ansi(false)
        .init()?;

    tracing::info!(component = "startup", "service ready");
    Ok(())
}

Keep the returned LoggerGuard alive for the whole application. Dropping it shuts down the OpenTelemetry providers and then releases any non-blocking file writer guard.

Environment-based configuration

Enable env when configuration should come from environment variables. It includes logger:

[dependencies]
tracing-otel = { version = "0.33.1", features = ["env"] }
#![allow(unused)]
fn main() {
use tracing_otel::Logger;

fn run() -> anyhow::Result<()> {
let _guard = Logger::from_env(Some("LOG"))?.init()?;
Ok(())
}
}

With the LOG prefix, fields map to names such as LOG_SERVICE_NAME, LOG_FORMAT, and LOG_SAMPLE_RATIO. See Logger configuration for the complete mapping used by the repository.

Axum middleware

axum-otel supplies the request-span callbacks; it does not replace subscriber initialization:

[dependencies]
axum-otel = "0.33.1"
tracing-otel = { version = "0.33.1", features = ["logger"] }
anyhow = "1"
axum = "0.8"
tokio = { version = "1", features = ["macros", "net", "rt-multi-thread"] }
tower-http = { version = "0.6", features = ["trace"] }

Continue with Axum request tracing for a complete layer setup.

Crates

The repository is a workspace, not one all-inclusive package.

CrateResponsibilityUse it when
otel-initResources, OTLP tracer/meter/logger providers, subscriber setup, and provider shutdownYou own the subscriber or only need provider-level initialization
tracing-otelHTTP fields/context/span helpers, dynamic tracing macros, and optional application logging bootstrapYou need shared tracing behavior or the opinionated Logger facade
axum-otelAxum-specific TraceLayer callbacksYou need inbound Axum request spans

The dependency direction is:

axum-otel -> tracing-otel -> otel-init

These arrows are feature-dependent. For example, tracing-otel does not pull in otel-init when a consumer enables only fields, and it has no default features.

Public entry points

otel-init

  • get_resource
  • init_tracer_provider, init_meter_provider, init_logger_provider
  • OtelGuard
  • init_env_filter and init_tracing_subscriber with the subscriber feature

tracing-otel

  • http::fields, http::propagation, http::context, and http::span
  • dyn_span! and dyn_event!
  • Logger, LoggerGuard, LogFormat, and file-appender configuration
  • Logger::from_env and environment initialization helpers with env

axum-otel

  • AxumOtelSpanCreator
  • AxumOtelOnResponse
  • AxumOtelOnFailure

Use docs.rs as the exhaustive API reference; this book documents selection, composition, and runtime behavior.

Cargo features

tracing-otel has no default features. Every application must select the modules its source code uses.

FeatureAddsImplies
fieldsHTTP request field extractionhttp dependency only
macrosRuntime-level dyn_span! and dyn_event!tracing dependency
httpHTTP context extraction and injectionfields
contextCurrent trace/span IDs and remote-parent attachmenthttp
spanShared HTTP server span creationcontext, macros
otelRe-exports from otel-initotel-init dependency
loggerOpinionated subscriber, console/file logging, traces, metrics, and optional OTLP logsotel, otel-init/subscriber
envEnvironment deserialization for Loggerlogger

Examples:

# Shared request-span utilities without the Logger facade.
tracing-otel = { version = "0.33.1", features = ["span"] }

# Complete code-configured application bootstrap.
tracing-otel = { version = "0.33.1", features = ["logger"] }

# Logger plus environment configuration.
tracing-otel = { version = "0.33.1", features = ["env"] }

axum-otel enables the tracing-otel capabilities it uses internally. A consumer does not need to duplicate those internal feature choices unless it also imports tracing-otel APIs directly.

otel-init has one optional feature:

FeatureAdds
subscribertracing-subscriber integration and the OpenTelemetry log bridge

Useful contributor checks:

cargo check -p tracing-otel --no-default-features
cargo check -p tracing-otel --no-default-features --features span
cargo check -p tracing-otel --no-default-features --features logger
cargo check -p tracing-otel --no-default-features --features env

Logger configuration

tracing_otel::Logger is the opinionated application bootstrap. It creates the configured console and file layers, initializes trace and metric providers, optionally initializes an OpenTelemetry log provider, and installs the global tracing subscriber.

Builder configuration

#![allow(unused)]
fn main() {
use tracing::Level;
use tracing_otel::{LogFormat, Logger};

fn run() -> anyhow::Result<()> {
let _guard = Logger::new("payments-api")
    .with_format(LogFormat::Json)
    .with_level(Level::INFO)
    .with_ansi(false)
    .with_sample_ratio(0.25)
    .with_metrics_interval_secs(30)
    .with_console_enabled(true)
    .init()?;
Ok(())
}
}

Logger::default() uses compact output, ANSI enabled, INFO, full trace sampling, a 30-second metric interval, console output enabled, no file appender, and OpenTelemetry log export disabled. Environment deserialization uses false for an omitted LOG_ANSI, so set it explicitly when switching between builder and environment configuration. RUST_LOG, when valid, takes precedence over the logger level because subscriber setup reads the standard environment filter first.

Environment mapping

Logger::from_env(Some("LOG")) uses LOG as the prefix. Passing None also defaults to LOG.

Environment variableLogger fieldExample
LOG_SERVICE_NAMEservice_nameorders-api
LOG_FORMATformatcompact, pretty, or json
LOG_SPAN_EVENTSspan_events`FMT::NEW
LOG_ANSIansifalse
LOG_LEVELlevelinfo
LOG_SAMPLE_RATIOsample_ratio0.1
LOG_METRICS_INTERVAL_SECSmetrics_interval_secs30
LOG_ATTRIBUTESresource attributesdeployment.environment=prod
LOG_CONSOLE_ENABLEDconsole layertrue
LOG_OTEL_LOGS_ENABLEDOTLP log provider/layertrue

File settings use the LOG_FILE prefix:

Environment variableMeaning
LOG_FILE_ENABLEEnable the file layer
LOG_FILE_NON_BLOCKINGUse the non-blocking writer
LOG_FILE_LEVELOptional file-specific level
LOG_FILE_ANSIANSI in file output
LOG_FILE_FORMATOptional file-specific format
LOG_FILE_ROTATIONminutely, hourly, daily, or never
LOG_FILE_DIROutput directory
LOG_FILE_FILENAME_PREFIXFilename prefix
LOG_FILE_FILENAME_SUFFIXFilename suffix
LOG_FILE_MAX_LOG_FILESMaximum retained files

Environment-based configuration requires the env feature. Calling Logger::new(...) does not read LOG_*; use Logger::from_env(...) or init_logging_from_env(...) when those variables should apply.

OTLP export and lifecycle

otel-init creates trace, metric, and log providers. Export is opt-in through an endpoint environment variable.

# Enable all configured signals.
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc

Signal-specific endpoints can enable only one signal or override the shared endpoint for that signal:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf

The corresponding metric and log names are OTEL_EXPORTER_OTLP_METRICS_ENDPOINT / _PROTOCOL and OTEL_EXPORTER_OTLP_LOGS_ENDPOINT / _PROTOCOL. Supported protocol values in this workspace are grpc, http/protobuf (also http/proto), and http/json. A signal-specific protocol takes precedence over OTEL_EXPORTER_OTLP_PROTOCOL; the default is grpc.

When neither the signal-specific endpoint nor the shared endpoint contains a non-blank value, that provider remains local-only. It can still create trace IDs, but it does not construct an exporter or attempt a connection to localhost:4317.

Logger always initializes trace and metric providers. It initializes the log provider only when otel_logs_enabled is true, for example with LOG_OTEL_LOGS_ENABLED=true through the env feature.

Shutdown order

Keep LoggerGuard or OtelGuard alive until application shutdown. Explicit shutdown is available when the application needs to surface provider shutdown errors:

#![allow(unused)]
fn main() {
use tracing_otel::Logger;
fn run() -> anyhow::Result<()> {
let guard = Logger::new("worker").init()?;
tracing::info!("draining work");
guard.shutdown()?;
Ok(())
}
}

LoggerGuard shuts down telemetry providers before releasing its optional non-blocking file writer guard. Dropping a guard also performs cleanup, but an explicit shutdown() is the path that returns provider shutdown errors.

Axum request tracing

axum-otel supplies three callbacks for tower-http::TraceLayer:

  • AxumOtelSpanCreator creates and enriches the request span;
  • AxumOtelOnResponse records the response status and emits the completion event;
  • AxumOtelOnFailure marks classified server failures as OpenTelemetry errors.
use axum::{Router, routing::get};
use axum_otel::{
    AxumOtelOnFailure, AxumOtelOnResponse, AxumOtelSpanCreator, Level,
};
use tower_http::trace::TraceLayer;
use tracing_otel::Logger;

async fn health() -> &'static str {
    "OK"
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let _guard = Logger::new("web-api").init()?;

    let app = Router::new().route("/health", get(health)).layer(
        TraceLayer::new_for_http()
            .make_span_with(AxumOtelSpanCreator::new().level(Level::INFO))
            .on_response(AxumOtelOnResponse::new().level(Level::INFO))
            .on_failure(AxumOtelOnFailure::new().level(Level::ERROR)),
    );

    let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await?;
    axum::serve(listener, app).await?;
    Ok(())
}

The span creator delegates shared HTTP attributes and remote-parent extraction to tracing-otel. It owns only Axum-specific enrichment: matched route, transport peer from ConnectInfo<SocketAddr>, span name, and server span kind.

To record client.address, serve the router with Axum connect information:

axum::serve(listener, app.into_make_service_with_connect_info::<SocketAddr>())
    .await?;

The value comes from the socket peer, not Forwarded or X-Forwarded-For. Deployments behind a proxy should interpret it accordingly.

The defaults are TRACE for span creation, DEBUG for response events, and ERROR for failure events. Ensure the subscriber filter enables the selected levels; otherwise expected events or spans can be filtered out.

Comparison with axum-tracing-opentelemetry

Both axum-otel and axum-tracing-opentelemetry help Axum applications create OpenTelemetry-aware HTTP request spans. The difference is mainly the intended scope.

axum-tracing-opentelemetry

axum-tracing-opentelemetry is a focused choice when you:

  • want request tracing and context propagation with a small middleware surface;
  • already own the tracing subscriber, exporters, metrics, and logging setup;
  • prefer to assemble the observability stack from independent components.

For applications with an established observability bootstrap, that narrower scope can be exactly what is needed.

axum-otel and this workspace

axum-otel itself stays focused on three tower-http::TraceLayer callbacks:

  • AxumOtelSpanCreator creates and enriches request spans;
  • AxumOtelOnResponse records response status and emits completion events;
  • AxumOtelOnFailure marks classified server failures as OpenTelemetry errors.

The broader, opinionated setup comes from composing it with the other workspace crates. tracing-otel adds shared HTTP span helpers, structured console/file logging, and application bootstrap. otel-init adds trace, metric, and optional log providers plus RAII lifecycle management.

Choosing between them

Choose axum-tracing-opentelemetry when you primarily need focused middleware and already own the rest of the stack.

Choose this workspace when you want the same HTTP tracing boundary plus reusable HTTP span helpers or a cohesive tracing, metrics, logging, and shutdown setup.

The distinction is about composition and ownership, not a claim that axum-otel alone provides every capability in the workspace.

HTTP span attributes

tracing-otel::http::span::make_request_span records shared HTTP server attributes. axum-otel adds framework-specific values before the remote parent context is applied.

AttributeSource
http.request.methodRequest method
server.addressHost header
network.protocol.nameHTTP protocol family
network.protocol.versionHTTP version
url.pathURI path
url.queryURI query
url.schemeURI scheme or forwarding field used by the shared extractor
user_agent.originalUser-Agent header
request_idX-Request-Id, then Request-Id
trace_idActive OpenTelemetry trace context
http.routeAxum matched route
client.addressAxum ConnectInfo<SocketAddr> peer IP
otel.nameMethod plus matched route when available
otel.kindOpenTelemetry server span kind
http.response.status_codeInteger response status recorded on response
otel.status_code / otel.status_descriptionResponse/failure callbacks

The names follow the OpenTelemetry HTTP span semantic conventions where applicable. The library does not promise to emit every attribute in that specification; values are recorded only when the request or framework adapter provides them.

Attribute migration

Recent releases replaced older field names:

Previous attributeCurrent attribute
http.hostserver.address
http.user_agentuser_agent.original
http.client_ipclient.address

Update dashboards, alerts, queries, and sampling rules together with the library upgrade. http.response.status_code is recorded as an integer.

Adapter authors can call make_request_span(level, request, callback). The callback runs exactly once before remote-parent application. Record adapter-owned otel.name and otel.kind inside that callback because the OpenTelemetry span is materialized when the parent is applied.

Examples

The workspace contains runnable examples rather than embedding a second copy of their source in this book.

Basic Axum service

examples/otel initializes Logger from LOG_*, adds request ID middleware, and composes the three axum-otel callbacks.

cargo run -p axum-otel-demo

Without an OTLP endpoint, providers stay local-only. To export all supported signals to a local collector:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
cargo run -p axum-otel-demo

The service listens on 0.0.0.0:8080; call /hello for an instrumented route or /health for the simple health route.

Microservices demonstration

examples/microservices contains users and articles services, outbound Reqwest tracing, request ID propagation, and a Grafana/Loki/Tempo-oriented Docker setup. Follow its README and architecture guide when validating full cross-service propagation.

These examples are integration demonstrations, not additional public APIs. Use the crate-specific docs.rs pages for type and method details.

Migrating from 0.33.0

Version 0.33.1 introduced new crates.io package names. Cargo package names do not migrate automatically.

# Before
tracing-otel-extra = "0.33.0"
tracing-opentelemetry-extra = "0.33.0"

# After
tracing-otel = "0.33.1"
otel-init = "0.33.1"

Update Rust import paths at the same time:

// Before
use tracing_otel_extra::Logger;
use tracing_opentelemetry_extra::OtelGuard;

// After
use tracing_otel::Logger;
use otel_init::OtelGuard;

axum-otel keeps its package and import name, but version 0.33.1 depends on tracing-otel internally.

Also verify explicit tracing-otel features. The crate has no default features: use logger for code-based Logger, env for Logger::from_env, or the narrower HTTP features listed in Cargo features.

If the application is upgrading from an older HTTP-span release, migrate dashboard and alert field names using the table in HTTP span attributes.