OpenLLMetry

OpenLLMetry

OpenLLMetry is a lightweight, open-source OpenTelemetry-based observability SDK for LLM applications. Developed by Traceloop, it provides automatic instrumentation for various LLM providers (OpenAI, Anthropic, Cohere, and more), vector databases, and frameworks with minimal configuration.

OpenLLMetry automatically captures key metrics like latency, token usage, costs, and full prompt/response pairs without requiring manual span creation or extensive code changes. It's designed to work seamlessly with the OpenTelemetry ecosystem, making it easy to integrate with existing observability platforms like Dash0.

Use Cases

• Monitor LLM API calls across multiple providers (Anthropic, OpenAI, Cohere) with automatic instrumentation
• Track token usage and costs for every LLM request to understand resource consumption and optimize spending
• Analyze latency patterns to identify performance bottlenecks in AI-powered applications
• Capture full conversation context including prompts, responses, and model parameters for debugging and quality assurance
• Correlate LLM calls with application traces to understand how AI interactions affect overall system performance
• Monitor vector database operations (Pinecone, Chroma, Weaviate) alongside LLM calls for RAG applications
• Track streaming responses with automatic instrumentation for streaming API calls
• Measure model performance across different providers and models to optimize selection

Overview

OpenLLMetry (Traceloop SDK) provides automatic OpenTelemetry instrumentation for LLM applications. It automatically detects and instruments LLM provider SDKs (like Anthropic, OpenAI), vector databases, and AI frameworks without requiring manual span creation.

This guide shows how to set up OpenLLMetry to send traces to Dash0 via the OpenTelemetry Protocol (OTLP).

Prerequisites

• Python 3.9 or higher
• An LLM provider SDK installed (e.g., anthropic, openai, cohere)
• An OpenTelemetry Collector endpoint (Dash0 ingress or local collector)
• Dash0 authentication token

Installation

Install OpenLLMetry (Traceloop SDK) alongside your LLM provider SDK:

Or for other providers:

Configuration

Option 1: Via OpenTelemetry Collector (Recommended)

For production deployments, route traces through an OpenTelemetry Collector:

The collector will forward traces to Dash0. For collector setup, see:

• Dash0 Kubernetes Operator
• OpenTelemetry Collector

Option 2: Direct to Dash0

For simpler setups, send traces directly to Dash0:

Using OpenLLMetry in Your Application

Once initialized, OpenLLMetry automatically instruments all supported LLM calls:

Configuration Options

Privacy and Security

Control what data is captured in traces:

Verification

After running your application with OpenLLMetry:

1. Navigate to Traces in your Dash0 dashboard
2. Filter by service.name matching your app_name
3. Look for traces with spans containing gen_ai.* attributes
4. Verify that LLM calls show:
   • Complete timing information
   • Token usage metrics
   • Model and provider details
   • Prompt and response content (if enabled)

Supported Libraries

For the latest list of supported integrations, see the OpenLLMetry documentation.

References

• OpenLLMetry Documentation
• Traceloop SDK GitHub
• OpenTelemetry Semantic Conventions for GenAI
• Dash0 Examples - OpenLLMetry