agentgateway

Overview

agentgateway is an AI-first Kubernetes Gateway API implementation built in Rust, designed for agentic AI workloads. It provides native integrations with LLM providers like Anthropic and OpenAI, eliminating the need for custom proxy code while delivering comprehensive observability for AI traffic patterns.

The Dash0 integration provides full observability for agentgateway deployments using native OpenTelemetry instrumentation with GenAI semantic conventions, distributed tracing, Prometheus metrics for token usage tracking, and structured access logs with automatic trace correlation.

For more information, visit the agentgateway Documentation.

Use Cases

Gain comprehensive visibility into your AI Gateway setup and optimize LLM performance with the following observability features:

• Track LLM Token Usage and Costs: Monitor token consumption patterns with native GenAI metrics including input/output token histograms per model, provider, and route. Track usage trends to optimize costs and identify high-consumption patterns.
• Monitor AI Request Flows with Distributed Tracing: Visualize complete request lifecycles from gateway through LLM providers with W3C trace context propagation. agentgateway's native OTLP/gRPC tracing provides insights into latency, model selection, and upstream LLM API interactions.
• Analyze GenAI Traffic Patterns: Record detailed information about LLM requests including model names, operation types (chat, completion), providers (Anthropic, OpenAI), and response characteristics with automatic trace correlation for deep traffic analysis.
• Monitor Prompt Guard Security: Track rejected requests from prompt guards filtering sensitive data (emails, credit cards, PII) to ensure compliance and prevent data leakage to external LLM providers.
• Ensure Gateway Health: Monitor agentgateway controller health, proxy resource usage, and configuration synchronization to ensure your AI gateway infrastructure operates reliably.

Use this integration to monitor AI traffic patterns, track LLM costs, debug routing issues, analyze GenAI performance, and ensure your AI Gateway operates efficiently at scale.

Overview

agentgateway includes native support for observability via OpenTelemetry distributed tracing (OTLP/gRPC), Prometheus metrics with GenAI semantic conventions, and structured access logs using OTLP LogRecord format with automatic TraceID and SpanID correlation.

This integration uses the recommended pattern of deploying an OpenTelemetry Collector in your cluster to collect telemetry from agentgateway and forward it to Dash0.

For more details, refer to the agentgateway Documentation.

Configure

Prerequisites

You'll need an OpenTelemetry Collector deployed in your cluster to collect telemetry from agentgateway and forward it to Dash0.

For production environments, consider using one of these deployment methods:

• Dash0 operator for Kubernetes
• OpenTelemetry Operator for Kubernetes

• Helm Chart for The OpenTelemetry Collector

For a complete working example with collector configuration, see the Dash0 agentgateway Example.

Collecting Telemetry

Step 1: Configure OpenTelemetry Collector

The OpenTelemetry Collector needs specific configuration to collect all three signals from agentgateway:

OTLP Receiver for Traces and Logs

Configure the Collector to receive distributed traces and access logs from agentgateway using the otlp receiver:

agentgateway sends both traces and logs to the same OTLP receiver. Traces use W3C trace context propagation with GenAI semantic conventions, and access logs are sent as OTLP LogRecords with automatic TraceID and SpanID correlation—no manual parsing or transformation needed.

Prometheus Receiver for GenAI Metrics

Configure the Collector to scrape GenAI metrics from both the agentgateway controller and AI gateway proxies using the prometheus receiver:

Step 2: Configure AgentgatewayPolicy for Telemetry

Create an AgentgatewayPolicy to configure distributed tracing for your Gateway:

Step 3: Create ReferenceGrant for Cross-Namespace Access

Since the AgentgatewayPolicy in agentgateway-system needs to reference the collector service in opentelemetry namespace, create a ReferenceGrant:

Verification

To verify that telemetry is flowing:

1. Check traces in Dash0: Filter by service.name = "ai-gateway" - should include GenAI semantic conventions
2. Check GenAI metrics: Search for agentgateway_gen_ai_* metrics (token usage, operation duration)
3. Check logs: View structured logs with trace IDs and GenAI attributes

Additional Resources

• agentgateway Documentation
• agentgateway Observability Guide
• Gateway API Documentation
• Dash0 agentgateway Example
• GenAI Semantic Conventions