Technology
Emissary-ingress
Monitor Emissary-ingress ingress controller performance, request routing, and traffic patterns with OpenTelemetry metrics and distributed tracing.
Overview
Overview
Emissary-ingress is an open-source Kubernetes-native API Gateway and ingress controller built on Envoy Proxy. It provides advanced traffic management, observability, and security features for cloud-native applications.
The Dash0 integration provides complete observability for Emissary-ingress deployments using OpenTelemetry-native instrumentation for metrics and distributed tracing, plus structured access logs.
For more information, visit the Emissary-ingress Documentation.
Use Cases
Gain comprehensive visibility into your Emissary-ingress setup and optimize performance with the following observability features:
- Monitor Request Flows with Tracing: Visualize the full lifecycle of requests across services. Powered by OpenTelemetry, Emissary-ingress's distributed tracing provides insight into latency, headers, and spans—ideal for debugging and performance tuning.
- Track System and Traffic Metrics: Collect real-time performance data such as request counts, durations, error rates, and connection metrics.
- Audit and Analyze Traffic with Access Logs: Record detailed information about incoming requests, including IPs, headers, duration, response status, and more. Export structured logs via OTLP for centralized analysis in Dash0.
- Monitor Envoy Proxy Statistics: Access detailed Envoy proxy metrics for cluster health, upstream connections, and load balancer statistics.
Use this integration to streamline monitoring of your API gateway traffic, debug routing issues, and ensure your platform services are operating efficiently.
Setup
Overview
Emissary ingress includes built-in support for observability via Prometheus metrics, OpenTelemetry tracing, and structured access logs through its underlying Envoy Proxy. This integration uses the recommended pattern of deploying an OpenTelemetry Collector in your cluster to collect telemetry from Emissary ingress and forward it to Dash0.
For more details, refer to the Emissary ingress Observability Documentation.
Configure
Prerequisites
You'll need an OpenTelemetry Collector deployed in your cluster to collect telemetry from Emissary ingress and forward it to Dash0.
For production environments, consider using one of these deployment methods:
- Dash0 operator for Kubernetes
- OpenTelemetry Operator for Kubernetes
-
Helm Chart for The OpenTelemetry Collector
For a complete working example with collector configuration, see the Dash0 Emissary ingress Example.
Collecting Telemetry
Step 1: Configure OpenTelemetry Tracing Service
Create a dedicated TracingService resource to configure OpenTelemetry tracing:
apiVersion: getambassador.io/v3alpha1
kind: TracingService
metadata:
name: otel-tracing
namespace: emissary
spec:
service: "<collector-service>.<collector-namespace>.svc.cluster.local:4317"
driver: opentelemetry
config:
service_name: emissary-ingressStep 2: Configure the Ambassador Module
Configure the Ambassador module with structured logging and reference the tracing service:
apiVersion: getambassador.io/v3alpha1
kind: Module
metadata:
name: ambassador
namespace: emissary
spec:
config:
# Service identification
service_name: emissary-ingress
cluster: emissary-demo
# Enable structured access logging with trace correlation
enable_envoy_logs: true
envoy_log_type: json
envoy_log_format:
"@timestamp": "%START_TIME%"
authority: "%REQ(:AUTHORITY)%"
bytes_received: "%BYTES_RECEIVED%"
bytes_sent: "%BYTES_SENT%"
duration: "%DURATION%"
method: "%REQ(:METHOD)%"
path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
protocol: "%PROTOCOL%"
request_id: "%REQ(X-REQUEST-ID)%"
response_code: "%RESPONSE_CODE%"
response_flags: "%RESPONSE_FLAGS%"
traceparent: "%REQ(traceparent)%"
upstream_cluster: "%UPSTREAM_CLUSTER%"
upstream_host: "%UPSTREAM_HOST%"
upstream_service_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
user_agent: "%REQ(USER-AGENT)%"
x_forwarded_for: "%REQ(X-FORWARDED-FOR)%"
# Enable diagnostics
diagnostics:
enabled: true
# Reference the tracing service
default_tracing_service: otel-tracingImportant: The traceparent field in the log format above enables trace correlation. Your OpenTelemetry Collector must be configured to extract the trace_id and span_id from this field for proper log-trace correlation. Configure a transform processor like this:
processors:
transform/trace-extract:
log_statements:
- context: log
statements:
# Extract trace_id and span_id from traceparent header
# W3C format: 00-{trace_id}-{span_id}-{flags}
- set(trace_id.string, Substring(attributes["traceparent"], 3, 32)) where attributes["traceparent"] != nil and IsMatch(attributes["traceparent"], "^00-[a-f0-9]{32}-[a-f0-9]{16}-[0-9a-f]{2}$")
- set(span_id.string, Substring(attributes["traceparent"], 36, 16)) where attributes["traceparent"] != nil and IsMatch(attributes["traceparent"], "^00-[a-f0-9]{32}-[a-f0-9]{16}-[0-9a-f]{2}$")Step 3: Deploy with Helm
Install or upgrade Emissary ingress with optimized configuration for observability. You can use a values.yaml file:
# values.yaml - Emissary-ingress configuration for observability
replicaCount: 2
# Environment variables for telemetry
env:
AMBASSADOR_CLUSTER_ID: "emissary-demo"
# Resources
resources:
limits:
cpu: 1000m
memory: 600Mi
requests:
cpu: 200m
memory: 300Mi
# Service configuration
service:
type: LoadBalancer
# Enable admin service
adminService:
create: true
type: ClusterIP
port: 8877
# Security context
podSecurityContext:
runAsNonRoot: true
# Create RBAC resources
rbac:
create: true
# Service account
serviceAccount:
create: trueThen deploy with:
helm upgrade --install emissary-ingress \
datawire/emissary-ingress \
--namespace emissary \
--create-namespace \
--values values.yamlVerification
After deployment, verify that telemetry is being collected:
- Check OpenTelemetry Collector logs to ensure it's receiving traces from Emissary ingress
- Verify structured access logs are being generated in JSON format with trace correlation
- Generate test traffic to your ingress endpoints
- Monitor Dash0 for incoming metrics, traces, and structured logs from your Emissary ingress deployment
The integration will provide visibility into request routing, performance metrics, distributed traces, and correlated access logs across your API gateway infrastructure.
Refer to the complete example for detailed configuration options.
