AutoMerge is generally available

Last updated:

Code RED Newsletter #42

Code RED Newsletter #42 header

KubeCon + CloudNativeCon North America in Salt Lake City is a wrap! It was another packed week: I co-chaired Agentics Day: MCP + Agents with Chris Mazur, moderated a panel on agents on the platform, and gave a talk with William Rizzo on platform guardrails for AI agents. I've written up Agentics Day and the rest of the week on my blog.

The Dash0 team in matching red jumpsuits in front of the Dash0 booth on the KubeCon + CloudNativeCon North America 2026 show floor

I headed home on Thursday to celebrate my 40th birthday on Friday. That same Friday, a telecommunications outage triggered a nationwide FAA ground stop for more than four hours, and many KubeCon attendees, including some of my Dash0 colleagues, were stuck in Salt Lake City longer than planned. I hope everyone made it home safely.

In focus: Agents in Production

We've spent a lot of time talking about what AI agents can do. At KubeCon, more of the conversations I had were about running them in production: who owns an agent, what it's allowed to do, where those boundaries are enforced, and how we find out what it did when something goes wrong.

That's where platform engineering and observability come in. We already know how to give workloads an identity, limit what they can reach and trace what they did, and agents shouldn't be the exception.

Continuous Profiling Is Now Generally Available in Dash0

Traces can tell you that a request took 900 milliseconds instead of 90, but not which function burned the CPU time. Continuous profiling is now generally available in Dash0, built on the Parca technology that came with Polar Signals (#36) and ingested as OpenTelemetry profiles over OTLP. The Dash0 Operator can deploy the eBPF profiler for you (off by default, setup in the changelog). It's designed to use under 10 millicores per node and covers Go, Rust, Java, Python and six more runtimes (Ayo's post has the full list). Agent0 and coding agents can query the profiles over MCP, so when you ask an agent to make a service faster, it can check which functions use the CPU before it changes anything.

Read the post

Continuous Profiling Is Now Generally Available in Dash0, with a flame graph and a BaseProcess.run frame selected

Building an Agent Harness for OTel Instrumentation with GenAI Semantic Conventions and Weaver

Huxing Zhang and Ziming Liu from Alibaba Cloud have a team of agents instrument GenAI frameworks, with people reviewing the plan and the pull request. Their first version used another LLM to review the result, and it "sometimes approved output that violated the conventions". So they replaced that step with Weaver live-check, which validates the actual OTLP output against the semantic conventions registry. It caught an agent setting span status OK on a tool span, which the conventions reserve for application code. When there's already a contract to check against, you don't need another LLM to decide whether the output is right. William and I used Weaver the same way in our KubeCon demo.

Read the post

Cover reading "Agents write the instrumentation. Weaver checks it." above a terminal where weaver registry live-check flags a tool span with status OK

Cyber Chaos Monkey: Staying ahead of security threats with AI agents

Shopify lets an AI agent attack its own production, with live credentials. Bug bounty reports are up nearly 300% since 2024, and Dan Scofield explains how they keep it safe: the attacker runs unprivileged, never sees the real secrets, and can only leave through an egress allowlist. A second "judge" agent on a different model watches the whole session in real time and can kill it, and if the judge stops responding, the run ends. Tens of thousands of engagements so far, "without causing an outage or incident." Attacker and judge run as sidecars in the same pod, so an eviction can't leave the attacker running unsupervised.

Read the post

Neon illustration of a winking monkey picking a padlock in a grid corridor

OpenShell on Kubernetes: A guide for platform engineers

Jim Bugwadia from Nirmata starts with what Kubernetes can't do: "A NetworkPolicy can say 'this Pod may reach api.github.com on 443.' It cannot say 'curl may read issues but not create them.'" OpenShell, started by NVIDIA and now a CNCF Sandbox project, can. Each agent runs in a sandbox, a supervisor outside it enforces filesystem, process, network and credential rules, and the agent only ever sees placeholder API keys. Jim's guide adds seven Kyverno policies on top. It's early, and the post says so: there's no CRD yet, and one supervisor pod per agent may not scale.

Read the guide

CNCF blog card for OpenShell on Kubernetes: A guide for platform engineers, with the Kubernetes logo

Securing AI Agents: Lessons From Real-World Deployment

Uber's starting point: identity and authorization tell you whether an agent may access a resource, but not "whether an agent's proposed action is appropriate in context." So every agent is registered before it runs, with an owner, permitted intents, constraints and a risk tier. At runtime, small local models check each tool call, and one execution trace ties actions, tool calls and guardrail decisions together, so a new policy can be replayed against production interactions before it's enforced. In the authors' words, the most effective mitigation "was also the least glamorous: restricting agents to explicit intents and constraints." Most teams are still missing that execution trace.

Read the post

Uber's layered approach to securing AI agents: provisioning, runtime and continuous assurance, joined by an improvement loop

Peek into your Kubernetes clusters with Agent0

Checkout is failing, and the inventory service reports healthy on its own spans. The cause is in the cluster: a redeploy changed the pod labels, and the Service selector no longer matches any pod. With the new Kubernetes integration in the Dash0 Operator, Agent0 reads that state itself during an investigation and connects it with your logs, metrics and traces, so you don't have to paste kubectl output into the chat. It's read-only: Agent0 can't change, restart or delete anything, gets no shell and can't read Secrets, and you choose which resources it sees. The connection starts inside your cluster, and the integration stays off until you turn it on (Operator v0.159.0 or later).

Read the changelog

Agent0 chat investigating checkout failures: it reads deployments, pods, services and endpoints in the shop namespace and finds that the inventory Service selector matches no pods

Code RED Podcast #51: Stop Fixing the Car. Fix the Factory: The Future of Software Development with Guy Podjarny

Guy Podjarny founded Snyk and now runs Tessl, and he tells Mirko what changed when his team stopped building on their own machines: every change starts as a ticket and ends as a pull request, and when the agent gets something wrong, they fix the skill that produced the code instead of the code. "Don't fix the car," as he puts it. The engineering work moves to the system that produces the software. More than 70% of the team improve the factory every week, and code review matters because it "codifies taste." It doesn't suit everyone, and Guy is open about that: one very good developer left because he didn't want to build that way.

Listen to the episode

Code RED podcast cover with Mirko Novakovic and Guy Podjarny: Stop Fixing the Car, Fix the Factory

Choice cuts

A few more, from the system that authenticates your agents to the plumbing that carries their telemetry.

Keycloak's observability journey: a guide for project maintainers

Keycloak authenticates people and machines, and its maintainers wanted telemetry built in. The OpenTelemetry Java agent added dozens of seconds to an otherwise near-instant startup, so they moved to the native Quarkus integration. Metrics came first, then traces, with exemplars linking the two. The Developer Experience SIG interview, co-written by Juliano Costa and others, also raises a question I hadn't thought much about: with push-based telemetry, can you verify which pod a metric really came from?

Read the post

Jaeger showing a Keycloak login trace, with spans for the authentication flow and its database queries

Deprecating the Collector Batch Processor

If you run the OpenTelemetry Collector, put this one on your radar. The batch processor is deprecated: exporter-level batching becomes the default in v0.171.0 (February 2027), and the processor is removed in v0.177.0 (May 2027). Taking it out of your pipelines is on you (a new queue_batch processor exists if you still want batching as a processor), and batching now happens after the sending queue, which changes how queue capacity behaves. We covered why this was coming in #24. Now there's a timeline.

Read the post

Cover reading "The Collector's batch processor has an end date." above a timeline: deprecated now, exporter batching by default in v0.171.0, batch processor removed in v0.177.0

When Missing Telemetry Is the Signal

"An empty dashboard can mean that nothing happened. It can also mean that the system stopped reporting what happened." This short post shows how to tell the two apart: reconcile counters across stages (accepted = completed + rejected + still processing), send a synthetic event through the whole path on a schedule, and alert on the Collector's own receiver and exporter counters instead of process uptime. It matters more once an agent reads your dashboards, because an agent handed silence will report that all is well.

Read the post

Cover reading "An empty dashboard can mean the telemetry stopped." above the check accepted = completed + rejected + still processing

Jaeger Previews AI-Assisted Troubleshooting and GenAI Observability for OpenTelemetry Traces at KubeCon

Jaeger v2.22 adds experimental AI features: an in-product assistant that shows its tool calls, an opt-in MCP endpoint with nine tools for services, traces, errors and critical paths, and trace views that understand the GenAI semantic conventions (model calls, tokens, tool execution). Agents fetch only the trace evidence each step needs, which keeps the model's context small and shows what each answer is based on.

Read the announcement

CNCF blog card for the Jaeger AI-assisted troubleshooting announcement, with the Jaeger mascot

A few more things from Dash0 over the last couple of weeks. If review is your bottleneck, AutoMerge, now generally available in Darkplane, takes the routine share off your reviewers: AI evaluators score each pull request against criteria you define, and anything below your threshold goes to a person. The Dash0 Operator can now instrument pods instead of workloads, so Argo CD, Flux and Terraform stop seeing drift. The SLO form builds latency SLOs on any histogram without hand-written PromQL. And Agent0 automations can start from changes in Linear.

Coming up: AWS re:Invent in Las Vegas, November 30 to December 3. Come find us at booth 250 in the Expo at The Venetian, left of the AWS Village, or at our coffee pop-up at Tra Viet on Thursday, 08:00–12:00. If it's your first re:Invent, Julia and Raphael, both AWS Heroes, did a 30-minute Conference Survival Guide on planning the week, first-timer tips and which observability, OpenTelemetry and AI sessions to book.

Until next time: let the platform enforce your agents' boundaries, check their output against a contract, and take batch out of your pipelines.

Kasper, out!

Hi, my name is Kasper!

I'm Kasper Borg Nissen, Director of Developer Relations at Dash0. I'm passionate about Observability and bridging the gap toward developers through Platform Engineering. I've previously worked 8 years as a platform engineer, I'm a former co-chair of KubeCon+CloudNativeCon, and I'm genuinely obsessed with all things cloud-native and open standards.