Presentation: Directing a Swarm of Agents for Fun and Profit

The shift from cloud-native to AI-native architecture represents a fundamental change in how we decompose problems. Instead of orchestrating microservices, we're now directing autonomous agents that make decisions within bounded contexts. This matters for data teams because our traditional separation between transformation logic and application logic collapses. When Claude or Cursor agents autonomously generate and execute queries, we lose explicit lineage tracking and governance checkpoints we've relied on for years. The practical implication: data engineers must shift from building pipelines to building guardrails—implementing validation layers, audit trails, and cost controls around agent-generated operations. The director-level approach Cockcroft describes isn't new conceptually; it mirrors supervisor patterns in distributed systems. What's different is the velocity of change and the opacity of decision-making. My recommendation is to invest immediately in agent observability infrastructure before agents proliferate across your organization. This includes tracing agent-to-data interactions, capturing decision rationale, and establishing clear boundaries around what data agents can access or modify. Without this foundation now, you'll inherit compliance nightmares later.