QCon London 2026: Running AI at the Edge - Running Real Workloads Directly in the Browser

Browser-based AI inference represents a meaningful shift in how we architect data pipelines, and I've started seeing this reflected in real production decisions. When workloads execute client-side using technologies like ONNX or WebAssembly, we eliminate the round-trip latency that typically haunts real-time applications, while simultaneously reducing pressure on our inference infrastructure. The privacy angle isn't just marketing—it's operationally significant. Keeping sensitive data off centralized servers changes our compliance posture and reduces data residency complexity. From a data engineering perspective, this forces us to reconsider our traditional hub-and-spoke model. We're now designing hybrid architectures where feature stores and lightweight models live closer to end users, while heavier computation and retraining pipelines remain centralized. The challenge isn't technical capability anymore; it's orchestrating consistent model versioning across distributed edge clients. My concrete recommendation: audit your current AI workloads for candidates that are latency-sensitive, privacy-constrained, or cost-sensitive at scale. Start with inference-only use cases where model staleness is acceptable, then gradually experiment with edge deployment patterns. This isn't a wholesale migration—it's strategic architectural evolution.