The Stack Finally Caught Up

Collecting customer data, governing how it moves, enriching it, and making it available in real time for experimentation and personalization has always meant assembling the pieces yourself. A collection engine. A governance layer. A real-time API. A historical store. A consumption platform. Each from different vendors, integrated by hand. I’ve worked inside that complexity. I know what it takes to make it work and what breaks when the seams give.

At Databricks Data + AI Summit this week, Ali Ghodsi laid out an architecture that looked familiar. Lakeflow replacing the message bus. Lakebase unifying the warehouse and the lakehouse. Unity AI Gateway as the single entry point for all agents, handling authentication, observability, safety, and compliance across any cloud. Genie Ontology constructing a knowledge graph across all organizational assets so agents have context before they start. CustomerLake as the agentic CDP, with profile agents handling identity and enrichment and campaign agents handling activation.

The architectural patterns aren’t new. What’s new is that they’re now a platform decision rather than an integration project. That changes the speed and the governance model entirely.

That last part matters most. When infrastructure is native, governance stops being something you bolt on at the end. When data never leaves the warehouse, compliance becomes structural rather than procedural. When agents operate on a unified governed surface, you have a fighting chance at controlling what they do.

Which brings me to what stood out in an otherwise overhyped session on autonomous agents. Someone said it plainly: agents fail silently, autonomously, and at machine speed. A bad decision doesn’t signal an error or a system compromise. It just happens. Most organizations experimenting with agents today can’t explain why an agent failed without runtime visibility they don’t yet have. Autonomous systems without that visibility become ungovernable.

The stack caught up. The governance model hasn’t.

AI transformation is data transformation. Databricks said it directly this week. The organizations that get there aren’t the ones with the most ambitious AI roadmaps. They’re the ones that did the hard work on the data foundation and are now asking the right questions about what comes next.

The infrastructure is ready. The harder problem is just beginning.