New platforms expand what AI can do. Trusted definitions, clear ownership and business context help determine whether it does the right thing.

By Randy Rouse

Watch the flagship video

Watch: We Modernized the Data Stack. Did We Modernize the Data?

The full video explores where value actually comes from after a data-platform modernization, including ownership, context, agent authority and the cost of proving a useful result.

An oil and gas customer of mine chose Snowflake as part of modernizing its data pipeline, expecting the move to simplify years of unresolved data management work. As the migration progressed, the team told us consumption spending had exceeded budget expectations. The conversation returned to a more fundamental question: which data should we move, and what purpose would it serve?

That is the conversation I keep having, platform by platform, customer by customer. Snowflake, Databricks, Microsoft Fabric and BigQuery each represent a genuine leap in capability, and each takes a different approach to delivering it. If I were a CDO, I would absolutely be evaluating how to use them.

But there is a question I think we need to ask more often: after we modernize the data stack, what have we changed about the data itself?

Do we understand it better? Have we resolved conflicting definitions? Do we know which sources to trust, who owns them and when the information is fit to use? Can we connect the investment to a better business decision?

Those questions become even more consequential when an AI agent uses the information to act.

With that oil and gas customer, we had previously discussed how the organization could understand, model, govern and make its data useful across the business. Modeling, cataloging, governance, lineage and data quality were all part of the conversation. The team began moving data in phases.

We did not review the underlying billing figures, and I cannot attribute the spending to a particular workload or processing pattern. What I can describe is the change in the conversation.

The team quickly recognized that lift-and-shift was not the right approach for its situation. Identifying trusted data became a critical part of deciding what to move. Why bring old, duplicate or untrusted information into a new environment without a clear purpose for it?

The discussion returned to understanding the data and selecting what would support production workflows. The platform remained valuable. Getting that value required more deliberate decisions about what belonged there.

Before we ask where to move the data, we should ask what deserves to move.

Historical and raw data can absolutely have a purpose, including investigation, reconciliation and retention. That purpose should inform where it lives, how it is accessed and how much processing it needs. Every dataset does not need the same treatment.

The experience reminded me of earlier public-cloud conversations: enthusiasm about a new operating environment followed by a harder examination of workload fit, cost and responsibility. I see a similar pattern in some of today's data conversations.

The spending concern reopened a broader discussion about purpose, ownership and workload design. Those same decisions become essential when agents use the data to make recommendations or take action.

At conferences and with customers, I'm seeing what I think of as a platform-first approach to modernization. In my conversations, that most often means Snowflake or Databricks, including Databricks on Azure. Fabric and BigQuery are part of the discussion too. Organizations may have decades of disparate systems behind them, without consistent modeling, cataloging or ownership practices. They are looking for a simpler way forward.

I understand the appeal. All four of these platforms provide capabilities most organizations would struggle to build and operate themselves. They deserve credit for making ambitious analytics and AI work possible.

Capability and discipline

IFCO shows what can improve when a team combines platform capabilities with deliberate engineering and operating discipline. In a published customer account on Databricks' blog, its team describes engineering changes that reduced a core job's runtime from approximately seven hours to two hours and twenty minutes, and reduced daily compute cost by 58%. The account also describes ownership and testing requirements for its data models. These are customer-reported operating improvements published by the vendor, rather than independently audited migration returns. IFCO case

The specifics in IFCO's case are Databricks and dbt. But the operating questions apply across platforms: who owns a model, what gets tested, what triggers a rerun and what gets retired? The gains come from how the capabilities are put to work.

For some organizations, deliberate placement will mean a hybrid approach. Others may consolidate more extensively. The decision should follow the workload, business need and operating economics.

I can buy platform capabilities. I still have to own what my data means.

When meaning becomes action

Consider a hypothetical company where sales measures bookings and finance measures recognized revenue. Both figures can be valid. They answer different questions. Now ask an agent whether the business can afford additional hiring. Selecting a recent, approved figure is only part of the task. The agent needs the appropriate financial context and decision rules, and a boundary around what it may recommend or authorize.

A faster query does not settle which measure belongs in that decision. An approved definition does not, by itself, establish that the underlying calculation is correct or suitable for the question.

Platform providers are building tools to address this context problem. In an October 5 account of its own product team's work, Databricks describes using Genie Ontology for a weekly adoption review. Its example combines certified metrics with context from documents and other systems. It describes OntoRank prioritizing context using signals including certification, usage and authorship. This is a documented internal example reported by the vendor, rather than independent evidence of customer outcomes. Databricks internal-use account

The leadership question remains: who establishes those definitions, resolves disagreements and validates that the result fits the decision?

Trusted inputs are necessary. The calculation, interpretation and resulting action still need validation.

Six questions for leadership

Business accountability has to guide the technical work. Leaders need the people who understand the business working with the people who understand the systems. Neither group can settle the whole problem alone.

For one consequential AI workflow, I would want six questions answered:

  1. What outcome are we improving? Name the business owner, establish a baseline and define success. A faster process matters when we understand what that speed makes possible.
  2. Who owns the meaning? Identify who owns the definitions and which source is authoritative for this decision. Establish how exceptions and disagreements get resolved.
  3. When is the information fit to use? Define acceptable freshness and quality, and how changes in structure or meaning will be detected. Recent information can still be misleading. Decide when the workflow must pause or escalate.
  4. What may the agent do? Establish its identity, what information it may access and which actions it may take. Reading, recommending and committing a business transaction require different boundaries.
  5. How will we validate the result? Check calculations and decisions, capture what happened and create a way to correct the workflow. Feedback must affect the next decision.
  6. Was it worth doing? Compare the benefit with the cost of preparation, execution, review, exceptions and ongoing operation. Then decide whether to expand, adjust or stop.

These questions connect directly to the narrative I've been exploring. Value lives in the capability and outcome. Identity determines who may act. Trusted data supplies context. Observability provides evidence. Infrastructure and operations make the work repeatable.

The expertise inside an organization is part of what makes it distinctive. Modern platforms can help that expertise reach further and work faster. Leadership remains responsible for deciding whether the result serves the business.

Start with the data required for one valuable workflow. Define its owners, sources, quality requirements and action boundaries, then expand as the evidence supports it. You can make progress on that workflow while the broader work of improving your data continues.

The platform can accelerate the work. The business still has to prove the value.

Short videos this week

Reach Out, Let's Talk. @RandyJRouse