The conversation around enterprise AI has a pattern problem. Organizations pour resources into pilots, build enthusiasm around demos, and then watch momentum stall somewhere between proof of concept and production. Session 3 of the Architecting the Autonomous Enterprise webinar series tackled this head-on, not with more excitement about what AI can do, but with a diagnosis of why it so often fails to scale with John Santic and Jason Taylor

We now have more than 6,600 registered attendees across 80 countries, representing over 2,100 unique customer and partner organizations. That reach reflects a genuine appetite for grounded, architecturally serious conversations about AI.

John Santic examined the structural conditions that make enterprise AI fail to scale safely, and the architecture required to fix them. Jason Taylor grounded that framework in two specific SAP capabilities: Joule Work and the SAP Knowledge Graph.

Registration link for access to all the recordings and slide and to join the forthcoming sessions! https://events.sap.com/eaa-autent-web/en_us/home.html

The State of the Market: Ambition Without Architecture

Before diving into solutions, both speakers established why this conversation is urgent. Research cited in the session points to a 2x revenue boost available to organizations that can deliver safe and governed AI at scale. The appetite is real. The execution gap is equally real. The numbers are striking in their imbalance. By the end of 2026, enterprise AI technology spend is expected to reach $209 billion. Governance spend: less than $1 billion. The average AI initiative failure is estimated to cost $4.4 million. Only 21% of organizations have a mature governance model in place. More than half are running AI without any formal oversight.

The First Mile Problem: Why Judgment Cannot Be Improvised

John Santic opened with a provocation that deserves more attention than it typically gets in AI conversations: enterprise work was designed for human execution, not machine execution. That distinction sounds obvious. Its implications are not.

When humans work, they carry an enormous amount of invisible context, what John described as everything below the waterline in an iceberg. Linguistic understanding, semantic relationships between concepts, epistemological judgment about what is actually true versus what was merely told to us, and what Aristotle called phronesis, the practical wisdom to know when a rule applies, when it does not, and what to do in the space between.

AI agents inherit the task. They do not inherit the judgment. They are, as John put it, “prediction machines sophisticated enough to look like judgment but containing none.”

Companies fill that gap with prompts, fragments, and markdown files. That patchwork approach produces three specific categories of risk:

• Single agent context risk. The natural language prompt becomes the primary contract for agent execution. The result is that governable artifacts get compressed into unstructured text with no audit trail, unacceptable for any regulated process.
• Multi-agent fragmentation risk. As agent networks grow, enterprise knowledge becomes noise stitched together. Agents reason over fragments instead of operational truth. The risk compounds with every additional agent.
• Prompt sprawl and MD file chaos. What begins as an organized system of instructions quickly degrades into overlapping, duplicated, out-of-date files that bear no reliable relationship to how the business actually operates. It is not a sustainable governance model.

Compounding all three is the economics reality. AI usage is currently heavily subsidized, that will not last. As compute costs normalize and context burdens grow, organizations spending frontier model economics on low-value work will face a hard reckoning. John's phrase for it: “We are baking bread with gold.” The conclusion: context cannot be improvised at query time. It has to be engineered as a system before any agent fires.

Context as a System: A Five-Layer Architecture

John Santic's answer is not a governance policy document. It is a governance architecture, a five-layer context stack where each layer resolves a specific dimension of the judgment gap that AI systems cannot fill on their own.

Every agent draws from the same complete operational record, regardless of where data lives. SAP Business Data Cloud, with Apache Iceberg-native capability, anchors this layer with an open, enterprise-wide data fabric, the most complete operational picture of the business available to any agent at runtime.

• Layer 2, Semantic and Knowledge Context.

Every agent operates on the same semantic truth about the business. The SAP Knowledge Graph resolves relationships of meaning, master data, behavioral conditions, how concepts connect across domains, so agents do not have to infer or approximate what things mean. This is the layer where agents stop guessing and start reasoning from shared definitions.

• Layer 3, Process and Predictive Context.

Signavio's process intelligence surfaces not just how human and system workflows run, but how agents are running within them. The Prior Labs acquisition adds a causal dimension, identifying the specific reason a particular outcome will occur before it arrives. This layer also introduces one of the session's most important concepts: process atoms.

Process atoms are behavioral, context-bound units of work. They encode explicit human logic that was never systematized, the tacit rules, exception conditions, and stop criteria that experienced practitioners carry but rarely document. For an AI agent, a process atom defines what must happen and what must not, with guardrails built into the definition itself. When a business condition changes, updating the atom propagates that change to every agent using it, centrally, explicitly, and auditably. Decision traces harvested from agents in production can, in turn, be used to refine those atoms over time.

• Layer 4, Orchestration and Governance Context.

Most AI governance today is post hoc. The consequences of what an agent did are discovered after the fact. This layer inverts the logic. With SAP AI Agent Hub and LeanIX, authorizations, approval structures, and segregation of duties are embedded into the conditions of execution, not implied in an audit trail reviewed after the business has already felt the impact. This is the difference between governance as a document and governance as an architecture layer.

• Layer 5, Intent and Prescriptive Context: Company Memory.

This is the least discussed but arguably most critical layer in the market conversation about AI governance. Company memory is the governed, persistent repository of organizational behavior knowledge, everything agents need to act correctly, compounding and staying current as the organization evolves. Without it, every new agent and every new workflow must be endlessly re-described. With it, the enterprise gets progressively better at communicating itself to the machines it deploys.

The architecture matters for a reason that goes beyond governance. Every layer built into the context stack is context that no longer needs to be sent at query time. The burden moves out of the agent layer, where it is unstable, expensive, and ungovernable, and into a managed system. Better economics, better governance, and greater scale follow from the same structural decision.

Registration link for access to all the recordings and slide and to join the forthcoming sessions! https://events.sap.com/eaa-autent-web/en_us/home.html

The State of AI: Jason Taylor's Market Context

Jason Taylor opened his session with a broader orientation to the AI landscape before drilling into SAP-specific architecture. The growth curve of AI capability has been extraordinary. From AlexNet's early image recognition in 2012, to the “Attention Is All You Need” transformer architecture paper from Google in 2017, to the emergence of the first true large language models and the explosion that followed, the pace of change has been genuinely unprecedented. Models are now multimodal, capable of extended reasoning, and operate with context windows exceeding one million tokens. RAG enables access to data outside model training. MCP enables external tool calling. Agent AI, a model in a loop with tools, memory, and a goal, has arrived as the defining architecture of the current era.

The challenge Jason was clear about: impressive as these capabilities are, the enterprise production bar is substantially higher than a prototype or a single-user deployment. Security, explainability, identity and access management, observability, auditability, the full set of non-functional requirements that good architects have always required, apply with equal force in the agentic era.

The North Star Architecture and Joule Work

Jason grounded the broader AI discussion in SAP's AI-Native North Star Architecture, a four-layer vision that provides the structural context for where Joule Work sits. At the top, an adaptive UX layer with Joule as the central engagement point. Below it, an agentic process layer where every application becomes a capability provider for agents. Below that, a foundation layer serving up data and business context. At the base, a platform layer hosting applications and agents reliably. Agent-to-agent protocol governs how agents communicate. MCP servers handle tool calling. BDC Connect facilitates zero-copy data sharing.

Joule Work represents a fundamental inversion in how enterprise software operates. Rather than navigating between applications to accomplish work, users express intent in plain language, and Joule pulls the data, coordinates the agents, and runs the work across every connected system, SAP and non-SAP. Three experiences define the interface:

• Conversations, intent-led dialogue, the familiar chat surface
• Spaces, dynamic, AI-generated work environments that adapt content to user intent and context, surfacing relevant information for real-time analysis without the noise of irrelevant data
• Develop, the developer environment for building, testing, and deploying AI agents and applications with Joule Studio

For enterprise architects, the implication is substantial. Designing an engagement architecture now requires accounting for AI orchestration as a first-class element. That means headless access to underlying system capabilities and a strong API-first posture. Organizations that invested early in API governance will find themselves ahead of the curve.

Behind the interface, each autonomous domain, Finance, Supply Chain, Procurement, HCM, Customer Experience, brings specialized assistants and agents built on deep process knowledge, data context, and governance frameworks. Jason illustrated this with a lead-to-cash example: agents can handle structured, repeatable tasks across closing activities, replenishment cycles, sourcing events, and service resolutions, while maintaining enterprise controls and full auditability. People focus on strategic thinking and judgment calls. Agents handle the structured work in between.

Joule is also certified to ISO/IEC 42001, the international standard for responsible development, deployment, and operation of AI systems, and works within established SAP role-based access controls. Customer data is not shared with third-party LLM providers for model training.

 

The SAP Knowledge Graph: Making Agents Genuinely Useful

The SAP Knowledge Graph provides the semantic foundation that separates agents that are confidently wrong from agents that reason accurately over how the business actually operates.

It links natural language inputs to SAP's structured metadata, spanning API definitions and endpoints, business semantics (what revenue means, how customer relates to order, how processes connect across domains), data product metadata including schema, lineage, freshness, and authorization rules, and customer-specific extensions including custom fields, custom APIs, and tenant-specific configurations. The graph runs on SAP HANA Cloud and uses open standards: Resource Description Framework (RDF) and Web Ontology Language (OWL). It traverses knowledge across the full SAP portfolio and can be enriched over time as agent decision lineage and human corrections are fed back into it.

The practical effect: agents do not have to infer, guess, or apply generic training data that is not relevant to a specific organization. Finance, sales, procurement, and supply chain are understood as an integrated whole, with relationships, interdependencies, and domain-specific meaning built into the context the agent reasons from. Hallucinations decrease. Decisions can be trusted and acted upon.

For architects and data teams, Jason's guidance was clear: treat the semantic architecture as a first-class design element. Canonical business object definitions, enterprise-wide semantic standards, and quality metadata governance will directly determine how useful the Knowledge Graph becomes as an agent substrate

Three Imperatives for Architects

Both speakers converged on a clear set of responsibilities for the enterprise architecture community:

Own the context stack. The architecture that supports managed, governed context for AI is not an infrastructure decision to be delegated. It is an architectural one that requires EA leadership. Without it, context will continue to be improvised at runtime, unstable, expensive, and ungovernable.

Operationalize AI governance as architecture, not policy. The difference between a governance policy document and a governance architecture layer is the difference between discovering what an agent did and determining what an agent can do before it fires. EAs have the tools, the mandate, and the positioning to lead this shift.

Anchor on outcomes, not outputs. Translate AI activity into AI capability, with measurable business value, clear ownership, and the structural accountability that turns a pilot into a production system. Jason's framing was direct: we all lived through big data, RPA, low-code/no-code, and microservices. Each required EA to be the tip of the spear for thoughtful, value-anchored adoption. Agentic AI is no different.

What Comes Next

Session 4 moves from architecture to build. The focus will be Joule Studio, covering its architecture, the path from intent to deployment-ready agent, and a live hands-on demonstration with the product team.

If you are designing for the autonomous enterprise, this is the session where the blueprint meets the development environment.

Registration link for access to all the recordings and slide and to join the forthcoming sessions! https://events.sap.com/eaa-autent-web/en_us/home.html

Architecting the Autonomous Enterprise is a 13-episode webinar series. Recordings, session summaries, and community discussions are posted to the SAP Community Enterprise Architecture Group after each session.