Understanding how Joule Assistants orchestrate

I've had a lot of customer meetings lately where I'm asked to explain something I genuinely believe is one of the most impressive things SAP has built in AI: how Joule Assistants actually orchestrate work behind the scenes. With more meetings coming up and the same question certain to land again, I wasn't happy with the answer I'd been giving. I needed an easier way to explain it.

One night, I booked a restaurant for my wife and me. My wife has some dietary constraints due to allergies, so when the waiter arrived, she ordered the tasting menu but told him “no shellfish, no gluten, and no lactose.” The waiter went to the kitchen, and while we were waiting, I found myself looking through the pass window. And something about the way the head chef was operating and directing the kitchen made me immediately see the analogy I'd been looking for.

In the kitchen, the head chef received our order, and I could see him reading and reasoning about it. He was understanding the request, breaking it down into tasks, and triggering, coordinating, and calling out tailored instructions to each station — accounting for every dietary constraint, adapting every course. “Course one, swap the crab for avocado, check the dressing is gluten-free, clean your board first.” “Course three, seabass instead of scallop, olive oil only, no lactose on that pan.” “Pastry — scrap the tart shell, do the berry sorbet, double-check it's gluten-free.”

Each kitchen station got their specific instructions and nothing else. Some were working in parallel. Others were waiting for a prior step to complete. They didn't need to know what the other stations were doing. They just executed their piece and reported back.

Meanwhile, we were sitting at the table, having simply ordered dinner and waiting for it to arrive. What was happening in the kitchen was something we didn't need to think about. That's exactly how Joule Assistants work.

By the time dessert arrived, I had the whole thing mapped out. Let me walk you through it, because once you see the parallel, it's hard to unsee.

When a supply chain planner, a product engineer, or a maintenance supervisor type a request in Joule, that request moves through four distinct layers before anything happens in a SAP system. Each layer has a specific job, and each one maps neatly to someone in that restaurant.

Figure 1: The four layers between a user request and a system action

The first layer is the Joule Orchestrator. In the restaurant story, it would be the waiter. It reads your request, classifies the intent, checks the catalog of registered capabilities, enforces your access permissions, and routes to the best-matched Assistant. It doesn't cook anything. It just makes sure your order gets to the right kitchen and the right head chef.

The second layer is the Domain Assistant; in our story, it would be the head chef. This is where the real orchestration happens. The Assistant doesn't follow a predefined script. It plans and reasons by leveraging an LLM, the same way a skilled manager would think through a complex problem. It reads the request, identifies what's actually being asked, figures out which specialists can help, decides the order of operations, dispatches them, evaluates what they return, and synthesizes everything into a recommendation. That's a lot of invisible work happening before you see a single result.

The third layer is the Domain Agents; the kitchen stations. Each one is a specialist. They have a narrow, well-defined scope: one might be focused on demand trend analysis, another on inventory allocation modelling, a third on network capacity. They don't decide the broader workflow. They execute their piece, access the relevant SAP systems, and report back to the Assistant with structured findings.

The part that keeps coming back to me is the reasoning loop. Every request triggers a process inside the Domain Assistant. Understanding this is what separates a surface-level explanation from a real one.

Figure 2: The five-step reasoning loop every Domain Assistant runs

Step one is understanding the job. Not just reading the words, but working out the domain, the scope, the constraints, and what “done” actually looks like.

Step two is assembling the team. The Assistant selects which Agents to involve and deliberately excludes the ones that aren't relevant.

Step three is sequencing and dispatching tasks. If the tasks are independent of each other, the Assistant fans them out in parallel to save time. If they depend on each other — like a root cause analysis that must precede a resolution plan — they run in sequence, each one building on the last.

Step four is adaptive replanning. This is what separates orchestration from automation. Take a manufacturing quality use case example: the Manufacturing Assistant might initially plan to run an inspection readiness agent. But when the diagnostic agent comes back and reports that the root cause is contaminated supplier material — and not an equipment issue — the Assistant drops the inspection agent and adds a supplier nonconformance agent instead, mid-task. A rigid workflow would have run the wrong agents. The Assistant evaluated the evidence and changed course.

Step five is synthesis and governance. The Assistant combines all findings into a coherent recommendation and then decides about execution: sometimes the right answer is just showing you the insight, and sometimes it executes a direct action in a system.

Now we’re able to understand how a Joule Assistant orchestrates, but one piece is still missing: how do Assistants and Agents talk to each other? Every agent could be built by a different team, running on a different platform, in a different part of the enterprise. Without a shared protocol, connecting them would require building a custom bridge for every pair. This is exactly the problem the Agent-to-Agent (A2A) protocol was designed to solve.

Figure 3: The three phases of A2A communication between Assistants and Agents

The protocol works in three steps. The Assistant first discovers an Agent by reading its “digital business card” — a structured document that describes what the Agent can do, what inputs it expects, how to reach it, and what outputs it can deliver. Then it delegates by sending a formal task object that includes the intent, parameters, constraints, and a channel for real-time updates. Then the Agent executes and collaborates, streaming progress back as it works, not just returning a result at the end.

Alongside A2A, agents use MCP (Model Context Protocol) to connect to APIs, planning models, data products, knowledge graphs, and third-party services. If A2A is how agents talk to each other, MCP is how agents access the tools they need to do the work.

One additional thing to understand is that a Joule Assistant and a Joule Agent are built from the same technology. Both are developed in Joule Studio. Both use LLMs for planning and reasoning. Both run on SAP Business AI Platform. Both are registered in the SAP AI Agent Hub. Both use the A2A and MCP protocols. Technically, they are the same construct, however, what differentiates them is a design decision. An Assistant is built with a broad, role-aware scope and the authority to coordinate others. An Agent is built with a narrow, task-specific scope and the authority to execute within it. Same kitchen, but different job descriptions.

When a planner, engineer, or supervisor makes a request in Joule, they express their intent once. The Joule Orchestrator routes it. The Assistant reasons through it and delegates the tasks. The Agents execute. The systems get updated. The human gets a recommendation and approves the action. Users never needed to know which agents were involved, in what order, or which systems were touched. They simply got an answer.

Source link