SAP Community already has a great beginner series on Joule. This post is for architects and consultants who already know what Joule is and want to understand how it actually works under the hood.
Before going further, a quick scope note. This post focuses on Joule as an end-user AI copilot embedded within SAP business applications such as S/4HANA, SuccessFactors and Ariba. SAP has since expanded the Joule portfolio to include Joule Studio, Joule for Developers and Joule for Consultants, among others. The grounding architecture described here is most directly relevant to Joule's role as a business copilot, though many of the underlying mechanisms apply across the portfolio.
What is Grounding and Why Does it Matter?
Without grounding, an AI answers questions based on its general training data. The responses may sound fluent and confident but have nothing to do with your organisation's actual data, processes or documents.
Grounding connects the AI to your specific business context. It combines generative LLMs with information retrieval techniques to improve response quality and accuracy, without the complexity and cost of fine-tuning a model on company-specific data. The result is responses that are relevant, accurate and actionable.
This is not a nice to have. It is the difference between a generic chatbot and an enterprise grade AI copilot.
How Joule is Grounded
Joule does not rely on a single grounding mechanism. It combines multiple approaches, each suited to a different type of data and scenario.
SAP Help Documentation
Joule retrieves information from SAP's official Help Portal and documentation using Retrieval-Augmented Generation (RAG). SAP's documentation corpus covers millions of pages across all SAP applications. It is processed using chunking, vectorisation and embedding techniques, then stored in the SAP HANA Cloud Vector Engine for semantic retrieval.
Ask Joule a procedural question like “How do I create a purchase order in Ariba?” and it retrieves the most relevant documentation chunks and generates a contextual answer grounded in SAP's official content. This layer is managed entirely by SAP. Customers do not configure it.
SAP Domain Models
SAP Domain Models are AI models trained specifically on SAP domain knowledge – including code, data, metadata, business processes, architecture knowledge and documentation. When a user initiates an enquiry or wishes to create code, these models are designed to provide results firmly grounded in the SAP context rather than relying on generic internet knowledge.
Combined with context graphs and agents, SAP Domain Models bring deep SAP knowledge directly to Joule, Joule Agents and SAP applications. This is a distinct grounding layer that complements documentation retrieval – while SAP Help Documentation retrieves relevant content at query time, SAP Domain Models carry SAP knowledge baked into the model itself.
https://www.sap.com/products/artificial-intelligence/ai-models/domain-models.html
Document Grounding
For organisations that need Joule to answer questions based on internal knowledge, SAP AI Core provides document grounding capabilities. Think SOPs, process guides, compliance policies, onboarding manuals.
The process works as follows. Documents are retrieved from connected enterprise content repositories, chunked and embedded into vector representations, then stored in the SAP HANA Cloud Vector Engine. When a user asks a question, a similarity search retrieves the most relevant chunks, which are then injected into Joule's context to generate a grounded response.
Out of the box, Microsoft SharePoint is the primary supported connector for Joule's built-in shared document grounding service. Supported file formats include PDF, Word, HTML and common image formats. Content is refreshed daily.
Organisations that need broader connectivity can use SAP AI Core's private document grounding pattern, which supports additional repositories including SAP Build Work Zone, SAP Document Management Service, Google Drive, AWS S3, SFTP and ServiceNow.
https://help.sap.com/docs/sap-ai-core/generative-ai/grounding-035c455a5a424697b60f4a24b6d791fe
SAP Business Data Cloud and SAP Knowledge Graph
SAP Business Data Cloud (BDC) provides the unified data foundation – federating and harmonising data from S/4HANA, SuccessFactors, Ariba and other SAP and non-SAP applications into managed data products. These data products are made available directly on the SAP Generative AI Hub, which grounds the LLM in trusted organisational data. This is the architectural link between BDC and Joule: BDC data products flow into the Generative AI Hub, which uses them to inform Joule's responses.
But data alone is not enough. Joule also needs to understand the meaning and relationships between business objects across systems. An Employee in SuccessFactors and a Cost Center Owner in S/4HANA are the same person. A Vendor in Ariba and a Supplier in S/4HANA Finance refer to the same entity.
This is where SAP Knowledge Graph comes in. Working alongside BDC, it provides fully managed ontologies that help Joule understand processes, relationships and context across the enterprise. Because Joule uses SAP BDC data products and the SAP Knowledge Graph, it can use fully managed ontologies to understand processes, relationships and context across the enterprise, enabling more comprehensive and accurate insight.
BDC and Knowledge Graph are distinct but work together. BDC provides the data. Knowledge Graph provides the semantic understanding of what that data means and how it relates.
Underpinning all of this is SAP HANA Cloud, the database at the core of BDC. It brings together vectors, graphs, relational and spatial data with business context in a single in-memory database, helping to ensure reliable agent reasoning.
Direct API and OData Integration
For scenarios requiring live transactional data, Joule can invoke SAP application APIs directly rather than relying on replicated analytical data.
What is worth highlighting here is that Joule does not just read data. It can act on it. For example, Creating a purchase order, approving a workflow item, triggering an automation via SAP Build Process Automation – all through natural language. This is where Joule moves from being an informational copilot to an operational AI agent.
How it All Fits Together
The foundation for everything described above is the SAP Business AI Platform – the infrastructure layer that brings together SAP AI Core, the Generative AI Hub, BDC data products and the grounding pipeline. Before any grounding mechanism is invoked, Joule performs intent detection and orchestration within this platform, routing queries to the appropriate grounding source.
User Question
↓
Intent Detection and Orchestration
(SAP Generative AI Hub on SAP Business AI Platform)
↓
┌────────────────────────────────────────────────────────────────┐
│ SAP Help Documentation (HANA Cloud Vector Engine) │
│ SAP Domain Models (SAP-trained, context graph grounded) │
│ Document Grounding (SharePoint / AI Core connectors) │
│ SAP Business Data Cloud data products │
│ SAP Knowledge Graph (ontology and semantic layer) │
│ SAP HANA Cloud (vectors, graphs, relational, spatial) │
│ Direct API → SAP Applications (read and write) │
└────────────────────────────────────────────────────────────────┘
↓
LLM reasoning over grounded context
↓
Response or ActionWhat This Means for Your AI Roadmap
Organisations planning a Joule rollout should assess their grounding readiness before going live, not after.
SAP Help documentation grounding and SAP Domain Models require nothing from the customer – SAP manages them. For the remaining layers, there are real questions to answer. Where does your internal knowledge live, and is SharePoint available or do you need a private grounding setup? Is SAP Business Data Cloud part of your landscape – if not, the cross-system grounding will be limited. Are you tracking the maturity of SAP Knowledge Graph for your specific applications? And for agentic scenarios, what authorisation and governance controls need to be in place before Joule starts executing business actions?
These are not technical edge cases. They are architecture decisions that directly determine how useful Joule will be in practice.
