During this year I ran a workshop showing how to connect Cline directly to SAP AI Core. 

In that case there is no middleware, it is a direct integration. After the session, several attendees asked the same question: “Can I do this with other AI coding agents?”

This post answers that question. Using LiteLLM as a universal proxy layer you can route for example Claude Code through SAP Generative AI Hub.

This post focuses on getting the integration working end-to-end. We won't deep-dive into LiteLLM topics. You can check the LiteLLM documentation. If you hit an issue with LiteLLM itself, that's your reference.

For more information about the integration between LiteLLM and SAP Generative AI Hub, please check the following guide.

How It Works

The integration adds one layer between your AI coding agent and SAP's models:

Claude Code
    │
    │  Anthropic API format
    ▼
LiteLLM Proxy (localhost:4000)
    │
    │  SAP AI Core API format
    ▼
SAP Generative AI Hub
    │
    ▼
Anthropic / OpenAI models

1. Claude Code sends standard Anthropic API requests.
2. LiteLLM receives those requests and translates them into the format SAP AI Core expects, including authentication via your service key.
3. SAP AI Core enforces your organization's compliance requirements, then routes the request to the underlying model.

Prerequisites

Before you start, make sure you have:

1. SAP BTP account with an AI Core service (extended plan) instance provisioned and a service key downloaded.
2. Models deployed in your SAP AI Core resource group. This post assumes you already have models available. Deploying models in AI Core is out of scope here.
3. Claude Code installed.
4. Docker and Docker Compose.

Note: LiteLLM can run locally according to the following guide or containerized with Docker. This walkthrough uses Docker Compose because it bundles LiteLLM together with a Postgres database and Prometheus metrics out of the box, a good fit if you want a persistent managed setup. If you just want a quick local test, the LiteLLM docs cover that install path. For this demo, we go with Docker.

The Setup

Step 1: Pull the LiteLLM image

LiteLLM provides a dedicated image for proxy deployments that connect to Postgres:

docker pull ghcr.io/berriai/litellm-database:main-latest

Step 2: Get docker-compose.yml

Create a new folder that would be our workspace, in my case I called it: “litellm-sapaicore”, and then, download the official Docker Compose file:

curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml

The file defines three services:

Service Port Purpose

Before moving on, open docker-compose.yml and verify that the config.yaml volume mount and --config flag are uncommented:

services:
  litellm:
    volumes:
      - ./config.yaml:/app/config.yaml  # must be uncommented
    command:
      - "--config=/app/config.yaml"     # must be uncommented

Step 3: Create .env

Create a .env file in the same directory as docker-compose.yml:

LITELLM_MASTER_KEY="sk-your-master-key"
LITELLM_SALT_KEY="sk-your-salt-key"
AICORE_SERVICE_KEY='{"clientid": "your-clientid", "clientsecret": "your-clientsecret", ... }'
AICORE_RESOURCE_GROUP="..."

Step 4: Create config.yaml

This file maps the model names Claude Code (or any client) will use to the actual SAP AI Core model identifiers:

model_list:
  # OpenAI models
  - model_name: gpt-4o
    litellm_params:
      model: sap/gpt-4o

  # Anthropic models — note the double-dash in the model ID
  - model_name: claude-sonnet-4-5
    litellm_params:
      model: sap/anthropic--claude-4.5-sonnet

  - model_name: claude-opus-4-7
    litellm_params:
      model: sap/anthropic--claude-4.7-opus

  # Embeddings
  - model_name: text-embedding-3-small
    litellm_params:
      model: sap/text-embedding-3-small

litellm_settings:
  drop_params: true
  set_verbose: false
  request_timeout: 600
  num_retries: 2
  forward_client_headers_to_llm_api: ["anthropic-version"]

Anthropic model IDs in SAP AI Core use a double-dash separator: sap/anthropic--claude-*. OpenAI models use a single slash: sap/gpt-4o. It is important that you prefix the model name with sap/. For a full list of the correct model names, check SAP Note 3437766.

Step 5: Create prometheus.yml

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]

Important: This file must exist as a file on disk before you run docker compose up. If it is missing, Docker will auto-create it as an empty directory, and the Prometheus container will fail to start with a confusing mount error.

Step 6: Start the stack

docker compose up -d

Wait about 30–40 seconds for the deployment, then do a quick end-to-end smoke test:

curl -X POST http://0.0.0.0:4000/v1/messages \
  -H "Authorization: Bearer sk-your-master-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-7",
    "max_tokens": 1000,
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

A successful response looks like:

{
  "id": "...",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-7",
  "content": [{"type": "text", "text": "The capital of France is Paris."}],
  "stop_reason": "end_turn"
}

If you get a valid JSON response, the full chain is working, LiteLLM received the request, forwarded it to SAP AI Core, and got a model response back.

Step 7: Configure Claude Code

Tell Claude Code to use your LiteLLM proxy instead of the Anthropic API directly:

export ANTHROPIC_AUTH_TOKEN="sk-your-master-key"
export ANTHROPIC_BASE_URL="http://0.0.0.0:4000"

Now start Claude Code and specify the model you configured in config.yaml:

claude --model claude-opus-4-7

If everything is wired up correctly, Claude Code will respond normally, but all traffic is now flowing through your SAP Generative AI Hub tenant.

Look how it answered to me:

And how the interaction is registered in the LiteLLM Proxy UI:

What You've Built and What's Next

You now have Claude Code running through SAP Generative AI Hub, with all traffic governed by your SAP BTP tenant's access controls, compliance policies…

Because LiteLLM speaks the OpenAI-compatible API, any AI coding agent listed in the LiteLLM docs, will be able to integrate with LiteLLM, then with SAP AI Core. More info.

If you attended the Cline + SAP AI Core workshop and this was the follow-up you were looking for, I hope it helps.