SAP Business OneAI SolutionsCloud vs. Local

Local AI or Cloud for SAP Business One

Local model, hosting in Germany, your own API key or SAP BTP: data flows, hardware, costs and limits of the four operating models compared.

Paul Müller
Paul Müller
· 7 min read
Local AI or Cloud for SAP Business One

A supplier sends an invoice as a PDF. The AI is supposed to recognize the supplier, amount, tax, cost center and purchase order reference and prepare the document for SAP Business One. This is exactly where the question of local AI or cloud becomes concrete: may the document leave your network? Does it even have to? And which solution really saves work instead of creating a new technical construction site?

The short answer: there is no overall winner, but four operating models with clear strengths. Local gives you the greatest data sovereignty and the most operating effort. A model hosted in Germany is the pragmatic middle ground for many SMEs. Your own API key gives you the strongest models, and SAP’s own AI always runs via SAP BTP. This article compares the four options, states the hardware requirements of local models and shows how to decide.

Four operating models at a glance

SAP AI via SAP BTPHosted in Germany (RC.AI)Your own API keyLocal in your own network
Where the data is processedSAP AI Core, region selectable during setupexclusively in Germanyat the model provider, depending on the contractin your network
Data processing agreementwith SAPwith RConsult, includedwith the model providernot required for the model
Your own operating effortset up and monitor the BTP accountlowmanage key, costs and settingshardware, updates, monitoring, backup
Running costsusage in tokens15 EUR per user and monthusage with the providerelectricity, maintenance, hardware replacement
Invoice PDFs in RC.AInot relevantcurrently noyes, with Anthropic Claudecurrently no
Fits ifyou use the Web Client and only need questions and queriesyou want to start quickly and expect processing in Germanyyou already have contracts with a providerdata must not leave the building

In all variants with RC.AI, the RC.AI backend runs on or next to your SAP server. The only difference is where the language model does its computing. Permissions always come from SAP: every user works under their own SAP login and only sees what they are allowed to see in SAP.

Which data may the AI see?

The right architecture does not start with choosing a model, but with a sober classification of your data. An AI that suggests texts for internal knowledge articles does not need access to financial accounting. An assistant for incoming invoices, on the other hand, needs document data, supplier master data, account assignment rules and purchase orders.

For general texts, translations or drafts without confidential content, a cloud service is often sufficient. For processes involving financial data, payroll, prices or customer agreements, you set up stricter rules. That does not mean everything has to run locally. It means access rights and data paths have to match the risk. Anyone who may only read purchase orders should not be able to trigger payments or change master data through the AI.

Cloud: start quickly, but do not transfer blindly

Cloud AI is attractive because getting started is quick. You need no graphics card, no model installation and no capacity planning. The providers’ large models are usually stronger at open, complex tasks than anything that can be run economically on premises.

The catch is often not the model but the integration. If employees copy documents into a chat window, store results in Excel and transfer values to SAP by hand, all you get is a new isolated solution. Also check storage location, retention periods and logging and whether the provider uses your data for training.

Calculate the running costs beforehand. With few requests, usage-based models are convenient, but with many documents and users they can become hard to predict. A fixed price per user keeps the calculation simple: with RC.AI, 10 users with the model hosted in Germany cost 150 EUR per month, which is 1,800 EUR per year.

Local: what hardware you need

A local model mainly needs graphics memory. As a rule of thumb: the size of the model file plus 10 to 30 percent for context and runtime, more for long documents. The file sizes below come from the Ollama model library and apply to the usual 4-bit quantization.

Model classExampleFile sizeRealistic graphics memory
8 billion parametersLlama 3.1 8B, Qwen3 8B4.9 to 5.2 GB8 to 12 GB
around 20 billiongpt-oss-20b14 GB16 GB, sufficient according to the manufacturer
24 to 32 billionMistral Small 3.2 24B, Qwen3 32B15 to 20 GB24 GB, more with long context
around 70 billionLlama 3.3 70B43 GB48 GB, for example two cards with 24 GB each
around 120 billiongpt-oss-120b65 GBone card with 80 GB according to the manufacturer

Models such as gpt-oss and Qwen3 30B-A3B only compute with part of their parameters per request. They still need memory for all weights, but respond faster than models of the same size that always use all parameters. As a runtime, Ollama is the easiest way to get started, and RC.AI connects local models through it. For many concurrent users, vLLM is the more powerful choice. Check the model’s license before use: gpt-oss is licensed under Apache 2.0, Llama under Meta’s own license.

For clearly defined tasks such as questions about inventory, open items or sales orders, a model in the 20 to 32 billion class is often enough. Bigger is not automatically better. What matters in daily work is whether the answers are correct, traceable and fast enough. So test with real questions from your business before you buy hardware.

Local means operations

With a local model, you take on an operational task. Hardware, updates, access protection, backup and monitoring all have to work. You should be able to trace which model version was active when and who approves changes. The AI system only needs access to the services it requires for its purpose, not to the entire network.

Local does not automatically mean secure. A locally operated model with overly broad SAP authorizations remains a risk. And there is a limit you should know about: RC.AI’s PDF analysis currently requires Anthropic Claude. If you want to process incoming invoices with AI, you therefore cannot run fully locally today.

Graduated instead of black or white

You do not have to choose the same path for all tasks. Sensitive queries about payroll or terms can run locally, drafts and general texts via a cloud service. What matters is that the boundaries are technically enforced and not just written down in a work instruction. Data minimization is part of this from the start: the AI only receives the data it needs for the respective task.

Your own API key is also a deliberate choice. You keep the contract, billing and settings in your own hands. In return, you have to store the key securely, separate access and define how it is rotated or revoked. Your own key does not replace a clear data architecture.

How to make the decision

CriterionSpeaks for localSpeaks for hosting in GermanySpeaks for your own key
Data must not leave the buildingyesnono
In-house IT available for operationsrequirednot requiredpartly
Incoming invoices from PDFcurrently nocurrently noyes
Predictable costs per userupfront investmentyesno, usage-based
Strength with open, complex questionslimitedgoodhighest

Start with a process that visibly costs time today and define which data it needs. The operating model usually follows from that. Which use cases pay off is shown in the article AI in SAP Business One: Features and Use Cases. How to clarify the contract, the AI Act and approvals before the start is covered in AI in ERP: Data Protection, AI Act and Getting Started. You can find the RC.AI options on the page for the RC.AI assistant.

Frequently asked questions

What hardware does a local language model need?
That depends on the model size. A model with 8 billion parameters is around 5 GB in 4-bit quantization and runs on a graphics card with 8 to 12 GB of memory. Models with 20 to 32 billion parameters need 16 to 24 GB, a 70 billion model around 48 GB, for example two cards with 24 GB each.
Can a local model process incoming invoices from PDFs?
Not with RC.AI at the moment. The PDF analysis currently requires Anthropic Claude. Questions to SAP, analyses and creating documents via chat also work with a local model through Ollama.
Is a local AI automatically GDPR-compliant?
No. A local model avoids external data flows, but you still have to regulate authorizations, logs, retention periods and purpose limitation. A poorly maintained server in your own building is not a data protection concept.
What does the model hosted in Germany cost with RC.AI?
15 EUR per user and month, net. Language processing takes place exclusively in Germany, with no transfer to third countries and no use of your data for model training. The data processing agreement is included.
Can I run SAP's AI features locally?
No. Ask AI and the AI queries from Feature Package 2608 run through SAP AI Core on SAP BTP. You choose the region, for example EU, during setup, but processing always takes place in the cloud.

Updated on November 6, 2026

Paul Müller
Paul Müller
Virtual Sales Representative
LinkedIn