Local AI Models with Ollama & Data Sovereignty

This guide shows how to run a local AI model such as Gemma 4 with Ollama on your VPS or dedicated server and connect it to OpenClaw. Without cloud providers or external API keys, your data stays on your own server and no usage-based API costs apply.

Why go local? With cloud-based AI, your prompts are processed by an external provider. With Ollama + OpenClaw, your data stays on your server. Note: on servers without a GPU, local models run on the CPU and respond noticeably slower than cloud models. Plan for at least 16 GB of RAM for models in the size range of gemma4; models with tool support are required.

Step 1

Install Ollama Engine

Install Ollama using the official script. Ollama is then available as a service at 127.0.0.1:11434:

Server Terminal

curl -fsSL https://ollama.com/install.sh | sh

Download a model with tool support. OpenClaw suggests gemma4 as the default local model; smaller alternatives are e.g. gemma4:e2b or qwen3.5:4b:

Server Terminal

ollama pull gemma4

Step 2

Set the Context Window (Optional)

OpenClaw requires a context window of at least 16K tokens. Current models such as gemma4 support considerably more, and OpenClaw passes the context size to Ollama automatically. If you want to fix the context size, for example to limit memory usage, create a customized model variant:

1. Create Modelfile

nano Modelfile

FROM gemma4 PARAMETER num_ctx 32768

2. Generate the model

Server Terminal

ollama create gemma4-agent -f Modelfile

Step 3

Select the Model in OpenClaw

Start the onboarding, select Ollama as the provider and use the following settings. Alternatively, set the model directly with openclaw models set ollama/gemma4:

Server Terminal

openclaw onboard

Prompt	Input / Selection
Ollama mode	Local only
Select AI Provider	Ollama
Model Name	gemma4 (or gemma4-agent)
Base URL (without /v1)	http://127.0.0.1:11434
API Key	not required locally

Step 4

Restart the Gateway & Test

Restart the Gateway service to load the new configuration and check which models are available:

Gateway Restart

openclaw gateway restart openclaw models list --provider ollama

Open the TUI for your first chat test:

Chat Interface

openclaw tui

Setup Complete

Your local model is now connected to OpenClaw. If you encounter memory issues, use a smaller model or a smaller context window and restart the Ollama service with systemctl restart ollama.

Next Step: Security