Using Ollama With Claude Code: Local Model Setup Guide

Claude Code is Anthropic’s official terminal coding agent, and it talks to Anthropic’s cloud API by default. Ollama is a local model runner that exposes an OpenAI-compatible HTTP API. The two do not speak the same protocol out of the box, so pairing them requires a small proxy layer that translates Anthropic’s Messages API into Ollama’s chat API. This guide covers how that works, which models are worth running, and where the setup breaks down.

Why Ollama and Claude Code Don’t Connect Directly

Claude Code issues requests in Anthropic’s Messages API format (/v1/messages), with Anthropic-specific fields for tool use, cache control, and system prompts. Ollama exposes /api/chat and an OpenAI-compatible /v1/chat/completions endpoint. Neither speaks the Anthropic dialect. To bridge them you need a proxy that:

• Accepts Anthropic-format requests on an HTTPS endpoint.
• Rewrites them into OpenAI-compatible chat completions.
• Forwards to Ollama, then translates streaming responses and tool calls back into the Anthropic event stream Claude Code expects.

Two projects handle this reliably as of 2026: claude-code-router (a purpose-built router for Claude Code that supports Ollama, OpenRouter, and other backends) and LiteLLM (a general-purpose proxy with an anthropic pass-through mode). Either works; claude-code-router has fewer moving parts if Ollama is your only backend.

Prerequisites

• Ollama installed and running. See how to install Ollama if you don’t have it yet.
• Claude Code installed (npm install -g @anthropic-ai/claude-code).
• At least one coding-capable model pulled locally.
• Node.js 18+ for claude-code-router, or Python 3.10+ for LiteLLM.

Verify Ollama is reachable:

curl http://localhost:11434/api/tags

Choosing a Local Model

Claude Code leans heavily on tool-use, structured edits, and long-context reasoning. Smaller general-chat models handle these badly. Stick to coder-tuned models or large instruct models.

Quantization level, context length, and KV cache size all move VRAM requirements. Use the VRAM calculator to size a specific configuration, and see best local LLMs for Ollama for a wider comparison.

Pull one before configuring the proxy:

ollama pull qwen3-coder:30b
ollama pull deepseek-coder-v2:16b

Option 1: claude-code-router

claude-code-router is the shortest path. Install it globally:

npm install -g @musistudio/claude-code-router

Create ~/.claude-code-router/config.json:

{
  "Providers": [
    {
      "name": "ollama",
      "api_base_url": "http://localhost:11434/v1/chat/completions",
      "api_key": "ollama",
      "models": ["qwen3-coder:30b", "deepseek-coder-v2:16b"]
    }
  ],
  "Router": {
    "default": "ollama,qwen3-coder:30b",
    "background": "ollama,deepseek-coder-v2:16b"
  }
}

Start Claude Code through the router:

ccr code

The router launches a local Anthropic-compatible endpoint, sets the environment variables for Claude Code automatically, and proxies traffic to Ollama. Exact config field names have shifted between minor versions — check the project’s README if a key is rejected.

Option 2: LiteLLM Proxy

If you already run LiteLLM for other services, reuse it. Create config.yaml:

model_list:
  - model_name: claude-sonnet-4
    litellm_params:
      model: ollama_chat/qwen3-coder:30b
      api_base: http://localhost:11434
  - model_name: claude-haiku-4
    litellm_params:
      model: ollama_chat/deepseek-coder-v2:16b
      api_base: http://localhost:11434

Run it:

litellm --config config.yaml --port 4000

Then point Claude Code at the proxy (see next section). LiteLLM handles the Anthropic-to-Ollama translation on the /anthropic route.

Pointing Claude Code at the Proxy

Claude Code reads two environment variables to redirect its API traffic. Set them in your shell before running claude.

macOS and Linux

export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_API_KEY="sk-anything"
claude

Add these to ~/.zshrc or ~/.bashrc to persist. The key value is unused by the local proxy but Claude Code refuses to start without one set.

Windows (PowerShell)

$env:ANTHROPIC_BASE_URL="http://localhost:4000"
$env:ANTHROPIC_API_KEY="sk-anything"
claude

To persist across sessions, use [Environment]::SetEnvironmentVariable("ANTHROPIC_BASE_URL", "http://localhost:4000", "User"). If Claude Code is installed via WSL, configure the variables inside the WSL shell instead — Ollama running on Windows is reachable from WSL at http://host.docker.internal:11434 or the Windows host IP.

Windows (native, without WSL)

Claude Code officially targets macOS, Linux, and WSL. Native Windows support has been rough historically; run it under WSL2 unless you have confirmed the current release runs cleanly on your setup.

Configuring Context and Timeouts

Claude Code assumes 200K-token context and fast time-to-first-token. Local models will not match either. Two tuning points matter:

• Ollama context length. Set it explicitly per model via a Modelfile (PARAMETER num_ctx 32768) or the OLLAMA_CONTEXT_LENGTH environment variable on the Ollama server. Default context is small and will silently truncate long conversations.
• KV cache VRAM. A 32K context on a 30B model consumes several GB of KV cache alone. Check total memory usage with ollama ps.

For a full breakdown of memory needs at various context sizes, see VRAM requirements by model.

Known Limitations

• Tool use is fragile. Claude Code’s file-edit, bash, and search tools depend on strict JSON tool-call output. Local models fail this more often, producing malformed calls or hallucinating file contents. qwen3-coder and deepseek-coder-v2 are among the most reliable, but neither matches Claude Sonnet.
• Prompt caching is a no-op. Anthropic’s cache_control fields are ignored by the proxy. Long system prompts get re-sent every turn.
• Speed. Even on a 24 GB GPU, a 30B model at 32K context produces 15–40 tokens/sec. Claude Code’s agentic loop makes many calls per task, so wall-clock time can be 5–10× slower than the cloud API.
• Sub-agents and MCP. Advanced Claude Code features (background agents, MCP servers) generally still work because they route through the same proxy, but any feature relying on Anthropic-specific server behavior can break silently.

When to Use This Setup vs the API

Local Claude Code makes sense when: code cannot leave your network, you’re on a metered API budget and doing bulk refactors, or you’re experimenting with self-hosting. It makes less sense for daily agentic coding where Sonnet-class reasoning is the point.

To decide numerically, run the numbers on both sides. The API cost calculator estimates monthly Anthropic spend, and the self-hosting vs API break-even calculator compares that against GPU amortization. For most solo developers the API wins; for teams hitting the API hard, a shared local box can pay back inside a year. If you’re spec’ing that box, the best GPUs for local LLMs guide covers the current tier.

Alternatives to Ollama for This Workflow

If Ollama’s performance is limiting, other runners with OpenAI-compatible APIs work identically behind the same proxy: LM Studio, vLLM, and llama.cpp’s server all fit. See the LM Studio guide for a GUI-first option, or the Ollama complete guide for a deeper look at Ollama itself.

Frequently Asked Questions

Can Claude Code use Ollama without a proxy?

No. Claude Code speaks Anthropic’s Messages API and Ollama does not implement that dialect. You need a translation layer such as claude-code-router or LiteLLM. Setting ANTHROPIC_BASE_URL directly to http://localhost:11434 will fail on the first request.

Which local model comes closest to Claude Sonnet for coding?

At the sizes most people can actually run, qwen3-coder (30B MoE) and deepseek-coder-v2 (16B lite) are the current top choices. Neither matches Sonnet on multi-file agentic tasks, but both are usable for single-file edits, completions, and code review. Compare intelligence scores on the LLM leaderboard.

Does prompt caching work with Ollama behind Claude Code?

No. Anthropic’s prompt caching is a server-side feature of their API. Proxies strip or ignore the cache_control fields, so every request re-processes the full system prompt and conversation history. This is one reason local setups feel slower per turn than the cloud API even at similar raw token throughput.

How much VRAM do I need for a decent experience?

A single 24 GB GPU (RTX 3090/4090/5090 class) runs a 30B coder model at Q4 with roughly 32K context. For 70B-class models or longer contexts, plan on 48 GB (RTX 6000 Ada, dual 3090s) or more. Use the VRAM calculator for exact numbers per model and quantization.

Can I mix local and cloud models in the same Claude Code session?

Yes, via a router. claude-code-router lets you assign different models to different roles — for example, cloud Sonnet for the main agent and a local model for background tasks or completions. This can cut API costs substantially while keeping quality high on the critical path.

Is there an official Anthropic-supported way to run Claude Code locally?

No. Anthropic ships Claude Code as a client for their API and doesn’t distribute Claude weights. All local setups are community proxies pointing at third-party models. If Anthropic changes the Messages API, proxies may need updates before Claude Code works again against them.