Ollama Windows: Installation Guide and Setup Instructions

Ollama on Windows is a native application that runs large language models locally without Docker or WSL2. The Windows installer sets up Ollama as a system service that starts automatically and provides GPU acceleration through DirectML and CUDA. Installation takes under two minutes, and you can run models like Llama 3.3 70B or Mistral 7B immediately from the command line.

System Requirements for Ollama on Windows

Ollama requires Windows 10 (build 1903 or later) or Windows 11. The minimum hardware specifications are:

Component	Minimum	Recommended
Operating System	Windows 10 (1903+)	Windows 11
RAM	8 GB	16 GB or more
Disk Space	10 GB free	50+ GB for multiple models
GPU (optional)	NVIDIA GTX 1050 or AMD equivalent	NVIDIA RTX 3060 12GB or better

GPU acceleration works with NVIDIA GPUs (CUDA 11.8 or later) and recent AMD GPUs (via ROCm on Windows). The installer includes the necessary GPU libraries; you do not need to install CUDA separately. Without a GPU, Ollama falls back to CPU inference, which is 5-10× slower but functional for smaller models.

Model size determines actual RAM or VRAM requirements. For example, Mistral 7B needs approximately 4.5 GB VRAM at 4-bit quantization, while Llama 3.3 70B requires around 40 GB. Check the VRAM calculator to estimate requirements for any model before downloading.

Installing Ollama on Windows

The official Windows installer is a single executable that handles all dependencies:

1. Download OllamaSetup.exe from ollama.com/download
2. Run the installer—administrator privileges are required
3. Accept the license and click Install (default location is C:Program FilesOllama)
4. The installer completes in 30-60 seconds and starts the Ollama service automatically

After installation, open Command Prompt or PowerShell and verify with:

ollama --version

You should see output like ollama version 0.x.x. The Ollama service runs in the background—you will see an Ollama icon in the system tray when it is active.

GPU Support and Detection

Ollama automatically detects and uses available GPUs on Windows. To confirm GPU acceleration is working:

ollama run llama3.3:70b "test"

While the model loads, watch Task Manager (Performance tab). If GPU utilization appears under GPU 0 or GPU 1, acceleration is active. The first run downloads the model (5-40 GB depending on size) to %USERPROFILE%.ollamamodels, which can take several minutes on slower connections.

NVIDIA users with RTX 3060 or newer GPUs get the best performance due to 12+ GB VRAM and Ampere/Ada architecture optimizations. AMD GPU support on Windows is improving but still lags NVIDIA—expect 20-30% slower inference on equivalent AMD hardware. Intel Arc GPUs are not officially supported as of September 2026.

If Ollama does not detect your GPU, check that your GPU drivers are current (NVIDIA: 537.13 or later, AMD: Adrenalin 23.11 or later). Ollama’s error messages usually indicate whether the GPU was skipped due to insufficient VRAM or missing drivers.

Running Your First Model

The ollama run command downloads, loads, and starts an interactive session with a model:

ollama run phi4

This downloads Microsoft’s Phi-4 (approximately 9 GB VRAM at 4-bit quantization) and opens a chat prompt. Type your question, press Enter, and the model responds locally. Exit with /bye or Ctrl+C.

Common models to try on Windows hardware:

Model	VRAM (4-bit)	Best For
Mistral 7B	~4.5 GB	General tasks, fast responses
Llama 3.1 8B	~5 GB	Balanced quality and speed
Phi-4	~9 GB	Reasoning, compact size
Llama 3.3 70B	~40 GB	High-end GPUs, best quality

Browse available models at Ollama models list or search the library with ollama list after pulling at least one model.

Configuration and Environment Variables

Ollama reads configuration from Windows environment variables. Set these in System Properties → Environment Variables or in PowerShell with $env:VARIABLE_NAME.

Key variables:

• OLLAMA_MODELS: Directory for model storage (default: %USERPROFILE%.ollamamodels). Change this if your C: drive has limited space.
• OLLAMA_HOST: Server address (default: 127.0.0.1:11434). Set to 0.0.0.0:11434 to accept network connections.
• OLLAMA_NUM_PARALLEL: Number of concurrent requests (default: 1). Increase if serving multiple clients.
• OLLAMA_MAX_LOADED_MODELS: Models kept in VRAM (default: 1). Set to 2+ if you have 24+ GB VRAM and want instant switching.

After changing environment variables, restart the Ollama service from Services (Win+R, type services.msc, find Ollama, right-click Restart) or reboot.

To move models to a different drive, set OLLAMA_MODELS, then restart the service. Existing models must be re-pulled—Ollama does not automatically migrate them.

Performance Tuning

Inference speed depends on VRAM, model size, and quantization level. A 7B parameter model at 4-bit quantization generates 30-60 tokens per second on an RTX 3060, while 70B models drop to 3-8 tokens per second unless you have 48+ GB VRAM across multiple GPUs.

Multi-GPU setups work automatically—Ollama spreads model layers across detected GPUs. For example, two RTX 3090s (24 GB each) can run Llama 3.3 70B at acceptable speed, while a single 3090 would offload layers to system RAM and crawl.

If performance is slower than expected:

• Check Task Manager during inference—CPU usage above 30% means the model is partially in system RAM due to insufficient VRAM
• Try a smaller quantization (e.g., ollama pull mistral:7b-instruct-q4_K_M instead of the default Q8)
• Close other GPU applications (browsers with hardware acceleration, games, video editors)
• Update GPU drivers—newer NVIDIA drivers improve inference speed by 5-10% over 6-month-old versions

For hardware upgrades, consult the best GPUs for local LLMs guide to compare price-to-performance across NVIDIA and AMD options.

Integrating with Applications

Ollama exposes a REST API on localhost:11434 compatible with OpenAI’s API structure. Applications that support OpenAI-compatible endpoints can point to Ollama with minimal changes.

Example curl command to generate text:

curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Explain Docker in one sentence"
}'

Popular integrations include Open WebUI (web interface), Continue.dev (VS Code extension), and LangChain (Python). The Ollama service must be running for API calls to work—it starts automatically on boot by default.

Frequently Asked Questions

Does Ollama require WSL2 or Docker on Windows?

No. The Windows installer is a native application that runs Ollama as a Windows service without virtualization or containers. Earlier versions required Docker, but the native Windows release eliminated that dependency in late 2024. If you installed Ollama before November 2024, uninstall the Docker version and download the current native installer.

Can I use Ollama offline after downloading models?

Yes. Once a model is pulled with ollama pull <model>, it is stored locally and runs without internet access. The only network requirement is the initial download—models range from 2 GB (small 7B models) to 80+ GB (large 70B+ models). After downloading, disconnect from the network and Ollama continues to function normally.

How much does it cost to run Ollama models compared to APIs?

Ollama is free—you pay only for hardware and electricity. Compare this to API costs: Claude Sonnet 5 charges $2.00 per million input tokens, so 10 million tokens (roughly 7.5 million words) costs $20. A self-hosted setup breaks even quickly if you process significant volume. Use the self-hosting vs API calculator to estimate your break-even point based on expected usage.

Which models work best on consumer GPUs?

For GPUs with 8-12 GB VRAM (RTX 3060, 4060 Ti), Mistral 7B (~4.5 GB at 4-bit), Llama 3.1 8B (~5 GB), and Phi-4 (~9 GB) provide strong quality-to-performance ratios. If you have 16-24 GB (RTX 3090, 4090), Mistral NeMo 12B (~7.5 GB) and even Llama 3.3 70B (~40 GB with aggressive quantization or offloading) become viable. See VRAM requirements by model for a complete list.

Can Ollama use both NVIDIA and AMD GPUs simultaneously?

No. Ollama uses either CUDA (NVIDIA) or ROCm (AMD) depending on what it detects first, but it cannot mix GPU backends in one session. If you have both NVIDIA and AMD GPUs, Ollama prioritizes NVIDIA. Multi-GPU support works only when all GPUs use the same driver stack—two NVIDIA cards or two AMD cards, not one of each.

How do I uninstall Ollama from Windows?

Open Settings → Apps → Installed Apps, find Ollama, and click Uninstall. This removes the application and service but leaves models in %USERPROFILE%.ollama. Delete that folder manually to reclaim disk space. If the service does not stop during uninstallation, open Task Manager, find OllamaService.exe under Background Processes, and end it before retrying.

Next Steps

After installing Ollama on Windows, explore the full model library at Ollama complete guide to understand how model selection, quantization, and context windows affect quality. For production workloads, calculate ongoing API costs versus self-hosting with the API cost calculator to determine whether local inference or cloud APIs better fit your budget and scale requirements.