Wednesday, 18 November 2026 | Updating Daily AI insight, written for builders

Ollama Gemma 4: Current Gemma Models and How to Run Them

Updated ยท Originally published
  • Gemma 4 is out and in the Ollama library as gemma4. Google released it on 31 March 2026 in E2B, E4B, 26B A4B and 31B sizes, and added the 12B Unified model on 3 June 2026.
  • Run ollama run gemma4 for the default E4B model, or pick a size: gemma4:e2b, gemma4:12b, gemma4:26b or gemma4:31b.
  • The 4-bit (q4_K_M) downloads are 4.6 GB (E2B), 6.6 GB (E4B), 8.0 GB (12B), 18 GB (26B) and 20 GB (31B). Context is 128K tokens on E2B and E4B and 256K on the larger sizes.
  • Gemma 4 is released under Apache 2.0. Gemma 3 and Gemma 2, still on Ollama, use Google’s Gemma Terms of Use.
  • The speed figures on this page are labelled memory-bandwidth ceilings, not measurements; real speeds come in lower.

Is Gemma 4 available on Ollama?

Yes. Gemma 4 is in the Ollama library as gemma4, in five sizes: E2B, E4B, 12B, 26B (a mixture-of-experts model with about 3.8B active parameters) and 31B. Install or update Ollama, then run ollama run gemma4 for the default E4B model, or add a size tag such as gemma4:12b. Every tag accepts text and images, and the larger sizes read up to 256K tokens of context.

Gemma 4 sizes on Ollama

Parameter counts and context windows below come from Google’s Gemma 4 model card. Download sizes are the q4_K_M builds listed on Ollama’s tags page.

Model Parameters Context Ollama command 4-bit download (q4_K_M)
Gemma 4 E2B 2.3B effective (5.1B with embeddings) 128K ollama run gemma4:e2b 4.6 GB
Gemma 4 E4B (default) 4.5B effective (8B with embeddings) 128K ollama run gemma4 6.6 GB
Gemma 4 12B Unified 11.95B 256K ollama run gemma4:12b 8.0 GB
Gemma 4 26B A4B (MoE) 25.2B total, 3.8B active 256K ollama run gemma4:26b 18 GB
Gemma 4 31B (dense) 30.7B 256K ollama run gemma4:31b 20 GB

The “E” in E2B and E4B stands for effective parameters: Google says the small models use Per-Layer Embeddings to maximise parameter efficiency on devices, and its card lists 5.1B and 8B parameters once those embeddings are counted, which is why the files are larger than the effective count suggests. The “A” in 26B A4B stands for active parameters. Google’s model card says it “runs almost as fast as a 4B-parameter model”, while it still needs the memory of a model with about 25B parameters.

The plain tags such as gemma4:31b show a size range on Ollama’s page, because more than one build sits behind them. If you want a predictable file, name the build: gemma4:31b-it-q4_K_M (20 GB), gemma4:31b-it-qat (19 GB), gemma4:31b-it-q8_0 (34 GB) or gemma4:31b-it-bf16 (63 GB). The same pattern works for the other sizes. Ollama also lists cloud tags, gemma4:cloud and gemma4:31b-cloud, which run on Ollama’s servers instead of your GPU; our Ollama Cloud explainer covers how those work.

How to run Gemma 4 with Ollama

If you do not have Ollama yet, follow the Ollama installation guide. If you already have it and the pull fails, update Ollama first, since new model families usually need a recent release. Then, on macOS, Linux or Windows:

# Default size (E4B)
ollama run gemma4

# Pick a size
ollama run gemma4:e2b
ollama run gemma4:12b
ollama run gemma4:26b
ollama run gemma4:31b

# Download without starting a chat
ollama pull gemma4:12b

Thinking mode

All Gemma 4 sizes can reason before they answer. Google’s model card says thinking is switched on by putting the <|think|> token at the start of the system prompt, and switched off by removing it. Ollama’s API also has a think parameter: true requests thinking, false requests no thinking output if the model permits it, and leaving it unset uses the model’s default.

Using Gemma 4 from code

Ollama serves a local API on http://localhost:11434:

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4",
  "prompt": "Explain quantum entanglement in simple terms.",
  "stream": false
}'

It also offers an OpenAI-compatible endpoint, so existing OpenAI client code works by changing the base URL. The client needs an API key value, but Ollama ignores it:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1/",
    api_key="ollama",  # required but ignored
)

response = client.chat.completions.create(
    model="gemma4:12b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

How much memory does each size need?

The download size is the floor: the weights have to fit in GPU memory (or Apple unified memory) for full speed. The context window needs memory on top of that, and it grows with the context length you allow. Ollama’s documentation says it picks a default context from the VRAM it detects: 4K tokens below 24 GiB, 32K from 24 to 48 GiB, and 256K at 48 GiB or more. You can override it with OLLAMA_CONTEXT_LENGTH when starting the server, and ollama ps shows in its PROCESSOR column whether part of the model has spilled onto the CPU.

Size 4-bit weights Our rule of thumb for a full GPU fit
E2B 4.6 GB 8 GB GPUs and most recent laptops
E4B 6.6 GB 8 GB GPUs with a short context; 12 GB is more comfortable
12B 8.0 GB 12 GB GPUs; 16 GB for longer contexts
26B A4B 18 GB 24 GB GPUs, or Macs and Strix Halo PCs with 32 GB or more
31B 20 GB 24 GB GPUs with a modest context, or 32 GB-plus unified memory

The rule of thumb is our estimate from the file sizes, not a measured limit. For a figure that accounts for your context length, use the VRAM calculator, and see VRAM requirements by model for other families.

How fast is Gemma 4? Labelled estimates

Neither Google’s model card nor Ollama’s library page publishes tokens-per-second figures for Gemma 4 on consumer GPUs. What can be worked out is a ceiling. When one user is generating text, each new token requires reading the model’s weights from memory, so memory bandwidth divided by the gigabytes read per token is an upper bound on generation speed.

Model (q4_K_M) NVIDIA RTX 4090 (1,008 GB/s, 24 GB) Apple M4 Pro (273 GB/s) AMD Ryzen AI Max+ 395 (256 GB/s)
Gemma 4 12B (8.0 GB) ~126 tokens/s ceiling ~34 tokens/s ceiling ~32 tokens/s ceiling
Gemma 4 31B (20 GB) ~50 tokens/s ceiling ~14 tokens/s ceiling ~13 tokens/s ceiling

Assumptions: one user, the whole download read once per generated token, and no allowance for the KV cache, compute limits or software overhead. These are arithmetic ceilings, not measurements by Convly AI, and real speeds come in lower. As a calibration point, the public llama.cpp CUDA scoreboard shows an RTX 4090 generating about 186 tokens/s on Llama 2 7B at Q4_0, a 3.83 GB file. Its ceiling by the same arithmetic is about 263 tokens/s, so that result is roughly 70% of the ceiling.

Two sizes do not fit this simple model. The 26B A4B reads only its roughly 3.8B active parameters for each token, so it generates far faster than its 18 GB file suggests; Google describes it as running almost as fast as a 4B model. E2B and E4B include Per-Layer Embeddings that are not all read for every token, so a ceiling based on their file size would understate them. For GPU buying advice, see the best GPUs for running LLMs locally, and for how Apple and AMD unified memory compare, our Strix Halo vs M4 Pro comparison.

Older Gemma generations on Ollama

Gemma 3 and Gemma 2 are still in the Ollama library. Gemma 4 replaces both for new projects, with longer context and a more permissive licence.

Family Ollama tags (download size) Context Input Licence
Gemma 3 (March 2025; 270M added August 2025) gemma3:270m (292 MB), gemma3:1b (806 MB), gemma3:4b (3.4 GB), gemma3:12b (8.2 GB), gemma3:27b (17 GB) 32K (270M, 1B); 128K (4B, 12B, 27B) Text (270M, 1B); text and image (4B and up) Gemma Terms of Use
Gemma 2 (June and July 2024) gemma2:2b (1.6 GB), gemma2:9b (5.4 GB), gemma2:27b (16 GB) 8K Text Gemma Terms of Use

Google’s Gemma 4 model card, whose table Ollama’s Gemma 4 page reproduces, puts Gemma 4 31B at 85.2% on MMLU Pro and 84.3% on GPQA Diamond, against 67.6% and 42.4% for Gemma 3 27B (without thinking). Those are vendor-reported scores. Our Gemma 3 27B model page has that model’s specs, and the Ollama models list and best local models for Ollama cover alternatives such as Llama, Mistral and Qwen.

Tuning Gemma 4 with a Modelfile

Google’s model card recommends one sampling configuration for every use case: temperature 1.0, top_p 0.95 and top_k 64. You can bake those, a context length and a system prompt into your own model with a Modelfile:

FROM gemma4:12b

PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k 64
PARAMETER num_ctx 32768

SYSTEM You are a technical documentation assistant. Give accurate, concise answers with code examples.

Then create and run it:

ollama create my-gemma -f ./Modelfile
ollama run my-gemma
  • temperature: higher values make answers more creative.
  • top_p and top_k: limit which tokens the model samples from.
  • num_ctx: the context window; larger values use more memory.
  • num_predict: the maximum number of tokens to generate.

To compare running Gemma locally against a hosted API, the self-hosting vs API calculator finds the break-even point for your usage.

Troubleshooting

“Model not found” or the pull fails: check the tag spelling against Ollama’s gemma4 page and update Ollama.

Generation is slow: run ollama ps. If the PROCESSOR column shows part of the model on the CPU, the model and its context do not fit in GPU memory. Lower the context with OLLAMA_CONTEXT_LENGTH or move down a size. Our guide to Ollama not using the GPU covers the other causes.

GPU not detected: Ollama’s documentation lists NVIDIA GPUs with compute capability 5.0 or newer and driver 550 or newer. On AMD it needs the ROCm v7 driver stack; on Windows its ROCm list covers Radeon RX 7000 and Radeon PRO W7000 cards, and Vulkan provides extra GPU support on Windows and Linux. On Macs, Ollama uses Apple’s Metal API for GPU acceleration.

API returns nothing: make sure the server is running and reachable at http://localhost:11434, and that the model has been pulled.

FAQ

Is Gemma 4 available on Ollama?

Yes. It is in the Ollama library as gemma4, in E2B, E4B, 12B, 26B and 31B sizes. Google released Gemma 4 on 31 March 2026 and added the 12B Unified model on 3 June 2026. Run ollama run gemma4 for the default E4B model, or add a size tag.

Which Gemma 4 size should I start with?

Our suggestion: E4B, the default, on laptops and 8 GB GPUs; 12B on 12 to 16 GB GPUs; 26B A4B on 24 GB GPUs if you want speed; 31B on 24 GB GPUs or 32 GB-plus Macs if you want the strongest dense model. The table above lists each download size.

Can I use Gemma 4 commercially?

Google’s model card lists Gemma 4 under the Apache 2.0 licence, which permits commercial use, modification and redistribution under its terms. Gemma 3 and Gemma 2 use Google’s separate Gemma Terms of Use, so read those before shipping a product on an older model.

Does Gemma 4 on Ollama understand images and audio?

Images, yes: Ollama’s tags page lists text and image input for every Gemma 4 tag. Audio depends on the size. Google’s model card lists audio input for E2B, E4B and the 12B Unified model, but not for 26B A4B or 31B. Ollama’s library page shows an audio badge for the family, so if you need audio, pick one of the three smaller sizes.

How do I turn Gemma 4’s thinking on or off?

Google says thinking is enabled by putting the think token at the start of the system prompt and disabled by removing it. Through Ollama’s API you can also set the think parameter to true or false, or leave it unset to use the model’s default.

Can Gemma 4 run without a GPU?

Yes. Ollama can run a model on the CPU, but generation is then limited by system memory bandwidth, which is far lower than a modern GPU’s. E2B and E4B are the realistic choices on CPU only.

Written by Mustafa Ihsan

Mustafa Ihsan is the founder and editor of Convly.ai. He built and maintains the site's live AI models database, its price-performance index, and its free calculators for VRAM requirements, API costs and self-hosting economics. He writes about model pricing, benchmark results and the hardware needed to run AI models locally, and consistently prefers measured numbers to vendor claims.

Scroll to Top