llamafile: Run Any LLM as a Single Portable Executable

llamafile is a single executable file that contains both a language model and the inference runtime. Built on llama.cpp and Cosmopolitan Libc, the same file runs natively on Linux, macOS, Windows, FreeBSD, and others — no container, no Python environment, no package manager required.

How a Single File Runs on Every OS

llamafile relies on two technologies. llama.cpp provides the C++ inference engine for GGUF-format models. Cosmopolitan Libc produces a polyglot binary: a single set of bytes that simultaneously satisfies the PE/COFF format Windows expects, the ELF format Linux expects, and the Mach-O header macOS expects. When you run the file, each OS loader finds its own valid header and executes the binary natively — no emulation or translation layer involved.

The model weights are appended to this binary using a ZIP-compatible container. ZIP allows arbitrary data before the central directory record, so the GGUF file lives at the end without corrupting the executable. The runtime locates the weights by seeking to the end of the file at startup. A practical side effect: you can inspect or extract the weights from any llamafile using a standard ZIP utility.

Downloading and Running a llamafile

Pre-built llamafiles are published on Hugging Face by model authors and linked from the Mozilla-Ocho/llamafile GitHub repository. Files use the .llamafile extension.

Linux and macOS

# Make executable — downloaded files lack the execute bit by default
chmod +x mistral-7b-instruct.llamafile

# Start the server and web UI (browser opens automatically)
./mistral-7b-instruct.llamafile

# One-shot CLI inference without starting the server
./mistral-7b-instruct.llamafile -p "Explain llamafile in one paragraph"

The server binds to 127.0.0.1:8080 by default. Pass --port 9000 to change the port, or --host 0.0.0.0 to listen on all network interfaces — useful for serving from a headless machine on a local network. Run./model.llamafile --help to see all available flags, including options for suppressing the automatic browser tab when running headlessly.

Windows

Windows requires executables to end in .exe. Rename the file before running:

ren mistral-7b-instruct.llamafile mistral-7b-instruct.llamafile.exe

Then double-click it in Explorer or run it from Command Prompt. A browser tab opens automatically.

The 4 GB limit. The Windows PE loader cannot handle executables larger than 4 GB. Most 7B models in Q8 or higher quantization and all 13B+ models exceed this threshold. If the file is over 4 GB, Windows will refuse to launch it. Your options: download a Q4_K_M quantized variant (typically 4–5 GB for a 7B model, often just under the limit), or download the standalone llamafile runtime binary and pass the GGUF as a separate argument. Before choosing a model and quantization, check the VRAM and file-size requirements for major LLMs to pick the right quantization for your hardware.

The Built-In Web UI

Running a llamafile without the -p flag starts an HTTP server and opens your default browser to http://localhost:8080. The interface is a fully self-contained single-page application with no external CDN dependencies — it works completely offline. It exposes:

• System prompt configuration
• Sampling parameters: temperature, top-p, top-k, repeat penalty
• Token count display
• Persistent conversation history within the session

Any device on your LAN can access the UI if you started the server with --host 0.0.0.0.

The OpenAI-Compatible API

While the server is running, llamafile exposes an OpenAI-compatible REST API at http://localhost:8080/v1. Any OpenAI SDK client, LangChain, or LlamaIndex integration can target it by overriding the base URL:

curl http://localhost:8080/v1/chat/completions 
  -H "Content-Type: application/json" 
  -d '{
    "model": "local",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

In Python, set base_url="http://localhost:8080/v1" and any non-empty string as the API key when constructing the openai.OpenAI client. The model field in requests is ignored — the loaded model is always used.

Creating a llamafile from an Existing GGUF

If you already have a GGUF model — from the llama.cpp ecosystem, Ollama’s local cache, or a Hugging Face download — you can wrap it into a self-contained llamafile. The process uses the llamafile-zipalign utility shipped with the llamafile toolset:

1. Download the llamafile release archive for your platform from the GitHub releases page.
2. Extract the standalonellamafile binary (runtime only, no model bundled).
3. Use llamafile-zipalign to append your GGUF to a copy of the runtime binary.
4. Mark the result executable and run it.

The exact flag syntax has changed across releases. Follow the official README section on creating llamafiles for the current syntax. The output is a single portable file you can distribute like any binary.

llamafile vs Ollama: When to Use Each

Both tools run GGUF models locally and expose OpenAI-compatible APIs. They solve different problems. See the Ollama complete guide for a deeper look at the other side of this comparison.

Choose llamafile when distributing to a machine you do not control, running in an air-gapped environment, or embedding a model in a project that must work without system-level dependencies. Choose Ollama when you manage a library of models, want automatic updates, or switch between models frequently in the same session.

If you are weighing whether local inference makes economic sense for your workload at all, the self-hosting vs API break-even calculator can quantify the trade-off against cloud API costs.

Hardware Requirements

llamafile adds negligible overhead above llama.cpp. The bottleneck is always model size and quantization. llamafile detects available accelerators at startup — CUDA for NVIDIA GPUs, Metal for Apple Silicon, ROCm for AMD — and uses them automatically. If no GPU is found, it falls back to CPU inference using AVX2 or AVX-512 instructions where available.

A 7B parameter model in Q4_K_M quantization requires roughly 4–5 GB of VRAM to run fully on GPU. Use the VRAM calculator to estimate requirements for any specific model and quantization before downloading. For hardware purchase decisions, the best GPUs for local LLMs guide covers current options across price points.

Frequently Asked Questions

Can I run llamafile on Apple Silicon?

Yes. llamafile produces a native Mach-O binary on Apple Silicon and uses Metal GPU acceleration automatically. Performance is solid for 7B and 13B models; larger models are constrained by available unified memory. An M2 Max or M3 Pro with 36GB of unified memory can run 30B-class models comfortably.

Does llamafile support multimodal (vision) models?

llamafile is built on llama.cpp, which supports LLaVA-style vision models. Whether a specific llamafile supports image input depends on whether the bundled model is a multimodal model. Check the model card for the file you download — the capability lives in the weights, not the runtime.

How do I stop the server?

Press Ctrl+C in the terminal where llamafile is running. If you launched it by double-clicking on Windows or macOS, close the terminal window that appeared, or end the process from Task Manager or Activity Monitor.

Can I run llamafile as a background service?

Yes. On Linux, wrap it in a systemd unit file. On macOS, create a launchd plist. llamafile accepts standard Unix signals and works with any process supervisor. Run ./model.llamafile --help to find the flag for suppressing the automatic browser tab when running in a headless environment.

Is llamafile the same as a GGUF file?

No. A GGUF file contains only model weights and requires a separate runtime — llama.cpp, Ollama, LM Studio, or similar — to run. A llamafile bundles weights and runtime into one file. Because the weights are appended in ZIP format, you can extract the raw GGUF from any llamafile with a standard ZIP utility and use it elsewhere.

How does llamafile compare to LM Studio?

LM Studio is a GUI desktop application for model discovery and inference — it requires installation and manages a library of models, making it closer to Ollama than to llamafile. llamafile is a deployment format: no GUI, no model library, just a file you execute. LM Studio suits users who want a visual interface; llamafile suits automated, headless, or portable deployments. See the LM Studio complete guide for a full breakdown.