Jan AI: Open-Source Desktop App for Running LLMs Locally

Jan is an open-source desktop application for running large language models locally, built by Homebrew Research (janhq). It packages a chat interface, a model hub, and an inference engine into a single installable app. Everything runs offline: no account is required, no data is sent to external servers, and after your models are downloaded, no internet connection is needed at all. The source code lives at github.com/janhq/jan under the AGPL-3.0 license.

Hardware Requirements

Jan can run on CPU alone, but a GPU makes a significant difference in response speed. The limiting resource is VRAM — how much GPU memory your card has determines which models you can load at usable speeds.

A 7B parameter model in 4-bit quantization (Q4_K_M) requires roughly 4–5 GB of VRAM; a 13B model needs around 8–9 GB. Use the Convly VRAM calculator to estimate memory requirements for your specific model and quantization before downloading. If you are choosing hardware, the best GPUs for local LLMs guide covers current options across price tiers.

On Apple Silicon Macs, Jan activates Metal acceleration automatically. On Windows and Linux with an NVIDIA card, it uses CUDA. AMD GPU support is available via Vulkan on Linux.

Installing Jan

Download the installer from jan.ai or the GitHub Releases page. Jan bundles its own llama.cpp inference engine — no separate dependencies required.

Windows

Run the .exe installer. Jan installs to %LOCALAPPDATA%ProgramsJan by default. No administrator rights required.

macOS

Download the .dmg. Separate builds exist for Apple Silicon and Intel — pick the one matching your Mac. Drag Jan to /Applications. On first launch, right-click the icon and choose Open to bypass the Gatekeeper warning.

Linux

Jan provides an .AppImage, a .deb, and an .rpm on the releases page.

# AppImage
chmod +x jan-linux-x86_64-*.AppImage
./jan-linux-x86_64-*.AppImage

# Debian/Ubuntu
sudo dpkg -i jan-linux-amd64-*.deb

Downloading Your First Model

When Jan opens, the Hub tab shows a curated browser of GGUF models from Hugging Face: Llama, Mistral, Qwen, Gemma, Phi, and others in multiple quantizations.

1. Click Hub in the left sidebar.
2. Search for a model — “Llama 3.2” is a good starting point for most hardware.
3. Select a quantization. Q4_K_M is the default recommendation: smaller than Q6/Q8 with acceptable quality loss.
4. Click Download and wait for the file transfer to complete.

Jan stores downloaded models at ~/jan/models/ on macOS and Linux, and at C:Users<username>janmodels on Windows. Each model lives in its own named subfolder containing the .gguf file and a model.json metadata file. You can also drop any third-party .gguf file into a new folder there — Jan detects it on next launch. GGUF files from LM Studio are directly compatible.

The Local API Server

Jan includes an OpenAI-compatible REST API server, letting any tool that supports the OpenAI client library — the Python openai SDK, LangChain, Continue.dev, and others — use Jan as a local backend.

To start it: open Settings → Local API Server, toggle the server on. It listens on port 1337 by default; this is configurable in that same panel.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1337/v1",
    api_key="jan"  # any non-empty string
)

response = client.chat.completions.create(
    model="llama3.2-3b-instruct",  # matches folder name under ~/jan/models/
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

The model name in API calls must match the folder name Jan uses internally — visible in the Hub or in the model dropdown in Chat view.

Jan vs LM Studio vs Ollama

All three tools run GGUF models locally. The differences are in philosophy and workflow.

Choose Jan if you want a fully open-source GUI tool and care about the AGPL guarantee that no one is building a closed product on top of the code you use. Jan’s chat interface is clean and the extension system lets you add capabilities without touching config files.

Choose LM Studio if polish matters more than open-source licensing. LM Studio has a more refined UI and slightly broader hardware compatibility documentation. See the LM Studio complete guide for a full walkthrough.

Choose Ollama if you are a developer who wants a fast, scriptable backend with no GUI overhead. Ollama is easier to integrate into CI pipelines, shell scripts, or Docker containers. The Ollama complete guide covers that path in detail.

If you are comparing the running costs of local inference against using a cloud API, the self-hosting vs API break-even calculator helps quantify the trade-off based on your actual usage volume and hardware costs.

Privacy and Offline Use

Jan’s core privacy guarantee is architectural: inference happens inside llama.cpp on your local machine. There is no telemetry sent during inference, no model query logged to a remote server, and no account required. Once a model file is downloaded, you can run Jan with no network connection indefinitely.

This makes Jan suitable for working with documents that cannot leave your machine — legal, medical, or commercially sensitive text — in a way that cloud API tools cannot match regardless of their privacy policies.

Jan does check for app updates and fetch the Hub model list from the internet when a connection is available. Both can be disabled: go to Settings → Advanced to turn off update checks, and the Hub simply shows no new models when offline without breaking the rest of the app.

Frequently Asked Questions

Is Jan AI completely free?

Yes. Jan is free to download and use with no subscription, no usage limits, and no paywalled features. The source code is published under AGPL-3.0 on GitHub. The only cost is your own hardware and electricity.

What models can Jan run?

Jan runs any model in GGUF format. This includes most popular open-weight models: the Llama family, Mistral, Mixtral, Qwen, Gemma, Phi, DeepSeek, and many others. The built-in Hub surfaces a curated subset; you can add any GGUF file manually by placing it in the models directory.

How much VRAM do I need?

It depends on model size and quantization. A 7B model at Q4_K_M needs roughly 4–5 GB of VRAM; a 13B model needs 8–9 GB. CPU-only inference works but is slow — expect 2–5 tokens per second on a modern CPU versus 30–80+ on a mid-range GPU. The VRAM requirements guide lists figures for major models.

Can I use Jan with VS Code or other coding tools?

Yes. Enable Jan’s local API server and point any OpenAI-compatible tool at http://localhost:1337/v1. Continue.dev, Aider, and similar coding assistants all support custom base URLs. Set the model name to match the folder name Jan uses for your downloaded model.

What is the difference between Jan and Ollama?

Jan is a desktop GUI application — you interact through a chat interface and manage models through a visual hub. Ollama is a CLI tool and background service designed for developers who want to call models from scripts or integrate them into applications. Both expose an OpenAI-compatible API. Jan suits non-technical users and GUI-first workflows; Ollama suits automation and server deployments.

Does Jan support multi-modal models (images)?

Jan has added experimental support for vision models in recent releases, depending on the version installed. Check the Hub for models labeled with vision or multimodal capability. Support and model availability vary by release, so consult the GitHub release notes for your installed version rather than assuming a specific model will work.