Text Generation WebUI (Oobabooga): Installation, Loaders, and Usage Guide

text-generation-webui is an open-source, self-hosted web interface for running large language models locally. Maintained on GitHub as oobabooga/text-generation-webui, it runs in your browser at http://localhost:7860, supports a wide range of model formats through swappable inference backends, and exposes an OpenAI-compatible REST API. It is the most feature-complete local LLM frontend available, at the cost of a steeper setup curve than desktop apps like LM Studio.

What text-generation-webui Is (and Why People Call It Oobabooga)

The project’s GitHub username is oobabooga, which has become the community shorthand for the tool itself. Both names refer to the same project at github.com/oobabooga/text-generation-webui.

The interface is built on Gradio and runs entirely on your local machine — no data leaves your system. Beyond basic chat, it supports:

• Three input modes: Chat (instruct), Chat (roleplay with character cards), and Notebook (raw completion)
• LoRA loading for fine-tuned adapter weights on top of a base model
• An extension system with community plugins for summarization, speech synthesis, image captioning, and more
• An OpenAI-compatible API endpoint for connecting third-party clients and automation scripts

Before choosing a model, use the VRAM calculator to confirm your GPU can fit it — requirements vary widely by model size and quantization level.

Installing text-generation-webui

The recommended path on every platform is the one-click installer. It creates an isolated Conda environment and installs all Python dependencies automatically. Do not install into your system Python unless you have a specific reason to do so.

Windows

1. Clone the repository or download a release zip from the GitHub page:
   git clone https://github.com/oobabooga/text-generation-webui
2. Double-click start_windows.bat in the cloned folder.
3. The script detects your GPU type (NVIDIA, AMD, or CPU-only) and installs matching dependencies — select the appropriate option when prompted.
4. After setup completes, the server launches automatically. Open http://localhost:7860 in your browser.

On future runs, double-click start_windows.bat again. The Conda environment is already built; startup takes only a few seconds.

Linux

1. Clone the repository and enter the directory:

   git clone https://github.com/oobabooga/text-generation-webui
   cd text-generation-webui

2. Make the script executable and run it:

   chmod +x start_linux.sh
   ./start_linux.sh

3. Select your GPU type when prompted: NVIDIA, AMD, CPU-only, or Apple Silicon (not applicable on Linux, but the prompt appears).
4. Access the UI at http://localhost:7860 once the server is running.

macOS

1. Clone the repository and run the start script:

   git clone https://github.com/oobabooga/text-generation-webui
   cd text-generation-webui
   ./start_macos.sh

2. Select option D (Apple Silicon / Metal) or C (CPU-only) when prompted.
3. The llama.cpp backend uses Metal for GPU acceleration on M-series chips — no CUDA required.

macOS users are limited to the llama.cpp and Transformers loaders. ExLlamaV2 requires CUDA and does not run on Apple Silicon.

Model Loaders: Which One to Use

text-generation-webui decouples the inference engine from the UI. You select a loader per model from the Model tab. Each loader accepts specific file formats and has different hardware requirements.

Practical default: If you downloaded a .gguf file (the most common format on HuggingFace model pages), use llama.cpp. If you have an NVIDIA GPU and want maximum generation speed, look for an EXL2 variant of the same model and use ExLlamaV2.

For a breakdown of which models realistically fit which GPUs, see VRAM requirements for major LLMs.

Loading a GGUF Model Step by Step

1. Copy the file into text-generation-webui/models/. Single-file GGUFs (e.g., mistral-7b-instruct.Q4_K_M.gguf) go directly in that folder. Multi-part split files should go in a named subfolder.
2. Open the Model tab at http://localhost:7860.
3. Select your file from the model dropdown (click the refresh icon if it does not appear).
4. Set Loader to llama.cpp.
5. Set n-gpu-layers to control GPU offloading. Enter a large number (e.g., 999) to push as many layers as possible into VRAM; enter 0 for CPU-only inference.
6. Click Load. A confirmation message appears in the status box once the model is ready.

Switch to the Chat tab to start a conversation, or the Default tab for raw prompt completion. The instruct template is set automatically for known model families; for unknown models, select it manually from the Instruction template dropdown in the Parameters tab.

The OpenAI-Compatible API Extension

The built-in openai extension exposes REST endpoints that mirror the OpenAI Chat Completions and Completions API format. Any client that accepts a custom base URL — LangChain, Open WebUI, Continue.dev, or a plain curl script — can connect to your local model without code changes.

Enable it by passing a flag at startup:

# Linux / macOS
./start_linux.sh --extensions openai

# or launch server.py directly inside the Conda environment
python server.py --extensions openai

Alternatively, enable it from the Session tab in the UI, then click Apply flags / Restart.

By default the API listens on port 5000, separate from the Gradio UI on port 7860. Point your client to:

base_url = "http://localhost:5000/v1"
api_key  = "anything"  # required by most clients but not validated locally

Supported endpoints include /v1/chat/completions, /v1/completions, and /v1/models. The port is configurable via the --api-port startup flag.

text-generation-webui vs LM Studio vs Jan

All three tools run models locally without cloud dependencies. They target different users and use cases.

Choose text-generation-webui when you need multiple loader backends, EXL2 performance on NVIDIA, LoRA loading, the extension ecosystem, or scripted API access for automation and development workflows.

Choose LM Studio or Jan when you want a polished installer, built-in model search with one-click downloads, and minimal configuration. See the LM Studio complete guide for a detailed walkthrough of that option.

If you are deciding whether local inference is worth the hardware cost at all, the self-hosting vs API break-even calculator can quantify the trade-off against paying for API access at your usage volume.

Frequently Asked Questions

Why does the one-click installer take so long the first time?

It downloads Miniconda and builds an isolated Python environment with PyTorch and all model-loading libraries from scratch. On a fast connection this typically takes 5–15 minutes. Subsequent launches skip this step and start in a few seconds.

Can I run text-generation-webui without a GPU?

Yes. Select the CPU-only option during installation, then set n-gpu-layers to 0 when loading a GGUF. A 7B model may produce 2–5 tokens per second on a modern desktop CPU — usable for testing but slow for conversation. Smaller quantizations (Q4 and below) improve throughput. Check best GPUs for local LLMs if you are considering a hardware upgrade.

How do I update text-generation-webui?

Run git pull inside the repository directory to fetch the latest code, then re-run the start script. The script detects environment changes and updates dependencies automatically. You can also run pip install -r requirements.txt manually inside the active Conda environment if you prefer a targeted update.

What is the difference between GGUF and EXL2?

Both are quantized formats that reduce model file size and VRAM requirements. GGUF (via llama.cpp) runs on NVIDIA, AMD, and Apple Silicon — it is the more portable choice and has the widest model availability. EXL2 (via ExLlamaV2) is NVIDIA-only but typically generates tokens faster at equivalent quality. If you have an NVIDIA GPU and speed is the priority, EXL2 is worth sourcing.

Where are conversation logs saved?

Logs are stored under text-generation-webui/logs/. Each conversation is saved as a JSON file. You can also export directly from the Chat tab UI using the download button below the conversation window.

Can multiple users connect to one instance?

The --listen startup flag makes the server accessible on your local network instead of localhost only. However, text-generation-webui is not designed for multi-user production deployments — there is no built-in authentication and the Gradio UI is single-session. The OpenAI API extension handles concurrent API requests better than the web UI for multi-client scenarios, but you should add a reverse proxy with authentication in front of it if exposing beyond localhost.