How to Set Up Text Generation WebUI

How to Set Up Text Generation WebUI

Text Generation WebUI (oobabooga) is a full-featured interface for running LLMs with advanced options for sampling, LoRA loading, and model comparison on your Breeze.

Requirements

• A Breeze with at least 8 GB RAM
• Python 3.11 or newer
• Git

Installation

git clone https://github.com/oobabooga/text-generation-webui.git
cd text-generation-webui
python3 -m venv venv
source venv/bin/activate
pip install -r requirements_cpu_only.txt

Download a Model

Use the built-in downloader:

python3 download-model.py TheBloke/Mistral-7B-Instruct-v0.2-GGUF

Or manually place GGUF files in the models/ directory.

Launch the Server

python3 server.py --listen --cpu

Access the UI at http://your-breeze-ip:7860.

Key Features

• Chat, instruct, and notebook modes
• Adjustable generation parameters (temperature, top-p, repetition penalty)
• LoRA adapter loading for fine-tuned models
• API endpoint compatible with OpenAI format
• Extension system for adding functionality

Enable the API

Add the API extension flag:

python3 server.py --listen --cpu --api

The API will be available at http://your-breeze-ip:5000.