Text Generation WebUI (oobabooga) on a Cloud GPU: Full Setup Guide

Running oobabooga on a cloud GPU comes down to matching your model size and quantization level to the right card: an RTX 4090 covers most 7B-13B models, an L40S handles 30B models or lightly-quantized 70B models, and an A100 80GB runs 70B models with minimal quality loss.

Most oobabooga setup guides are written for a desktop with a monitor attached. They assume you'll click through an installer, see a window open, and troubleshoot by looking at it. None of that applies on a rented GPU. There's no display, no double-clicking, and the new one-click installer expects a desktop environment that a cloud instance doesn't have.

That mismatch is why cloud deployment trips people up even when they've run oobabooga locally before: the commands are different, the failure modes are different, and getting it wrong usually means paying for GPU time while staring at a blank terminal. This guide covers the setup that actually works headless, how to pick a GPU that fits your model without overpaying, and the mistakes that waste the most time and money.

One naming note before we start: the project maintainer renamed the repository from text-generation-webui to TextGen, now hosted at github.com/oobabooga/textgen. Most people, documentation, and search behavior still say oobabooga, so this guide does too, but the commands below reflect the current repository.

Oobabooga VRAM Requirements: Model Size vs Quantization

VRAM is the constraint that decides everything else, and unlike a fixed-architecture image model, it's partly a choice you make. Quantization trades precision for memory: a 70B model at full precision needs far more VRAM than the same model quantized to 4-bit, at some cost to output quality. Context length adds on top of that, since GGUF's context cache is preallocated, so a longer context window claims VRAM before you've even loaded a prompt.

GGUF is the format to default to on a cloud GPU, since it runs through llama.cpp, degrades gracefully if you undersize the card, and falls back to CPU offload if a layer doesn't fit. GPTQ and EXL3 are GPU-only formats that run faster once loaded, but a wrong VRAM estimate fails outright instead of slowing down.

Which GPU for Oobabooga: Quick Comparison

Before the deep dive on each card, here's the decision most people are actually trying to make, model size and workload mapped straight to a GPU.

RTX 4090 for Oobabooga: 7B-13B Models at the Lowest Cost

The RTX 4090 is the right default because most oobabooga usage genuinely fits in 24GB: 7B and 13B models at any quantization, and 30B models once quantized to Q4_K_M. The reasoning is straightforward VRAM math, not a compromise. It's only once you want a longer context window on a larger model that the ceiling shows up, since a bigger context cache eats into the same 24GB the model weights need.

L40S for Oobabooga: 30B Models and Higher-Quality 70B Quantization

The L40S's 48GB solves two problems at once: it fits 30B models without the tight quantization a 24GB card forces, and it leaves enough headroom on a 70B model at Q4_K_M to push context length up considerably. That extra context room is the real reason to pick this tier over the 4090, not just "more VRAM for a bigger model." A 30B model at half the L40S's capacity has room to spare for a context window that would leave a 24GB card with almost nothing left.

A100 80GB for Oobabooga: Full 70B Models Without Compromise

The A100 80GB earns its cost when quantization stops being an acceptable tradeoff: a 70B model at Q8 or near full precision needs 65-70GB or more, which fits comfortably with room for a generous context window. NVLink also makes this the practical choice if you're running a model too large for one card, oobabooga supports splitting a model across multiple GPUs with the --gpu-split flag, and NVLink's bandwidth matters more once a model's layers are spread across devices. For serving multiple models or handling batched requests as a shared service, the larger memory pool is what makes that workable without a tight margin.

GPU Decision Framework: Answer These Five Questions

Model size alone doesn't tell you which GPU to rent. Run through these five questions in order, since each one can override the answer the previous one suggested.

If questions 1 and 2 point to different GPUs, quantization tolerance wins, since it's the variable you actually control. A 70B model doesn't require an A100 by default; it requires one only once you've decided Q4 quality loss isn't acceptable.

Text Generation WebUI Setup: Installing Oobabooga on a Cloud GPU

Setup on a fresh Ubuntu cloud instance follows the same pattern regardless of which card you're on, using the full installation path built for headless servers, not the new portable desktop app.

git clone https://github.com/oobabooga/textgen
cd textgen
python -m venv venv
source venv/bin/activate

nvidia-smi
pip install -r requirements/portable/requirements.txt --upgrade

python download-model.py TheBloke/Llama-2-13B-chat-GGUF

python server.py --listen --api

On a cloud instance, reach the WebUI at http://your-server-public-ip:7860 once that port is open in your provider's firewall or security group. A basic deployment looks like this end to end: your terminal connects over SSH to the cloud GPU instance, the oobabooga server binds to the instance's network interface, and your browser or API client reaches it over the instance's public IP on port 7860 (WebUI) or 5000 (API).

Running Oobabooga Headless: API Access and Remote Config

Beyond a personal chat session, the OpenAI-compatible API on port 5000 matters more than the Gradio interface, giving you a drop-in replacement for OpenAI or Anthropic API calls pointed at your own model.

For a longer-lived setup, running the server inside tmux or screen keeps it alive after you disconnect, and a systemd service is worth setting up if you want automatic restarts after a reboot or crash.

Common Mistakes When Running Oobabooga on a Cloud GPU

Most cloud GPU setup problems trace back to one of five avoidable mistakes.

Choosing Your GPU, Then Deploying It on packet.ai

The framework above should leave you with a clear answer: RTX 4090 for 7B-13B models and most quantized 30B models, L40S for larger 30B models or 70B models at reasonable quantization, or A100 80GB for 70B models where quantization loss isn't acceptable or you need multiple GPUs.

Once you know which card fits, packet.ai deploys any of the three from the same Dedicated infrastructure, typically online in 5-10 minutes. If your usage is occasional or you're still testing model sizes, the Dynamic tier runs at roughly half the Dedicated rate, worth considering before settling into a long-running setup on Dedicated. Current rates for all three GPUs are always listed on the packet.ai pricing page.

If you're also running image generation workloads alongside your LLM setup, our guide to picking a GPU for ComfyUI covers the same RTX 4090, L40S, and A100 lineup from the image-generation side, useful if you're weighing a shared setup across both.

Frequently asked questions

Last reviewed: July 16, 2026. Commands and flags confirmed against the current oobabooga/textgen repository. Model VRAM estimates vary by quantization method and context length; treat the figures above as planning ranges.