R_REDDYX.XYZ
Models2026-07-26

Your AI Server in 2026: Open WebUI + Ollama for the Team in an Evening

Your AI Server in 2026: Open WebUI + Ollama for the Team in an Evening
TL;DR: A private ChatGPT alternative for a team can be set up in an evening: Ollama as the engine, Open WebUI as the interface, nginx with a certificate on top. Data never leaves your server, there is no cost, users and roles are built‑in. Below — docker‑compose, nginx config, RAG setup on internal documents and the pitfalls everyone runs into.

WHY THIS IS NEEDED

Three reasons why teams self‑host instead of subscribing:

  • Privacy. Code, contracts and correspondence do not go to someone else's cloud. For many industries this is not a wish but a requirement.
  • Cost. A subscription for a team of 20 people per year costs as much as a decent graphics card, which then works forever.
  • Availability. No dependence on blocks, foreign currency cards, or sudden tariff changes.

WHAT YOU NEED

ComponentMinimumComfortable
GPURTX 3060 12GBRTX 4090 / A6000
RAM16 GB64 GB
Disk100 GB SSD1 TB NVMe
OSUbuntu 22.04+Ubuntu 24.04 LTS

Without a GPU it will still work, but on CPU generation proceeds at 1‑2 tokens per second — fine for a demo, not for production.

STEP 1: DOCKER COMPOSE

One file brings up the whole stack. We place it in /opt/ai-server/docker-compose.yml:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ./ollama:/root/.ollama
    environment:
      - OLLAMA_KEEP_ALIVE=-1
      - OLLAMA_NUM_PARALLEL=4
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=True
      - ENABLE_SIGNUP=False

Two important notes. The port is forwarded to 127.0.0.1, not outward — only nginx will be exposed externally. And ENABLE_SIGNUP=False: otherwise anyone who finds your address can register themselves.

cd /opt/ai-server
docker compose up -d
docker compose exec ollama ollama pull qwen2.5:14b

The first registered user automatically becomes an administrator — log in right after startup and create your account before someone else does.

STEP 2: NGINX AND HTTPS

You cannot expose port 8080 to the internet: there is neither encryption nor protection against brute force. We install nginx.

server {
    server_name ai.example.com;

    client_max_body_size 100M;   # загрузка документов в RAG

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;

        # стриминг ответов: без этого текст появляется рывками
        proxy_buffering off;
        proxy_read_timeout 600s;
    }

    listen 443 ssl;
}

Two lines that are often forgotten: proxy_buffering off — otherwise the model's response accumulates in the buffer and comes out in chunks instead of character‑by‑character streaming, and proxy_read_timeout — otherwise long responses are cut off by timeout.

certbot --nginx -d ai.example.com

STEP 3: USERS AND ROLES

Open WebUI can restrict access out of the box. It is configured in the admin panel:

  • Roles — admin, user, pending. New accounts can be kept in pending until approval.
  • Model access — keep the heavy 70B model only for part of the team, others get 8B.
  • Groups — different sets of models and prompts for departments.
  • History — each user has their own; by default the admin does not see others' chats.

STEP 4: RAG ON COMPANY DOCUMENTS

The most valuable part for business: the model answers based on your regulations, not general knowledge from the internet.

In Open WebUI this is the Workspace → Knowledge section. You upload PDF, docx, markdown — the system automatically chunks them, computes embeddings and stores them in a vector database. Then in chat you specify the collection via # and the model answers based on it.

What to tweak for quality:

ParameterValueWhy
Chunk size800-1200Too small chunks lose context
Chunk overlap100-200so the thought isn't split between chunks
Top K4-6more — noise, less — missing facts
Embedding modelmultilingualmandatory for Russian documents

The default English embedding model works noticeably worse on Russian documents — this is the first thing to change.

THE PITFALLS EVERYONE STEPS ON

  • The model is unloaded between requests. The first answer after a pause takes a minute. Fixed with OLLAMA_KEEP_ALIVE=-1, the model stays in memory.
  • Open registration. Forgot ENABLE_SIGNUP=False — and your server is used by the whole internet.
  • Disk ran out. Models take up tens of gigabytes, chat history and vector database grow. Monitor disk space.
  • One request hangs everyone. With OLLAMA_NUM_PARALLEL=1 the command waits in queue. Set it according to the number of users if memory allows.
  • No backup. The webui folder contains all history and settings. Lose it — lose everything.

HOW MUCH DOES IT COST

One-time investments versus subscription for a team of 20 people:

ItemYour hardwareCloud subscription
Startprice of a GPU server0
Per monthelectricity20 × rate per user
After a yearhardware remains yourspay again
Privacyfullaccording to vendor agreement
Model qualitylower than topbest on the market

Honest conclusion: your hardware wins over a year horizon and where privacy is critical. If you need the maximum model right here and now — cloud is stronger.

FAQ

Will the server handle a team of 20 people?

If not everyone works simultaneously — yes, one 4090 is enough. With real parallel load, calculate based on OLLAMA_NUM_PARALLEL and memory: each parallel slot requires its own KV cache.

Can cloud models also be connected?

Yes, Open WebUI can work with any OpenAI-compatible API simultaneously with Ollama. Convenient scheme: private — locally, complex — in the cloud.

How to update?

docker compose pull && docker compose up -d. Data in mounted folders is not touched by the update. Still make a backup of the webui folder before updating.

Does it work with Russian documents in RAG?

It works, but you must switch the embedding model to a multilingual one in the settings. With the default English model, search quality on Russian texts noticeably drops.

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram

← All articles

Your AI Server in 2026: Open WebUI + Ollama for the Team in an Evening | REDDYX AI