INSTALL IN ONE COMMAND
By 2026, Ollama has become the de-facto standard for running LLMs locally — 95,000+ stars on GitHub. The reason is simple: it hides all the hassle with weights, quantization, and backends behind a docker run-level interface.
macOS
On Apple Silicon this is the most trouble-free option: Metal acceleration works out of the box, nothing extra to install.
brew install ollama
brew services start ollama
ollama run llama3.1:8b
Or download the .dmg from ollama.com — then the app will add itself to startup and appear in the menu bar.
Linux
curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama
ollama run llama3.1:8b
The script creates a systemd unit and the ollama user itself. If you have NVIDIA — it will also pull the CUDA backend, but only if a working driver is present (check below).
Windows and Windows Server
Starting with the native build, WSL is no longer mandatory — there's a regular installer. For Windows Server, where GUI is often disabled, install silently:
winget install Ollama.Ollama
# or silent install:
OllamaSetup.exe /VERYSILENT /NORESTART
After installation, open a new terminal window — otherwise ollama won't be found in PATH. This is reason number one for "command not recognized" right after install.
HOW MUCH MEMORY IS NEEDED: TABLE
The main beginner mistake — download a model, then wonder if it fits. You must calculate beforehand. Practical rule for Q4 quantization (the most common compression level): approximately 1 GB memory per 1 billion parameters, plus headroom for context.
| Model | Parameters | RAM/VRAM at Q4 | Comfortable hardware |
|---|---|---|---|
| llama3.2:3b | 3B | ~3 GB | any laptop from 8 GB |
| llama3.1:8b | 8B | ~5 GB | 16 GB RAM / RTX 3060 |
| qwen2.5-coder:14b | 14B | ~9 GB | M-series 16 GB / RTX 4070 |
| gemma2:27b | 27B | ~17 GB | 32 GB RAM / RTX 4090 |
| llama3.3:70b | 70B | ~42 GB | 64+ GB / 2×4090 / Mac Studio |
Context consumes from the top. At 32k tokens add roughly 2-4 GB, at 128k — already up to 15 GB depending on architecture. If you're stuck — enable KV-cache quantization, it gives 3-3.5x more context on the same memory.
ERROR 1: EVERYTHING WORKS, BUT PAINFULLY SLOW
Symptom: 1-2 tokens per second, answer to a simple question takes a minute. Cause almost always one — model runs entirely on CPU, GPU not utilized.
Check on NVIDIA:
nvidia-smi
ollama ps
In ollama ps output, look at the PROCESSOR column. If it shows 100% CPU — GPU not picked up. What to do:
- Driver too old. Need version 525+, realistically stable from 550+. This is the cause of most detection failures.
- Model didn't fit in VRAM. Ollama offloads extra layers to CPU. Take a smaller quant or a more compact model.
- CUDA toolkit version mismatch. Ollama pulls its own runtime, conflict with system CUDA breaks detection.
On Apple Silicon this problem doesn't exist: Metal works through system frameworks. If it's slow on Mac — means the model simply exceeds unified memory size and swapping started.
ERROR 2: CAN'T REACH OVER NETWORK (localhost:11434)
By default Ollama listens only on 127.0.0.1:11434. From another machine, from a Docker container, or from a phone you won't reach it — this is not a bug, but a conscious security default.
Open to outside on Linux:
systemctl edit ollama
# add:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
systemctl daemon-reload && systemctl restart ollama
On macOS:
launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
brew services restart ollama
On Windows — environment variable OLLAMA_HOST with value 0.0.0.0:11434, then restart the service.
Important about security. Ollama has no built-in auth. Exposing the port directly to the internet gives your GPU to anyone. Close with firewall to trusted subnet or put nginx with basic-auth in front.
ERROR 3: WHICH MODEL TO CHOOSE AT ALL
Ollama pulls models from its registry, but nothing prevents you from using your own GGUF file — for example, downloaded from Hugging Face.
# через реестр
ollama pull qwen2.5-coder:14b
# свой GGUF-файл
cat > Modelfile <<'EOF'
FROM ./model-q4_k_m.gguf
PARAMETER num_ctx 8192
EOF
ollama create mymodel -f Modelfile
Guideline for tasks as of mid-2026:
- Code — qwen2.5-coder, deepseek-coder-v2
- Russian language — qwen2.5, gemma2, from local options — GigaChat family
- Weak hardware — llama3.2:3b, phi-4-mini
- Reasoning — deepseek-r1 and its distillates
Fresh releases and their local builds are conveniently tracked in the REDDYX repository catalog — you can see what appeared in a week and how many stars it collected.
RUNNING AS A SERVICE ON A SERVER
For Windows Server and Linux servers, it's important that Ollama survives reboots and doesn't depend on a user session.
# Linux — уже настроено установщиком, просто включить автозапуск
systemctl enable --now ollama
# проверка снаружи
curl http://SERVER_IP:11434/api/tags
On Windows Server, it's more convenient to wrap it as a service via NSSM if the standard installer put Ollama in user autostart rather than system services.
Useful variables for the server:
| Variable | What it does |
|---|---|
OLLAMA_KEEP_ALIVE | How long to keep the model in memory after a request (default 5m, for servers set to -1) |
OLLAMA_MAX_LOADED_MODELS | How many models to keep loaded simultaneously |
OLLAMA_NUM_PARALLEL | Parallel requests to a single model |
OLLAMA_MODELS | Where to store weights (move to a large disk) |
CONNECTING TO APPLICATIONS: OPENAI-COMPATIBLE API
Ollama provides an OpenAI-compatible endpoint — this means almost any tool that works with OpenAI can switch to a local model by changing the base_url.
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama", # любое непустое значение
)
r = client.chat.completions.create(
model="llama3.1:8b",
messages=[{"role": "user", "content": "Привет"}],
)
print(r.choices[0].message.content)
This is how Open WebUI, Continue, Cline, LangChain, and dozens of other clients connect.
FAQ
Can you train a model on a MacBook via Ollama?
Ollama is for inference; it doesn't do training. For fine-tuning on Apple Silicon, check out MLX from Apple or LoRA tuning via unsloth on a rented GPU — on a laptop, full fine-tuning of even a small model will hit memory and time limits.
Why doesn't Ollama see the downloaded GGUF?
The file itself isn't picked up — you need a Modelfile with the FROM directive and the ollama create command. Just placing the .gguf in the models folder isn't enough.
What command to download Ollama on Linux?
curl -fsSL https://ollama.com/install.sh | sh — universal option for Debian/Ubuntu/Fedora/Arch. Some distros have a package in their repository, but it often lags several versions behind.
How many models can you keep loaded simultaneously?
Depends on memory and OLLAMA_MAX_LOADED_MODELS. In practice: two 8B models on 32 GB RAM coexist fine; beyond that, unloading and lag during switching begin.
