Why run LLM locally
Three reasons: data never leaves the machine (compliance), no latency on API calls, no cost for tokens. In 2026, models have been compressed: Llama 3 8B in 4-bit quantization weighs 4.7 GB and runs on any MacBook M-series or laptop with 16 GB RAM.
Installation in one command
# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh
# Windows — скачать installer с ollama.com
# После установки
ollama serve # запускает сервер на localhost:11434
Running models
# Llama 3 8B — баланс скорости и качества
ollama run llama3
# Mistral 7B — отличен для кода
ollama run mistral
# Gemma 2 9B — Google, сильный reasoning
ollama run gemma2
# Qwen 2.5 Coder — лучший для кода среди малых моделей
ollama run qwen2.5-coder
OpenAI-compatible API
Ollama provides a REST API fully compatible with OpenAI. Any code using the openai Python SDK works without changes — you only need to change the base_url.
from openai import OpenAI
client = OpenAI(
base_url='http://localhost:11434/v1',
api_key='ollama', # any string
)
response = client.chat.completions.create(
model='llama3',
messages=[{'role': 'user', 'content': 'Привет'}]
)
print(response.choices[0].message.content)
Integration with VS Code and Cursor
In Continue (VS Code extension), in settings.json you specify provider ollama and model llama3. Cursor supports Ollama via Custom API with base URL http://localhost:11434/v1. Autocomplete works fully offline.
Benchmark on different hardware
| Hardware | Model | Tokens/sec | RAM |
|---|---|---|---|
| MacBook M3 Pro 18GB | Llama 3 8B Q4 | 35 | 6.2 GB |
| MacBook M3 Pro 18GB | Mistral 7B Q4 | 38 | 5.1 GB |
| RTX 4090 24GB | Llama 3 70B Q4 | 48 | 21 GB |
| CPU Intel i7-13700 | Llama 3 8B Q4 | 8 | 6.2 GB |
FAQ
Can Ollama be run in Docker?
Yes. docker run -d -v ollama:/root/.ollama -p 11434:11434 ollama/ollama. GPU support via nvidia-container-toolkit.
Which model to choose for code?
qwen2.5-coder:7b — best result on HumanEval among models under 10B parameters in 2026. For 32B+ — deepseek-coder-v2.