WHY RENT, NOT BUY
Let's do the math. An H100 SXM in 2026 costs about $25-30k per card, and a full 8-GPU node with NVLink, fast network storage, and cooling is already hundreds of thousands in capex plus electricity, rack, and maintenance. The card becomes obsolete in 2-3 years: Blackwell (B200) is already in production, GB200 racks are deployed by major clouds, and the next generation is on the horizon.
GPU rentals eliminate three pains at once: zero capex, elasticity (grab 64 cards for a weekend run — then return), and access to hardware you’d otherwise wait months for. Purchase only makes sense with constant 24/7 load for years — then breakeven is somewhere around 12-18 months of continuous use. If your GPUs sit idle even a third of the time, the cloud wins mathematically.
PROVIDER TYPES: WHO’S WHO
The GPU rental market in 2026 splits into three clear camps, and mixing them up is a typical newcomer mistake that costs money.
- Hyperscalers (AWS, GCP, Azure) — maximum reliability, integration with the rest of the infrastructure, enterprise-SLA. You pay a 2-4x premium for this. Justified when GPU is part of a large production stack.
- Specialized GPU clouds (Lambda, CoreWeave, Crusoe, Nebius) — tuned specifically for ML. Fast InfiniBand between nodes, ready images with CUDA/PyTorch, reasonable prices. The sweet spot for serious model training.
- Marketplaces and decentralized networks (Vast.ai, RunPod, TensorDock, io.net) — aggregate others’ hardware, including cards from private owners and second-tier data centers. Lowest prices, but higher quality variability and fewer guarantees.
Fresh releases of new providers and tools around ML infrastructure are tracked in the REDDYX catalog — the 2026 market changes every week.
PRICE COMPARISON FOR H100 AND BEYOND
Below are indicative prices per GPU‑hour as of 2026. These are averaged from community measurements and public pricelists — spot and regional rates fluctuate, always check for your rental date. GPU prices on marketplaces change literally by the hour depending on demand.
| Provider | Type | H100 (on-demand, $/hr) | A100 80GB ($/hr) | Spot/cheap mode |
|---|---|---|---|---|
| AWS (p5) | HyperScaler | ~$12-13 (based on 8x node) | ~$5-7 | Spot -60-70% |
| GCP (a3) | HyperScaler | ~$11-12 | ~$4-6 | Preemptible |
| Azure (ND) | HyperScaler | ~$11-13 | ~$5-6 | Spot |
| Lambda | GPU cloud | ~$2.5-3.5 | ~$1.3-1.8 | Reserved cheaper |
| CoreWeave | GPU cloud | ~$3-5 | ~$2 | Reserved |
| RunPod | Marketplace | ~$2-3 | ~$1.2-1.7 | Community -30% |
| Vast.ai | Marketplace | ~$1.5-2.5 | ~$0.8-1.4 | Interruptible -50% |
The takeaway is simple: for the same H100, AWS on-demand costs roughly four times more than Vast.ai. The difference isn’t the silicon — it’s the same card. The difference lies in SLA, inter‑node network, support, and stability. For a single fine‑tune you don’t need that premium.
SPEED: WHAT ACTUALLY MATTERS
Hourly price is only half the equation. You must calculate the cost of a completed run, not per hour. A slow node at $2 can end up more expensive than a fast one at $3.
What kills throughput
- Inter-node network. In multi-GPU training on 8+ cards, the bottleneck is not the GPUs themselves, but InfiniBand between nodes. For hyperscalers and Lambda/CoreWeave this is 3200 Gbps NDR InfiniBand. On a cheap marketplace, often ordinary 10-25 Gbps Ethernet — distributed training stalls.
- Storage I/O. If the dataset is read from a slow disk, expensive H100s idle waiting for data. Need local NVMe or fast network parallel storage.
- Throttling and sharing. On marketplaces, a host may split PCIe or CPU between tenants. Measure it yourself.
Practical rule 2026: for single-node (up to 8 GPU on one machine with NVLink) take the marketplace boldly — inter-node network is not used. For distributed training of a model on tens of cards, pay for real InfiniBand from Lambda/CoreWeave/hyperscaler, otherwise money for GPU-hours will burn on communication.
HOW TO MEASURE REAL THROUGHPUT
Don't trust marketing TFLOPS. Before running a multi-hour job, spend a couple minutes checking that the rented card delivers what's expected. Here's a minimal script: it warms up the GPU with matrix multiplication in bf16 and measures achieved TFLOPS.
import torch, time
assert torch.cuda.is_available(), "CUDA не видна"
dev = "cuda"
n = 8192 # размер квадратной матрицы
a = torch.randn(n, n, device=dev, dtype=torch.bfloat16)
b = torch.randn(n, n, device=dev, dtype=torch.bfloat16)
# прогрев
for _ in range(10):
c = a @ b
torch.cuda.synchronize()
iters = 100
t0 = time.perf_counter()
for _ in range(iters):
c = a @ b
torch.cuda.synchronize()
dt = time.perf_counter() - t0
# 2*n^3 флопов на одно матричное умножение
flops = 2 * (n ** 3) * iters
tflops = flops / dt / 1e12
print(f"{torch.cuda.get_device_name(0)}: {tflops:.1f} TFLOPS bf16")
print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory/1e9:.0f} GB")
A healthy H100 SXM on bf16 matrices should show hundreds of TFLOPS. If you see a figure several times lower than the spec — the card is throttling, shared with a neighbor, or something is wrong with the driver. Better to find out now than after 6 hours of runtime. Additionally, run a network check if you plan multi-node:
# проверка InfiniBand / скорости интерконнекта
ib_write_bw -d mlx5_0 --report_gbits # на паре нод
# и NCCL all-reduce тест как ближе к реальности
nccl-tests/build/all_reduce_perf -b 8 -e 1G -f 2 -g 8
COST-SAVING STRATEGY: HOW TO PAY LESS
Several techniques that truly cut the cloud bill, verified in practice.
- Spot/interruptible with checkpoints. Preemptible instances are 50-70% cheaper than on-demand. Write a checkpoint every N steps, then interruption costs you minutes, not the whole run.
- Correct precision. bf16/fp8 instead of fp32 — it's faster and uses less VRAM, so you can take a smaller card. On Blackwell, fp8 training gives a noticeable throughput boost.
- Don't overpay for VRAM. For LoRA/QLoRA fine-tuning of a medium model, often a single A100 40GB or even a 4090/L40S for pennies is enough, not an H100 cluster.
- Reserved for steady load. If you run inference 24/7 — a yearly reserved with Lambda cuts the price nearly in half vs on-demand.
- Shut down the instance. Trivial, but the top reason for blown budgets — a forgotten running node on weekends. Set auto-shutdown on idle.
WHAT TO CHOOSE FOR THE TASK
Short decision matrix for 2026:
- Learning / pet-project / experiments: Vast.ai or RunPod Community, spot mode, one card. Cheap, quick to spin up.
- Serious fine-tuning of one model (single-node 8x): RunPod Secure Cloud or Lambda on-demand. Stable, NVLink in place.
- Pretraining / distributed training (tens‑hundreds of GPUs): Lambda, CoreWeave, Nebius or hyperscaler with InfiniBand and reserved contract.
- Production inference with SLA: hyperscaler or Lambda reserved, closer to your users by region.
Frequently Asked Questions
How much does it cost to rent an H100 in 2026?
On specialized GPU clouds and marketplaces — roughly $1.5-3.5 per GPU‑hour on‑demand, spot mode is even cheaper. With hyperscalers (AWS, GCP, Azure) the same H100 runs $11‑13 per hour due to the SLA and network premium. The exact price depends on region, volume, and contract type.
Is it better to buy your own GPU or rent?
Purchase pays off only with near‑round‑the‑clock utilization for 12‑18 months or longer. If the GPU sits idle a third of the time, cloud rental is cheaper thanks to zero CAPEX and no electricity, cooling, or maintenance costs. For most teams and solo developers, renting is preferable.
Можно ли обучать модель на дешёвом спот-инстансе?
Да, если реализованы чекпоинты. Прерываемые (спот/interruptible) инстансы стоят на 50-70% дешевле, а прерывание при регулярном сохранении состояния стоит лишь минуты потерянного прогресса, а не всего прогона. Для распределённого обучения на многих нодах спот рискованнее — там важна стабильность кластера.
Какая карта нужна для файн-тюна LLM?
Для LoRA/QLoRA средних моделей часто достаточно одной A100 40GB, L40S или даже RTX 4090 при работе в 4-bit. Полный файн-тюн крупных моделей требует H100 80GB и мульти-GPU ноды с NVLink. Считайте по объёму VRAM под веса, оптимизатор и активации, а не по названию карты.