Why the stack is changing
Pandas is slow on big data — everyone knew that. In 2026 Polars became production-ready and is displacing Pandas in new projects. PyTorch 2.x with torch.compile speeds up training by 30-50% without code changes. ONNX Runtime has become the standard for cross-platform model deployment.
Data: Polars instead of Pandas
pip install polars
import polars as pl
# Читаем 10GB CSV — Polars использует lazy evaluation
df = pl.scan_csv("big_dataset.csv")
result = (
df
.filter(pl.col("score") > 0.5)
.group_by("category")
.agg(pl.col("value").mean().alias("avg_value"))
.sort("avg_value", descending=True)
.collect() # выполняем только здесь
)
print(result)
On a dataset of 5M rows: Pandas 4.2 sec, Polars 0.3 sec. 14x faster on a single core.
Training: PyTorch 2.4 + compile
import torch
import torch.nn as nn
model = MyTransformerModel()
# torch.compile — один вызов, 30-50% прирост скорости
model = torch.compile(model)
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)
for batch in dataloader:
loss = model(batch)
loss.backward()
optimizer.step()
Export: ONNX for any platform
# Экспорт PyTorch модели в ONNX
torch.onnx.export(
model,
dummy_input,
"model.onnx",
opset_version=17,
input_names=["input"],
output_names=["output"],
dynamic_axes={"input": {0: "batch_size"}}
)
# Инференс на любой платформе через ONNX Runtime
import onnxruntime as ort
sess = ort.InferenceSession("model.onnx",
providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
output = sess.run(None, {"input": input_data})
API: FastAPI
from fastapi import FastAPI
import onnxruntime as ort
import numpy as np
app = FastAPI()
sess = ort.InferenceSession("model.onnx")
@app.post("/predict")
async def predict(data: dict):
input_arr = np.array(data["features"], dtype=np.float32)
output = sess.run(None, {"input": input_arr[None, :]})
return {"prediction": output[0].tolist()}
FAQ
When is Pandas still needed?
When the code is already written in Pandas and refactoring doesn't pay off, or when working with Jupyter in exploratory mode where speed is not critical.
Is MLflow needed in this stack?
For a team of 2+ people — yes. MLflow adds experiment tracking and model registry without extra complexity.