AIPython★ 90⑂ 4 forksApache-2.0
datologyai/zephon
Elastic Data Loading
Zephon offers fast, flexible, elastically deterministic data loading for foundation‑model training, ensuring reproducible sampling and high throughput.
// KEY FEATURES
- Provides elastic, deterministic shuffling for reproducible ordering across epochs.
- Integrates with PyTorch DataLoader, supporting custom samplers and collate functions.
- Scales to large‑scale foundation model training with low overhead and high throughput.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram