AIPython★ 214⑂ 111 forksApache-2.0
mindspore-ai/hyper-parallel
LLM TRAINING ACCELERATION ON SUPERPOD
Framework for distributed LLM training with SuperPod topology awareness. Optimizes process placement and inter-GPU communication. Supports PyTorch and scales to thousands of accelerators.
// KEY FEATURES
- Accounts for SuperPod topology for process affinity
- Minimizes inter-GPU communication during all-reduce
- Integrates with PyTorch Distributed without code changes
- Scales LLM training to thousands of H100 GPUs
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram