R_REDDYX.XYZ
DevToolsPython★ 43,332⑂ 5,075 forksApache-2.0

deepspeedai/DeepSpeed

DeepSpeed: optimization library for training

DeepSpeed is an optimization deep learning library ensuring efficient distributed training and inference. Supports ZeRO, MoE and parallelism for training models with hundreds of billions of parameters.

// KEY FEATURES

  • Distributed training of models up to 530B parameters
  • Support for ZeRO, MoE and 3D-parallelism
  • Integration with Transformers, Accelerate and Lightning
  • Installation via pip, independent of CUDA/PyTorch versions

// DEEP DIVE: HOW IT WORKS

DeepSpeed is an optimization deep learning library simplifying distributed training and model inference. It includes ZeRO, ZeRO-Infinity, 3D-parallelism, Ulysses Sequence Parallelism and DeepSpeed-MoE for scaling to hundreds of billions of parameters. Integrates with Transformers, Accelerate, Lightning, MosaicML, Determined and MMEngine frameworks.

The repository supports building on hardware NVIDIA, AMD, CPU, Intel Gaudi, Intel XPU and Huawei Ascend NPU, making it universal for different servers. pip-based installation is not tied to specific PyTorch or CUDA versions, simplifying the start of projects of any scale.

Recommended for training and inference of large models requiring efficient memory and computational resources. Project office hours take place on the last Tuesday of the month via Zoom, and the latest news cover optimizer updates, offloading and new parallelism methods.

// USE CASES

Best for: To developers and researchers working with large language models

  • Train models up to 530B parameters on multiple GPUs
  • Integrate LLM into projects based on Transformers
  • Scale training with ZeRO-Infinity and MoE
#billion-parameters#compression#data-parallelism#deep-learning#gpu#inference#machine-learning#mixture-of-experts#model-parallelism#pipeline-parallelism#pytorch#trillion-parameters
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

FRAMEWORK FOR CHEAPER LARGE MODEL TRAINING★ 41422SELF-HOSTED GIT SERVICE IN GO★ 47943API LAYER FOR LOCAL AI APPLICATIONS★ 57640OPEN-SOURCE DESIGN TOOL IN CODE★ 27239FAST AI-POWERED SEARCH ENGINE★ 59884HIGH-PERFORMANCE NEURAL NETWORK INFERENCE FRAMEWORK FOR MOBILE★ 24068

← FULL CATALOG

deepspeedai/DeepSpeed — DeepSpeed: optimization library for…