R_REDDYX.XYZ
AIPython★ 472⑂ 66 forksApache-2.0

leyten/shard

Shard: Distributed Inference

Shard implements pipeline parallelism for running large language models across multiple GPU nodes, allowing inference without needing to place the entire model in the memory of a single device.

// KEY FEATURES

  • Pipeline parallelism of model layers across GPUs on different machines.
  • Automatic tensor splitting and assembly, minimizing data-transfer delays.
  • Support for Hugging Face Transformers and dynamic batching to increase throughput.
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

PARSEBENCH: AI AGENT EVALUATION★ 472SPONSIO - AI AGENT SECURITY★ 472PILOT - Accelerate Development with AI★ 471BLOCKRUN-MCP: LIVE DATA FOR AI★ 471ROAM-CODE - AI AGENT FOR CODE★ 473NEXT-GEN JAVA AI FRAMEWORK★ 473

← FULL CATALOG

leyten/shard — Shard: Distributed Inference | REDDYX AI