R_REDDYX.XYZ
AIPython★ 1,585⑂ 16 forksApache-2.0

RunanywhereAI/trnrt

LLM Inference on AWS Trainium

Open-source LLM inference engine for AWS Trainium featuring NKI megakernels, 1M-token context, and OpenAI-compatible API.

// KEY FEATURES

  • Runs LLM inference on AWS Trainium via NKI megakernels
  • Supports 1M-token context window for long sequences
  • Provides OpenAI-compatible API for easy integration
#aws#glm#inference#llm#llm-serving#megakernels#neuron#nki#openai-api#trainium
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

UZU: Powerful Engine for AI Models★ 1586AGENTLY: SIMPLIFYING GEN-AI DEVELOPMENT★ 1584AGMSG - CROSS-VENDOR MESSENGER FOR CLI-AGENTS★ 1586UCCL: GPU Communication at Max★ 1586OPENSERP — AI SEARCH API SELF-HOSTED★ 1584AI-DEV-KIT FOR CODING AGENTS★ 1583

← FULL CATALOG

RunanywhereAI/trnrt — LLM Inference on AWS Trainium