R_REDDYX.XYZ

GPU: open-source projects on GitHub

48 repositories on GPU, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Inference 95Cuda 48AI Agents 1319Pytorch 98Machine Learning 190LLM Inference 76Self Hosted 359Deepseek 229Nvidia 24Local LLM 107Vllm 69Ollama 249LLM Serving 24
deepspeedai/DeepSpeed

DeepSpeed: optimization library for training

DeepSpeed is an optimization deep learning library ensuring efficient distributed training and inference. Supports ZeRO, MoE and parallelism for training models with hund

★ 43,332 · Python
bojieli/ai-infra-book

AI Infrastructure Book

Open‑source Chinese book on AI infrastructure, covering LLM inference and training system design with quantitative analysis, plus PDF, code tools, and experiments.

★ 7,748 · Python
deepflowio/deepflow

DEEPFLOW: eBPF TRACING AND PROFILING

Deepflow is a Go project for distributed tracing and profiling using eBPF. It lets you quickly find bottlenecks in microservices.

★ 4,325 · Go
lemonade-sdk/lemonade

Lemonade SDK

Lemonade helps users run local AI applications by optimizing LLM performance on their GPU and NPU. Join our Discord!

★ 4,059 · C++
llm-d/llm-d

LLM inference acceleration on Kubernetes

Achieve state of the art inference performance with modern accelerators on Kubernetes

★ 3,247 · Shell
thu-pacman/chitu

CHITU: ACCELERATING LLMS IN PYTHON

High-performance framework for fast inference of large language models in Python. Provides flexibility, efficiency, and ease of deployment.

★ 3,123 · Python
shinyobjectz/raea

AI GENERATES 3D AND VIDEO ON YOUR GPU

Raea is an engine for rendering video, 3D, and games via AI agents. Write tasks in React/TypeScript, your GPU computes. Built with Rust + Bevy.

★ 2,457 · Rust
kubeflow/trainer

Trainer: Distributed Training

The trainer project enables distributed training of AI models and fine‑tuning of LLMs in Kubernetes, written in Go.

★ 2,284 · Go
withcatai/node-llama-cpp

RUN LLM LOCALLY IN NODE.JS

Run AI models directly on your computer via Node.js. Enforced JSON schema validation right at the generation level.

★ 2,205 · TypeScript
beam-cloud/beta9

BETA9 - Lightning GPU Output

Ultrafast serverless GPU inference with sandboxes and background jobs, written in Go for instant scaling.

★ 1,664 · Go
uccl-project/uccl

UCCL: GPU Communication at Max

A fast library for efficient data exchange between GPUs: collective operations, P2P, and inference acceleration.

★ 1,586 · C++
autonomous-ai/autonomous-computer

BUILD YOUR AI COMPUTER

The project helps you build your personal AI computer with open-source code and support for autonomous agents.

★ 1,577
voyager-so/vork

Headless 3D Engine for AI Agents

A Rust-based headless 3D and VFX engine designed for AI agents to simulate, render, and interact with virtual environments without a GUI.

★ 1,565 · Rust
psalias2006/gpu-hot

🔥 GPU REAL‑TIME DASHBOARD

GPU-hot — a web dashboard for monitoring NVIDIA GPU showing temperature, load, and memory in real time. Already over 1,500 stars on GitHub! 🌟

★ 1,543 · JavaScript
tenstorrent/tt-metal

tt-metal - Metallic Operations Library

TT-NN operator library and low-level programming model for TT-Metalium kernels to accelerate AI workloads on proprietary hardware.

★ 1,505 · C++
l0ng-ai/tty7

TTY7: FUTURE TERMINAL IN RUST

A Rust-based terminal workspace with GPU rendering, persistent sessions, SSH, and AI agent integration for coding.

★ 1,501 · Rust
eagle-hq/EAGLE-4

EAGLE-4 Speculative Decoding

Official inference code for EAGLE-4, a speculative decoding method that accelerates LLM generation by drafting tokens via feature-level extrapolation.

★ 1,287 · Python
NVIDIA/raft

RAFT - Accelerate ML on CUDA

The RAFT library provides CUDA‑accelerated ML and IR algorithms, simplifying the creation of high‑performance applications.

★ 1,054 · Cuda
Xiangyue-Zhang/auto-deep-researcher-24x7

AI Agent 24/7

An autonomous AI agent runs your deep experiments 24/7, saving resources and memory. The Leader‑Worker architecture ensures stable operation at zero cost.

★ 1,035 · Python
kennss/SiliconScope

SiliconScope — Apple Silicon Monitor

Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.

★ 1,024 · Swift
Andyyyy64/whichllm

Which local LLM to choose?

Find a local model with the best performance on your hardware based on real benchmarks, not parameter count. One command — instant launch.

★ 947 · Python
mosecorg/mosec

MOSEC - MAXIMUM ML PRODUCTIVITY

Fast framework for serving ML models with dynamic batching and CPU/GPU pipelines.

★ 904 · Python
NVIDIA/cuvs

CUVS - Accelerated Vector Search

cuvs is a library for fast vector search and clustering on GPU, using CUDA for maximum performance.

★ 897 · Cuda
pegainfer-project/pegainfer

PEGAINFER: Rust + CUDA Infer

Pure Rust LLM engine with CUDA acceleration, OpenAI API compatible, runs Qwen3 and Kimi‑K2 without PyTorch.

★ 742 · Rust
devitocodes/devito

DEVITO — Computations on Steroids

DSL and compiler for automated finite-difference computations and stencil operations in Python.

★ 719 · Python
jmerelnyc/Talos

Talos worker client

GPU worker client for the Talos network. It connects to your account, receives open‑model inference tasks via WebSocket, and reports uptime for reward calculation.

★ 629 · Python
peters/horizon

Horizon — GPU terminal

GPU-accelerated terminal for sessions on an infinite canvas with intuitive control.

★ 607 · Rust
0xS4N4TG/kinetograph

FASTEST VIDEO ENGINE FOR AGENTS

Kinetograph is a programmatic video engine in TypeScript for AI agents. Uses GPU and FFmpeg to render in seconds. Supports MCP and Hyperframes concept.

★ 520 · TypeScript
david-xinyuwei/david-share

AI AGENTS ON AZURE AND GPU

Repository with code, benchmarks, and guides for AI agents and Deep Learning. Deployment on Microsoft Foundry and Azure, fine-tuning, inference optimization, and GPU test

★ 451 · Jupyter Notebook
ekzhang/resin

EXPERIMENTAL INFERENCE LIBRARY

resin — experimental Python library for building LLM inference engines. Supports CUDA, GPU acceleration, and models like DeepSeek and GPT-OSS. Early-stage project with 25

★ 432 · Python
starling-build/starling

Starling: New Linux DE

Starling is a new Linux desktop built with Swift, featuring its own compositor and a framework for porting Flutter apps. It includes a fast shell and a set of branded app

★ 417 · Swift
zolotukhin/zinc

ZINC - LOCAL LLM ON GPU

Zinc is an LLM inference engine in Zig that runs on AMD GPU and Apple Silicon without external dependencies.

★ 395 · Zig
openlake-project/openlake

OpenLake: GPU Storage

Ultra-efficient storage for GPU workloads. Deliver data at lightning speed.

★ 385 · Rust
ModelEngine-Group/unified-cache-management

ACCELERATE LLM WITH CACHE

The unified-cache-management project enables storing and reusing KV Cache to accelerate large language models.

★ 356 · Python
DataTalksClub/production-ai-agents-for-media-generation

PRODUCTION AI AGENTS: MEDIA GENERATION

Course teaches building production agents for generating images, video, and audio on serverless GPUs. 254 GitHub stars. Free, apply now.

★ 349
robzilla1738/harness-terminal

GPU TERMINAL FOR CODING

Native macOS terminal with GPU rendering, persistent sessions, and coding-agent workflow support. Enables SSH connections, runs as a daemon and CLI tool on Linux, and opt

★ 309 · Swift
T-Crypt/berth

GPU BY TURN WITHOUT CRASHES

Berth leases the GPU to chat, image, and voice engines in turn. Two processes never load together and crash each other. Single Go binary, no dependencies.

★ 297 · Go
mohitsoni48/TurboLLM

TURBO LLM: OFFLINE WITH AI

Run any local LLM with automatic GPU tuning, a web UI, and an OpenAI/Anthropic‑compatible API.

★ 296 · TypeScript
beehive-lab/jitllm

JITLLM: HIGH PERFORMANCE LLM ON JVM

JITLLM — Java framework for high‑performance LLM inference and serving on the JVM. Supports GPU acceleration, GGUF and quantized model formats, and integration with DeepS

★ 295 · Java
AMD-AGI/Hyperloom

HYPERLOOM AUTO-OPTIMIZATION LLM ON AMD

Hyperloom — agentic system that automatically optimizes LLM workloads on AMD GPUs by selecting kernels and tuning parameters for peak performance without manual setup.

★ 271 · Python
defilantech/LLMKube

LLMKUBE: LLM IN KUBERNETES

LLMKube - Kubernetes operator for LLM inference on heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal. Supports llama.cpp, vLLm, TGI, mlx-server, multi

★ 266 · Go
SikamikanikoBG/homelab-monitor

HOMELAB MONITOR DASHBOARD

Plug‑and‑play dashboard for homelab in a single container: monitoring GPU, local‑AI VRAM, Docker, systemd and host health. Built‑in read‑only MCP server gives AI agents a

★ 236 · Python
aryamandarda/tamserve

Live Multi-Camera SAM Tracking

Serves multiple cameras with one SAM model per GPU, achieving 4x memory reduction and faster inference than original implementations.

★ 221 · Python
uncSoft/anubis-oss

ANUBIS: BENCHMARK LLM ON APPLE SILICON

anubis-oss — Swift utility for local LLM launch and benchmarking on Apple Silicon. It measures speed and accuracy of GPU inference, comparing results with CPU baseline an

★ 217 · Swift
surya-koritala/Glyd

GLYD: LOSSLESS AI COMPRESSION

Glyd reduces GPU memory by 33% without losing data, ensuring bit‑for‑bit identity. The project is written in Rust and supports CUDA, C‑ABI, and CLI for convenient benchma

★ 200 · Rust
VillagePrefect/stable-diffusion-atelier

Local Stable Diffusion Reference

Curated notes on samplers, LoRAs, and VRAM optimization for running Stable Diffusion locally.

★ 98
BehemothDyer/stable-diffusion-atelier

Stable Diffusion Local Guide

Curated notes on samplers, LoRA training, and VRAM optimization for running diffusion models locally.

★ 97
DaoyuanLi2816/can-i-finetune-this

Can I finetune this?

Tool to check whether a Hugging Face model fits in your GPU memory and can be fine-tuned locally.

★ 31 · Python

New projects on this topic in Telegram

Telegram →
GPU — 48 open-source GitHub projects | REDDYX AI