GPU: open-source projects on GitHub
48 repositories on GPU, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
DeepSpeed: optimization library for training
DeepSpeed is an optimization deep learning library ensuring efficient distributed training and inference. Supports ZeRO, MoE and parallelism for training models with hund
★ 43,332 · PythonAI Infrastructure Book
Open‑source Chinese book on AI infrastructure, covering LLM inference and training system design with quantitative analysis, plus PDF, code tools, and experiments.
★ 7,748 · PythonDEEPFLOW: eBPF TRACING AND PROFILING
Deepflow is a Go project for distributed tracing and profiling using eBPF. It lets you quickly find bottlenecks in microservices.
★ 4,325 · GoLemonade SDK
Lemonade helps users run local AI applications by optimizing LLM performance on their GPU and NPU. Join our Discord!
★ 4,059 · C++LLM inference acceleration on Kubernetes
Achieve state of the art inference performance with modern accelerators on Kubernetes
★ 3,247 · ShellCHITU: ACCELERATING LLMS IN PYTHON
High-performance framework for fast inference of large language models in Python. Provides flexibility, efficiency, and ease of deployment.
★ 3,123 · PythonAI GENERATES 3D AND VIDEO ON YOUR GPU
Raea is an engine for rendering video, 3D, and games via AI agents. Write tasks in React/TypeScript, your GPU computes. Built with Rust + Bevy.
★ 2,457 · RustTrainer: Distributed Training
The trainer project enables distributed training of AI models and fine‑tuning of LLMs in Kubernetes, written in Go.
★ 2,284 · GoRUN LLM LOCALLY IN NODE.JS
Run AI models directly on your computer via Node.js. Enforced JSON schema validation right at the generation level.
★ 2,205 · TypeScriptBETA9 - Lightning GPU Output
Ultrafast serverless GPU inference with sandboxes and background jobs, written in Go for instant scaling.
★ 1,664 · GoUCCL: GPU Communication at Max
A fast library for efficient data exchange between GPUs: collective operations, P2P, and inference acceleration.
★ 1,586 · C++BUILD YOUR AI COMPUTER
The project helps you build your personal AI computer with open-source code and support for autonomous agents.
★ 1,577Headless 3D Engine for AI Agents
A Rust-based headless 3D and VFX engine designed for AI agents to simulate, render, and interact with virtual environments without a GUI.
★ 1,565 · Rust🔥 GPU REAL‑TIME DASHBOARD
GPU-hot — a web dashboard for monitoring NVIDIA GPU showing temperature, load, and memory in real time. Already over 1,500 stars on GitHub! 🌟
★ 1,543 · JavaScripttt-metal - Metallic Operations Library
TT-NN operator library and low-level programming model for TT-Metalium kernels to accelerate AI workloads on proprietary hardware.
★ 1,505 · C++TTY7: FUTURE TERMINAL IN RUST
A Rust-based terminal workspace with GPU rendering, persistent sessions, SSH, and AI agent integration for coding.
★ 1,501 · RustEAGLE-4 Speculative Decoding
Official inference code for EAGLE-4, a speculative decoding method that accelerates LLM generation by drafting tokens via feature-level extrapolation.
★ 1,287 · PythonRAFT - Accelerate ML on CUDA
The RAFT library provides CUDA‑accelerated ML and IR algorithms, simplifying the creation of high‑performance applications.
★ 1,054 · CudaAI Agent 24/7
An autonomous AI agent runs your deep experiments 24/7, saving resources and memory. The Leader‑Worker architecture ensures stable operation at zero cost.
★ 1,035 · PythonSiliconScope — Apple Silicon Monitor
Native Apple Silicon system monitor in SwiftUI without sudo: tracks ANE, Media Engine, and memory bandwidth.
★ 1,024 · SwiftWhich local LLM to choose?
Find a local model with the best performance on your hardware based on real benchmarks, not parameter count. One command — instant launch.
★ 947 · PythonMOSEC - MAXIMUM ML PRODUCTIVITY
Fast framework for serving ML models with dynamic batching and CPU/GPU pipelines.
★ 904 · PythonCUVS - Accelerated Vector Search
cuvs is a library for fast vector search and clustering on GPU, using CUDA for maximum performance.
★ 897 · CudaPEGAINFER: Rust + CUDA Infer
Pure Rust LLM engine with CUDA acceleration, OpenAI API compatible, runs Qwen3 and Kimi‑K2 without PyTorch.
★ 742 · RustDEVITO — Computations on Steroids
DSL and compiler for automated finite-difference computations and stencil operations in Python.
★ 719 · PythonTalos worker client
GPU worker client for the Talos network. It connects to your account, receives open‑model inference tasks via WebSocket, and reports uptime for reward calculation.
★ 629 · PythonHorizon — GPU terminal
GPU-accelerated terminal for sessions on an infinite canvas with intuitive control.
★ 607 · RustFASTEST VIDEO ENGINE FOR AGENTS
Kinetograph is a programmatic video engine in TypeScript for AI agents. Uses GPU and FFmpeg to render in seconds. Supports MCP and Hyperframes concept.
★ 520 · TypeScriptAI AGENTS ON AZURE AND GPU
Repository with code, benchmarks, and guides for AI agents and Deep Learning. Deployment on Microsoft Foundry and Azure, fine-tuning, inference optimization, and GPU test
★ 451 · Jupyter NotebookEXPERIMENTAL INFERENCE LIBRARY
resin — experimental Python library for building LLM inference engines. Supports CUDA, GPU acceleration, and models like DeepSeek and GPT-OSS. Early-stage project with 25
★ 432 · PythonStarling: New Linux DE
Starling is a new Linux desktop built with Swift, featuring its own compositor and a framework for porting Flutter apps. It includes a fast shell and a set of branded app
★ 417 · SwiftZINC - LOCAL LLM ON GPU
Zinc is an LLM inference engine in Zig that runs on AMD GPU and Apple Silicon without external dependencies.
★ 395 · ZigOpenLake: GPU Storage
Ultra-efficient storage for GPU workloads. Deliver data at lightning speed.
★ 385 · RustACCELERATE LLM WITH CACHE
The unified-cache-management project enables storing and reusing KV Cache to accelerate large language models.
★ 356 · PythonPRODUCTION AI AGENTS: MEDIA GENERATION
Course teaches building production agents for generating images, video, and audio on serverless GPUs. 254 GitHub stars. Free, apply now.
★ 349GPU TERMINAL FOR CODING
Native macOS terminal with GPU rendering, persistent sessions, and coding-agent workflow support. Enables SSH connections, runs as a daemon and CLI tool on Linux, and opt
★ 309 · SwiftGPU BY TURN WITHOUT CRASHES
Berth leases the GPU to chat, image, and voice engines in turn. Two processes never load together and crash each other. Single Go binary, no dependencies.
★ 297 · GoTURBO LLM: OFFLINE WITH AI
Run any local LLM with automatic GPU tuning, a web UI, and an OpenAI/Anthropic‑compatible API.
★ 296 · TypeScriptJITLLM: HIGH PERFORMANCE LLM ON JVM
JITLLM — Java framework for high‑performance LLM inference and serving on the JVM. Supports GPU acceleration, GGUF and quantized model formats, and integration with DeepS
★ 295 · JavaHYPERLOOM AUTO-OPTIMIZATION LLM ON AMD
Hyperloom — agentic system that automatically optimizes LLM workloads on AMD GPUs by selecting kernels and tuning parameters for peak performance without manual setup.
★ 271 · PythonLLMKUBE: LLM IN KUBERNETES
LLMKube - Kubernetes operator for LLM inference on heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal. Supports llama.cpp, vLLm, TGI, mlx-server, multi
★ 266 · GoHOMELAB MONITOR DASHBOARD
Plug‑and‑play dashboard for homelab in a single container: monitoring GPU, local‑AI VRAM, Docker, systemd and host health. Built‑in read‑only MCP server gives AI agents a
★ 236 · PythonLive Multi-Camera SAM Tracking
Serves multiple cameras with one SAM model per GPU, achieving 4x memory reduction and faster inference than original implementations.
★ 221 · PythonANUBIS: BENCHMARK LLM ON APPLE SILICON
anubis-oss — Swift utility for local LLM launch and benchmarking on Apple Silicon. It measures speed and accuracy of GPU inference, comparing results with CPU baseline an
★ 217 · SwiftGLYD: LOSSLESS AI COMPRESSION
Glyd reduces GPU memory by 33% without losing data, ensuring bit‑for‑bit identity. The project is written in Rust and supports CUDA, C‑ABI, and CLI for convenient benchma
★ 200 · RustLocal Stable Diffusion Reference
Curated notes on samplers, LoRAs, and VRAM optimization for running Stable Diffusion locally.
★ 98Stable Diffusion Local Guide
Curated notes on samplers, LoRA training, and VRAM optimization for running diffusion models locally.
★ 97Can I finetune this?
Tool to check whether a Hugging Face model fits in your GPU memory and can be fine-tuned locally.
★ 31 · PythonNew projects on this topic in Telegram
Telegram →