LLMOps: open-source projects on GitHub
82 repositories on LLMOps, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
Ready LLM templates
Ready cloud templates for RAG, AI‑pipeline, and search with live data, Docker‑compatible and synchronized with SharePoint, S3, Kafka, and PostgreSQL.
★ 59,750 · Jupyter NotebookUnified LLM Gateway
Python SDK and proxy server for invoking 100+ LLMs via an OpenAI‑compatible API, with cost tracking, security, load balancing, and logging.
★ 46,602 · PythonComposio for AI agents
Composio provides 1000+ toolkits, search, context management, authentication, and a sandbox for building AI agents that turn intent into action.
★ 28,202 · TypeScriptLangfuse: LLM Platform
Open platform for LLM: observability, metrics, prompt management, testing, and integration with popular tools.
★ 27,378 · TypeScriptMLflow — AI Platform
Open platform for AI engineering. Allows teams to debug, evaluate, and optimize AI applications while managing access and costs.
★ 26,029 · PythonBEST AI ENGINEERING RESOURCES
Collection with 22k+ stars: roadmaps, tools, and projects for LLMs, RAG, agents, and production. Everything to start and grow as an AI engineer in one place.
★ 22,566Testing LLM Models
Tool for testing prompts, agents, and RAG systems. Supports comparative analysis of GPT, Claude, Gemini, Llama, and other models.
★ 21,294 · TypeScriptLLM Debugging and Monitoring
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
★ 19,332 · PythonLLM EVALUATION RAG
Ragas is a framework for LLM application evaluation. It automates quality checks and tracks key metrics.
★ 16,320 · PythonOPENLLM: RUN LLMS AS OPENAI API
OpenLLM allows running any open-source LLMs (Llama, Qwen, DeepSeek) as OpenAI-compatible APIs with a single command. Includes chat UI, modern inference backends, and read
★ 12,607 · PythonPortkey-AI gateway
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
★ 11,847 · TypeScriptBISHENG — LLM DevOps Platform
An open platform for building next‑generation enterprise AI applications, featuring GenAI workflows, RAG, agents, and more.
★ 11,373 · TypeScriptTensorZero: LLMOps Platform
An open LLMOps platform that unifies LLM gateway, observability, evaluation, optimization, and experiments.
★ 11,366 · RustSoup: Train LLM Easily
Soup fine-tunes an LLM from a single YAML using a streaming layer — enabling an 8B model to run on a laptop with 4 GB VRAM.
★ 10,552 · PythonMETAFLOW — ML MANAGEMENT
Metaflow simplifies building, managing, and deploying AI/ML systems on Python, allowing focus on data science.
★ 10,114 · PythonAI observability platform
Platform for monitoring and evaluating AI models: quality monitoring, data drift detection, and interactive visualization.
★ 9,652 · PythonBENTOML PATH TO AI API
Serve models and AI applications in minutes: API, queues, multi‑model pipelines, and LLM services.
★ 8,663 · PythonModular LLM Apps in Rust
⚙️🦀 Build modular and scalable LLM applications in Rust
★ 7,409 · RustOPENLLMETRY LLM MONITORING
Open-source tool for GenAI and LLM application observability based on OpenTelemetry. 7.1k stars!
★ 7,158 · PythonPLANO: AI PROXY FOR AGENTS
AI-native proxy and data plane for agent applications with orchestration and security.
★ 6,567 · RustHELICONE LLM OBSERVE
Open source platform for LLM observability: monitoring, evaluation, and experiments in one line of code. Supports TypeScript and already has 5.8k stars.
★ 5,805 · TypeScriptZENML: AI PLATFORM FROM PIPELINE
ZenML is a Python framework for creating AI pipelines and agents, unifying data, models, and orchestration in a single platform.
★ 5,636 · PythonAI Agent Lifecycle Platform
Next-generation AI Agent Optimization Platform: Cozeloop tackles AI agent development challenges with full lifecycle management from development and debugging to evaluati
★ 5,471 · GoGISKARD-OSS - LLM AGENT EVALUATION
Library for evaluating and testing LLM agents with open-source code.
★ 5,417 · PythonGateway for corporate AI
The fastest AI gateway: 50x faster than LiteLLM, support for 1000+ models, load balancing, cluster mode, limits and overhead under 100 µs at 5k RPS.
★ 5,089 · GoACONTEXT: MEMORY LAYER FOR AI AGENTS
Acontext creates a memory layer for AI agents, enabling skill persistence and reuse across sessions. The platform provides contextual storage, versioning, and agent execu
★ 3,695 · JavaScriptPezzo LLMOps for Developers
Pezzo is an open-source LLMOps platform built with TypeScript: prompt management, collaboration, and observability.
★ 3,283 · TypeScriptLANGWATCH — TEST YOUR LLM
Platform for evaluating LLMs and testing AI agents. Convenient tools for debugging and monitoring AI applications.
★ 3,274 · TypeScriptAGiXT: AI Agent Automation
A platform for orchestrating AI agents with adaptive memory, plugins, and support for multiple AI providers.
★ 3,192 · PythonHAMILTON - MODULAR DATAFLOW
Apache Hamilton simplifies creating testable, self-documenting data pipelines with lineage tracking.
★ 2,512 · Jupyter NotebookOPENLIT AI FUTURE PLATFORM
Open source platform for AI Engineering: OpenTelemetry LLM, GPU monitoring, guardrails and prompt management. Integration with 50+ providers.
★ 2,509 · TypeScriptWAKU-AGENT — PERSONAL AI AGENT
A personal AI agent in Python: local launch, memory, task loop and evaluation — everything ready in just a few minutes of reading the code.
★ 2,173 · PythonVLLM-ASCEND: ASCEND PLUGIN ON GITHUB
Community plugin for vLLM on Ascend hardware from the community, written in C++. Gaining stars — already 2156!
★ 2,156 · C++UNIFIED MCP SERVER MANAGER
Unified application for managing MCP servers. Start, configure, and orchestrate everything from one place.
★ 2,127 · TypeScriptPADDLER - Simple LLM Load Balancer
Open load balancer and platform for LLM/VLM service on Rust, running on CPU and GPU with minimal dependencies.
★ 1,693 · RustAGENTLY: SIMPLIFYING GEN-AI DEVELOPMENT
Agently — a Python framework for rapid creation of GenAI applications: simple code, event-driven flow, and easy model swapping.
★ 1,584 · PythonRTP-LLM ENHANCED LLM INFERENCE
High-performance LLM inference engine from Alibaba, written in CUDA, enables fast model operation across diverse applications.
★ 1,400 · CudaTRACELY-AI - AUTOMATED AGENT TESTING
Tracely-ai automatically turns AI agent failures into regression tests that block PRs and reproduces them in CI for free.
★ 1,299 · PythonAWESOME AI SDKS – COLLECTION
A database of SDKs, frameworks, and tools for building, debugging, and deploying autonomous AI agents. It simplifies selection and speeds up development.
★ 1,241INTELLIGENT AUDIT SYSTEM: AI AUDIT
Enterprise audit agent workspace with Agentic RAG, controlled tool usage, and human‑review workflow.
★ 1,182 · PythonObserval - AI Agent Oversight
Observal is a platform for monitoring, evaluating, and debugging AI agents with human-in-the-loop feedback. It is essential for building reliable production pipelines.
★ 1,118 · PythonDYNAMIQ: AI APPLICATION ORCHESTRATION
Dynamiq is a framework for orchestrating agentic AI and LLM applications in Python. A simple way to create scalable AI workflows.
★ 1,052 · PythonOpenInference - Real-Time AI Tracing
OpenInference adds OpenTelemetry instrumentation for observing AI model operations, simplifying debugging and monitoring of LLM agents.
★ 1,015 · PythonTRACE-NATIVE CI/CD FOR AI AGENTS
Tracely-ai grades production agent traces, clusters failures, and turns them into regression tests. Blocks PRs with behavior regression. Self-hosted on your infrastructur
★ 948 · PythonLLM TUTORIALS IN PYTHON
Collection of notebooks for learning LLM: LangChain, LlamaIndex, OpenAI. Ready scripts for RAG, ChromaDB and Pinecone vector stores. 933 stars on GitHub.
★ 933 · Jupyter NotebookOTARI: GATEWAY FOR 40+ PROVIDERS
Self-hosted gateway compatible with OpenAI API, unifying over 40 LLM providers with virtual keys, budgets, and usage analytics.
★ 930 · PythonONE GATEWAY FOR ALL LLMS
respan-gateway — a single endpoint for all LLM providers. Supports any SDK, any model, and tools. One Rust binary replaces dozens of integrations.
★ 833 · RustOpenAgent: AI Agent Platform
Platform for creating AI agents with support for multiple models, knowledge bases, and workflow automation. One-click deployment via Docker.
★ 812 · PythonRust + LLM: Indexing Speed
Swiftide — fast streaming indexing and queries for AI agents on Rust. 710⭐ and growing.
★ 806 · RustTRINITY: PLATFORM FOR AI AGENTS
Trinity is a self‑hosted platform for running AI agents Claude Code, Codex and Gemini. It provides orchestration, versioning and access‑policy management. Licensed under
★ 775 · PythonAGENTACC: Agent Report
The panel shows AI agent actions: tools, modified files, launched tests, time and tokens. Works locally, without registration or telemetry.
★ 759 · PythonOVERMIND: PLATFORM FOR AI AGENTS
Overmind — platform for the continuous improvement of AI agents: automated evaluation, fine‑tuning and integration with LLM operations. It allows tracking metrics, runnin
★ 759 · PythonAI Engineer Notebooks
A practical collection of framework-free Jupyter notebooks for learning AI engineer skills: model APIs, structured output, tool calling, RAG, evaluation, agents, fine-tun
★ 749 · Jupyter NotebookAI CHAT IN RESUME
An interactive resume built with React 19 and TypeScript featuring a chat to talk with an AI avatar about my experience.
★ 660 · HTMLOS PLATFORM AI AGENTS
os — framework for creating AI agents: LLM support, real-time chat and voice, document training, analytics, user management, SSO and widgets. Built with React 19, Redux T
★ 638 · TypeScriptGATEWAY: UNIFIED SDK FOR LLM
Gateway is a local production‑grade Super SDK that unifies over 200 LLMs into a single interface. Use any model without complex setup.
★ 609 · TypeScriptVECTORHUB - DEPRECATED REPOSITORY FOR EMBEDDINGS
The archived VectorHub project showcases early work on vector search. The team is now focused on SIE — a self-hosted engine for embeddings, ranking, and document processi
★ 535 · Jupyter NotebookCONTINUOUS-EVAL - LLM EVALUATION ON DATA
Platform for automatic evaluation of LLM applications based on real data. Enables measuring quality and improving models without manual effort.
★ 519 · PythonAgency: Minimal Framework for Agents
Fast and minimalistic framework in Python for building agent systems. 486 stars on GitHub.
★ 491 · PythonLYNKR SIMPLIFYING CODE WORK
Lynkr is a CLI utility that acts as an HTTP proxy for fast interaction with Claude Code and streamlines development workflows.
★ 445 · JavaScriptNew projects on this topic in Telegram
Telegram →