R_REDDYX.XYZ

LLMOps: open-source projects on GitHub

82 repositories on LLMOps, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Openai 690MCP 1601Observability 92RAG 533Langchain 196AI Agents 1319Anthropic 452Prompt Engineering 175Generative AI 192Machine Learning 190LLM Evaluation 45AI Gateway 47Agents 400
pathwaycom/llm-app

Ready LLM templates

Ready cloud templates for RAG, AI‑pipeline, and search with live data, Docker‑compatible and synchronized with SharePoint, S3, Kafka, and PostgreSQL.

★ 59,750 · Jupyter Notebook
BerriAI/litellm

Unified LLM Gateway

Python SDK and proxy server for invoking 100+ LLMs via an OpenAI‑compatible API, with cost tracking, security, load balancing, and logging.

★ 46,602 · Python
ComposioHQ/composio

Composio for AI agents

Composio provides 1000+ toolkits, search, context management, authentication, and a sandbox for building AI agents that turn intent into action.

★ 28,202 · TypeScript
langfuse/langfuse

Langfuse: LLM Platform

Open platform for LLM: observability, metrics, prompt management, testing, and integration with popular tools.

★ 27,378 · TypeScript
mlflow/mlflow

MLflow — AI Platform

Open platform for AI engineering. Allows teams to debug, evaluate, and optimize AI applications while managing access and costs.

★ 26,029 · Python
dipakkr/awesome-ai-engineering

BEST AI ENGINEERING RESOURCES

Collection with 22k+ stars: roadmaps, tools, and projects for LLMs, RAG, agents, and production. Everything to start and grow as an AI engineer in one place.

★ 22,566
promptfoo/promptfoo

Testing LLM Models

Tool for testing prompts, agents, and RAG systems. Supports comparative analysis of GPT, Claude, Gemini, Llama, and other models.

★ 21,294 · TypeScript
comet-ml/opik

LLM Debugging and Monitoring

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 19,332 · Python
vibrantlabsai/ragas

LLM EVALUATION RAG

Ragas is a framework for LLM application evaluation. It automates quality checks and tracks key metrics.

★ 16,320 · Python
bentoml/OpenLLM

OPENLLM: RUN LLMS AS OPENAI API

OpenLLM allows running any open-source LLMs (Llama, Qwen, DeepSeek) as OpenAI-compatible APIs with a single command. Includes chat UI, modern inference backends, and read

★ 12,607 · Python
Portkey-AI/gateway

Portkey-AI gateway

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

★ 11,847 · TypeScript
dataelement/bisheng

BISHENG — LLM DevOps Platform

An open platform for building next‑generation enterprise AI applications, featuring GenAI workflows, RAG, agents, and more.

★ 11,373 · TypeScript
tensorzero/tensorzero

TensorZero: LLMOps Platform

An open LLMOps platform that unifies LLM gateway, observability, evaluation, optimization, and experiments.

★ 11,366 · Rust
MakazhanAlpamys/Soup

Soup: Train LLM Easily

Soup fine-tunes an LLM from a single YAML using a streaming layer — enabling an 8B model to run on a laptop with 4 GB VRAM.

★ 10,552 · Python
Netflix/metaflow

METAFLOW — ML MANAGEMENT

Metaflow simplifies building, managing, and deploying AI/ML systems on Python, allowing focus on data science.

★ 10,114 · Python
Arize-ai/phoenix

AI observability platform

Platform for monitoring and evaluating AI models: quality monitoring, data drift detection, and interactive visualization.

★ 9,652 · Python
bentoml/BentoML

BENTOML PATH TO AI API

Serve models and AI applications in minutes: API, queues, multi‑model pipelines, and LLM services.

★ 8,663 · Python
0xPlaygrounds/rig

Modular LLM Apps in Rust

⚙️🦀 Build modular and scalable LLM applications in Rust

★ 7,409 · Rust
traceloop/openllmetry

OPENLLMETRY LLM MONITORING

Open-source tool for GenAI and LLM application observability based on OpenTelemetry. 7.1k stars!

★ 7,158 · Python
katanemo/plano

PLANO: AI PROXY FOR AGENTS

AI-native proxy and data plane for agent applications with orchestration and security.

★ 6,567 · Rust
Helicone/helicone

HELICONE LLM OBSERVE

Open source platform for LLM observability: monitoring, evaluation, and experiments in one line of code. Supports TypeScript and already has 5.8k stars.

★ 5,805 · TypeScript
zenml-io/zenml

ZENML: AI PLATFORM FROM PIPELINE

ZenML is a Python framework for creating AI pipelines and agents, unifying data, models, and orchestration in a single platform.

★ 5,636 · Python
coze-dev/coze-loop

AI Agent Lifecycle Platform

Next-generation AI Agent Optimization Platform: Cozeloop tackles AI agent development challenges with full lifecycle management from development and debugging to evaluati

★ 5,471 · Go
Giskard-AI/giskard-oss

GISKARD-OSS - LLM AGENT EVALUATION

Library for evaluating and testing LLM agents with open-source code.

★ 5,417 · Python
maximhq/bifrost

Gateway for corporate AI

The fastest AI gateway: 50x faster than LiteLLM, support for 1000+ models, load balancing, cluster mode, limits and overhead under 100 µs at 5k RPS.

★ 5,089 · Go
memodb-io/Acontext

ACONTEXT: MEMORY LAYER FOR AI AGENTS

Acontext creates a memory layer for AI agents, enabling skill persistence and reuse across sessions. The platform provides contextual storage, versioning, and agent execu

★ 3,695 · JavaScript
pezzolabs/pezzo

Pezzo LLMOps for Developers

Pezzo is an open-source LLMOps platform built with TypeScript: prompt management, collaboration, and observability.

★ 3,283 · TypeScript
langwatch/langwatch

LANGWATCH — TEST YOUR LLM

Platform for evaluating LLMs and testing AI agents. Convenient tools for debugging and monitoring AI applications.

★ 3,274 · TypeScript
Josh-XT/AGiXT

AGiXT: AI Agent Automation

A platform for orchestrating AI agents with adaptive memory, plugins, and support for multiple AI providers.

★ 3,192 · Python
apache/hamilton

HAMILTON - MODULAR DATAFLOW

Apache Hamilton simplifies creating testable, self-documenting data pipelines with lineage tracking.

★ 2,512 · Jupyter Notebook
openlit/openlit

OPENLIT AI FUTURE PLATFORM

Open source platform for AI Engineering: OpenTelemetry LLM, GPU monitoring, guardrails and prompt management. Integration with 50+ providers.

★ 2,509 · TypeScript
ShenSeanChen/waku-agent

WAKU-AGENT — PERSONAL AI AGENT

A personal AI agent in Python: local launch, memory, task loop and evaluation — everything ready in just a few minutes of reading the code.

★ 2,173 · Python
vllm-project/vllm-ascend

VLLM-ASCEND: ASCEND PLUGIN ON GITHUB

Community plugin for vLLM on Ascend hardware from the community, written in C++. Gaining stars — already 2156!

★ 2,156 · C++
mcp-router/mcp-router

UNIFIED MCP SERVER MANAGER

Unified application for managing MCP servers. Start, configure, and orchestrate everything from one place.

★ 2,127 · TypeScript
intentee/paddler

PADDLER - Simple LLM Load Balancer

Open load balancer and platform for LLM/VLM service on Rust, running on CPU and GPU with minimal dependencies.

★ 1,693 · Rust
AgentEra/Agently

AGENTLY: SIMPLIFYING GEN-AI DEVELOPMENT

Agently — a Python framework for rapid creation of GenAI applications: simple code, event-driven flow, and easy model swapping.

★ 1,584 · Python
alibaba/rtp-llm

RTP-LLM ENHANCED LLM INFERENCE

High-performance LLM inference engine from Alibaba, written in CUDA, enables fast model operation across diverse applications.

★ 1,400 · Cuda
Jwuthri/Tracely-ai

TRACELY-AI - AUTOMATED AGENT TESTING

Tracely-ai automatically turns AI agent failures into regression tests that block PRs and reproduces them in CI for free.

★ 1,299 · Python
e2b-dev/awesome-ai-sdks

AWESOME AI SDKS – COLLECTION

A database of SDKs, frameworks, and tools for building, debugging, and deploying autonomous AI agents. It simplifies selection and speeds up development.

★ 1,241
Ricky-7-Yan/intelligent-audit-system

INTELLIGENT AUDIT SYSTEM: AI AUDIT

Enterprise audit agent workspace with Agentic RAG, controlled tool usage, and human‑review workflow.

★ 1,182 · Python
BlazeUp-AI/Observal

Observal - AI Agent Oversight

Observal is a platform for monitoring, evaluating, and debugging AI agents with human-in-the-loop feedback. It is essential for building reliable production pipelines.

★ 1,118 · Python
dynamiq-ai/dynamiq

DYNAMIQ: AI APPLICATION ORCHESTRATION

Dynamiq is a framework for orchestrating agentic AI and LLM applications in Python. A simple way to create scalable AI workflows.

★ 1,052 · Python
Arize-ai/openinference

OpenInference - Real-Time AI Tracing

OpenInference adds OpenTelemetry instrumentation for observing AI model operations, simplifying debugging and monitoring of LLM agents.

★ 1,015 · Python
jwuthri-cloud/Tracely-ai

TRACE-NATIVE CI/CD FOR AI AGENTS

Tracely-ai grades production agent traces, clusters failures, and turns them into regression tests. Blocks PRs with behavior regression. Self-hosted on your infrastructur

★ 948 · Python
onlyphantom/llm-python

LLM TUTORIALS IN PYTHON

Collection of notebooks for learning LLM: LangChain, LlamaIndex, OpenAI. Ready scripts for RAG, ChromaDB and Pinecone vector stores. 933 stars on GitHub.

★ 933 · Jupyter Notebook
mozilla-ai/otari

OTARI: GATEWAY FOR 40+ PROVIDERS

Self-hosted gateway compatible with OpenAI API, unifying over 40 LLM providers with virtual keys, budgets, and usage analytics.

★ 930 · Python
respanai/respan-gateway

ONE GATEWAY FOR ALL LLMS

respan-gateway — a single endpoint for all LLM providers. Supports any SDK, any model, and tools. One Rust binary replaces dozens of integrations.

★ 833 · Rust
Haohao-end/openagent

OpenAgent: AI Agent Platform

Platform for creating AI agents with support for multiple models, knowledge bases, and workflow automation. One-click deployment via Docker.

★ 812 · Python
bosun-ai/swiftide

Rust + LLM: Indexing Speed

Swiftide — fast streaming indexing and queries for AI agents on Rust. 710⭐ and growing.

★ 806 · Rust
Abilityai/trinity

TRINITY: PLATFORM FOR AI AGENTS

Trinity is a self‑hosted platform for running AI agents Claude Code, Codex and Gemini. It provides orchestration, versioning and access‑policy management. Licensed under

★ 775 · Python
mikehasa/agentacct

AGENTACC: Agent Report

The panel shows AI agent actions: tools, modified files, launched tests, time and tokens. Works locally, without registration or telemetry.

★ 759 · Python
overmind-core/overmind

OVERMIND: PLATFORM FOR AI AGENTS

Overmind — platform for the continuous improvement of AI agents: automated evaluation, fine‑tuning and integration with LLM operations. It allows tracking metrics, runnin

★ 759 · Python
calmrocks/ai-engineer-notebooks

AI Engineer Notebooks

A practical collection of framework-free Jupyter notebooks for learning AI engineer skills: model APIs, structured output, tool calling, RAG, evaluation, agents, fine-tun

★ 749 · Jupyter Notebook
santifer/cv-santiago

AI CHAT IN RESUME

An interactive resume built with React 19 and TypeScript featuring a chat to talk with an AI avatar about my experience.

★ 660 · HTML
iblai/os

OS PLATFORM AI AGENTS

os — framework for creating AI agents: LLM support, real-time chat and voice, document training, analytics, user management, SSO and widgets. Built with React 19, Redux T

★ 638 · TypeScript
adaline/gateway

GATEWAY: UNIFIED SDK FOR LLM

Gateway is a local production‑grade Super SDK that unifies over 200 LLMs into a single interface. Use any model without complex setup.

★ 609 · TypeScript
superlinked/VectorHub

VECTORHUB - DEPRECATED REPOSITORY FOR EMBEDDINGS

The archived VectorHub project showcases early work on vector search. The team is now focused on SIE — a self-hosted engine for embeddings, ranking, and document processi

★ 535 · Jupyter Notebook
relari-ai/continuous-eval

CONTINUOUS-EVAL - LLM EVALUATION ON DATA

Platform for automatic evaluation of LLM applications based on real data. Enables measuring quality and improving models without manual effort.

★ 519 · Python
operand/agency

Agency: Minimal Framework for Agents

Fast and minimalistic framework in Python for building agent systems. 486 stars on GitHub.

★ 491 · Python
Fast-Editor/Lynkr

LYNKR SIMPLIFYING CODE WORK

Lynkr is a CLI utility that acts as an HTTP proxy for fast interaction with Claude Code and streamlines development workflows.

★ 445 · JavaScript

New projects on this topic in Telegram

Telegram →
LLMOps — 82 open-source GitHub projects | REDDYX AI