LLM Observability: open-source projects on GitHub
24 repositories on LLM Observability, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
Langfuse: LLM Platform
Open platform for LLM: observability, metrics, prompt management, testing, and integration with popular tools.
★ 27,378 · TypeScriptLLM Debugging and Monitoring
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
★ 19,332 · PythonAI Agent Engineering Platform
An AI Agent Engineering Platform built on an open-source TypeScript AI agent framework.
★ 9,150 · TypeScriptCONNECT ANY AGENTS TO LLM
Connect any AI agents and tools to any LLM provider via a unified gateway. Supports Anthropic, OpenAI and other APIs, tracking cost and usage.
★ 7,636 · TypeScriptSmart Model Router
Automatically selects the optimal model for each agent request, reducing inference costs by up to 70%.
★ 6,565 · TypeScriptHELICONE LLM OBSERVE
Open source platform for LLM observability: monitoring, evaluation, and experiments in one line of code. Supports TypeScript and already has 5.8k stars.
★ 5,805 · TypeScriptAI Agent Lifecycle Platform
Next-generation AI Agent Optimization Platform: Cozeloop tackles AI agent development challenges with full lifecycle management from development and debugging to evaluati
★ 5,471 · GoGateway for corporate AI
The fastest AI gateway: 50x faster than LiteLLM, support for 1000+ models, load balancing, cluster mode, limits and overhead under 100 µs at 5k RPS.
★ 5,089 · GoAGENTA: AI AGENTS IN CHAT
Agenta is an open platform for creating AI agents via chat, sharing them, and running them on schedule or in response to app events.
★ 4,911 · TypeScriptLOGFIRE: LLM OBSERVABILITY
Platform for observing production LLMs and agentic systems: tracing, metrics, and real-time alerts.
★ 4,290 · PythonACONTEXT: MEMORY LAYER FOR AI AGENTS
Acontext creates a memory layer for AI agents, enabling skill persistence and reuse across sessions. The platform provides contextual storage, versioning, and agent execu
★ 3,695 · JavaScriptTRACELY-AI - AUTOMATED AGENT TESTING
Tracely-ai automatically turns AI agent failures into regression tests that block PRs and reproduces them in CI for free.
★ 1,299 · PythonObserval - AI Agent Oversight
Observal is a platform for monitoring, evaluating, and debugging AI agents with human-in-the-loop feedback. It is essential for building reliable production pipelines.
★ 1,118 · PythonLOCAL GATEWAY CLAUDE
ThinkWatch-Lite — local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux. Switches upstreams without touching clients, keeps API keys from
★ 1,070 · TypeScriptJUDGEVAL STACK FOR AGENTS
Platform for continuous improvement of AI agents: collecting environment data and evaluations for monitoring and growth.
★ 1,036 · PythonTRACE-NATIVE CI/CD FOR AI AGENTS
Tracely-ai grades production agent traces, clusters failures, and turns them into regression tests. Blocks PRs with behavior regression. Self-hosted on your infrastructur
★ 948 · PythonTRINITY: PLATFORM FOR AI AGENTS
Trinity is a self‑hosted platform for running AI agents Claude Code, Codex and Gemini. It provides orchestration, versioning and access‑policy management. Licensed under
★ 775 · PythonORCA REPLAY: TIME TRAVEL
OrcaReplay records any launch of an AI agent and allows you to replay it step by step. You can create forks for experiments and debug the state at each stage, using any m
★ 727 · TypeScriptAGENT PRISM: AI AGENT TRACES
React components in TypeScript for interactive visualization of AI agent traces.
★ 408 · TypeScriptAEGIS: AI AGENT PROTECTION
Aegis provides runtime policies for AI agents with a cryptographic audit trail, human approval, and a kill‑switch — without modifying code.
★ 361 · TypeScript1FLOWBASE AGENT PLATFORM ON RUST
Open-source full-stack for AI agents: gateways, backend, React blocks, auto-generated CRUD and tracing.
★ 293 · RustLIVE DASHBOARD FOR CLAUDE CODE
claude-flightdeck adds an observation panel to terminal for Claude Code: context and cost, advisor timeline, permission checks, subagent cards and swimlanes. 233 stars on
★ 287 · TypeScriptLOCAL TELEMETRY AI PILOT
loongsuite-pilot collects telemetry from AI coders in local mode, unifying OpenTelemetry events from Claude Code, Codex, Cursor and agents. It tracks token usage, cost, t
★ 253 · TypeScriptLLM TOKENOMICS: TOOLS TO SAVE
Curated list on AI token economics: costs, waste, and how to cut bills. Tools, benchmarks, papers, and ready configs for LLMs and coding agents. 206 stars on GitHub.
★ 219 · PythonNew projects on this topic in Telegram
Telegram →