R_REDDYX.XYZ

LLM Observability: open-source projects on GitHub

24 repositories on LLM Observability, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Observability 92LLMOps 82Openai 690LLM Evaluation 45AI Agents 1319Anthropic 452MCP 1601Langchain 196Self Hosted 359Agent Observability 17Evaluation 46Monitoring 52Cost Tracking 18
langfuse/langfuse

Langfuse: LLM Platform

Open platform for LLM: observability, metrics, prompt management, testing, and integration with popular tools.

★ 27,378 · TypeScript
comet-ml/opik

LLM Debugging and Monitoring

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 19,332 · Python
VoltAgent/voltagent

AI Agent Engineering Platform

An AI Agent Engineering Platform built on an open-source TypeScript AI agent framework.

★ 9,150 · TypeScript
mnfst/llm-gateway

CONNECT ANY AGENTS TO LLM

Connect any AI agents and tools to any LLM provider via a unified gateway. Supports Anthropic, OpenAI and other APIs, tracking cost and usage.

★ 7,636 · TypeScript
mnfst/manifest

Smart Model Router

Automatically selects the optimal model for each agent request, reducing inference costs by up to 70%.

★ 6,565 · TypeScript
Helicone/helicone

HELICONE LLM OBSERVE

Open source platform for LLM observability: monitoring, evaluation, and experiments in one line of code. Supports TypeScript and already has 5.8k stars.

★ 5,805 · TypeScript
coze-dev/coze-loop

AI Agent Lifecycle Platform

Next-generation AI Agent Optimization Platform: Cozeloop tackles AI agent development challenges with full lifecycle management from development and debugging to evaluati

★ 5,471 · Go
maximhq/bifrost

Gateway for corporate AI

The fastest AI gateway: 50x faster than LiteLLM, support for 1000+ models, load balancing, cluster mode, limits and overhead under 100 µs at 5k RPS.

★ 5,089 · Go
Agenta-AI/agenta

AGENTA: AI AGENTS IN CHAT

Agenta is an open platform for creating AI agents via chat, sharing them, and running them on schedule or in response to app events.

★ 4,911 · TypeScript
pydantic/logfire

LOGFIRE: LLM OBSERVABILITY

Platform for observing production LLMs and agentic systems: tracing, metrics, and real-time alerts.

★ 4,290 · Python
memodb-io/Acontext

ACONTEXT: MEMORY LAYER FOR AI AGENTS

Acontext creates a memory layer for AI agents, enabling skill persistence and reuse across sessions. The platform provides contextual storage, versioning, and agent execu

★ 3,695 · JavaScript
Jwuthri/Tracely-ai

TRACELY-AI - AUTOMATED AGENT TESTING

Tracely-ai automatically turns AI agent failures into regression tests that block PRs and reproduces them in CI for free.

★ 1,299 · Python
BlazeUp-AI/Observal

Observal - AI Agent Oversight

Observal is a platform for monitoring, evaluating, and debugging AI agents with human-in-the-loop feedback. It is essential for building reliable production pipelines.

★ 1,118 · Python
ThinkWatchProject/ThinkWatch-Lite

LOCAL GATEWAY CLAUDE

ThinkWatch-Lite — local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux. Switches upstreams without touching clients, keeps API keys from

★ 1,070 · TypeScript
JudgmentLabs/judgeval

JUDGEVAL STACK FOR AGENTS

Platform for continuous improvement of AI agents: collecting environment data and evaluations for monitoring and growth.

★ 1,036 · Python
jwuthri-cloud/Tracely-ai

TRACE-NATIVE CI/CD FOR AI AGENTS

Tracely-ai grades production agent traces, clusters failures, and turns them into regression tests. Blocks PRs with behavior regression. Self-hosted on your infrastructur

★ 948 · Python
Abilityai/trinity

TRINITY: PLATFORM FOR AI AGENTS

Trinity is a self‑hosted platform for running AI agents Claude Code, Codex and Gemini. It provides orchestration, versioning and access‑policy management. Licensed under

★ 775 · Python
Continuum-AI-Corp/OrcaReplay

ORCA REPLAY: TIME TRAVEL

OrcaReplay records any launch of an AI agent and allows you to replay it step by step. You can create forks for experiments and debug the state at each stage, using any m

★ 727 · TypeScript
evilmartians/agent-prism

AGENT PRISM: AI AGENT TRACES

React components in TypeScript for interactive visualization of AI agent traces.

★ 408 · TypeScript
Justin0504/Aegis

AEGIS: AI AGENT PROTECTION

Aegis provides runtime policies for AI agents with a cryptographic audit trail, human approval, and a kill‑switch — without modifying code.

★ 361 · TypeScript
taichuy/1flowbase

1FLOWBASE AGENT PLATFORM ON RUST

Open-source full-stack for AI agents: gateways, backend, React blocks, auto-generated CRUD and tracing.

★ 293 · Rust
scasella/claude-flightdeck

LIVE DASHBOARD FOR CLAUDE CODE

claude-flightdeck adds an observation panel to terminal for Claude Code: context and cost, advisor timeline, permission checks, subagent cards and swimlanes. 233 stars on

★ 287 · TypeScript
alibaba/loongsuite-pilot

LOCAL TELEMETRY AI PILOT

loongsuite-pilot collects telemetry from AI coders in local mode, unifying OpenTelemetry events from Claude Code, Codex, Cursor and agents. It tracks token usage, cost, t

★ 253 · TypeScript
QuesmaOrg/awesome-ai-tokenomics

LLM TOKENOMICS: TOOLS TO SAVE

Curated list on AI token economics: costs, waste, and how to cut bills. Tools, benchmarks, papers, and ready configs for LLMs and coding agents. 206 stars on GitHub.

★ 219 · Python

New projects on this topic in Telegram

Telegram →
LLM Observability — 24 open-source GitHub projects | REDDYX AI