R_REDDYX.XYZ

Evals: open-source projects on GitHub

20 repositories on Evals, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094AI Agents 1319MCP 1601LLMOps 82Openai 690LLM Evaluation 45Evaluation 46Observability 92Claude Code 1595Codex 849RAG 533Agents 400Anthropic 452Prompt Engineering 175
mastra-ai/mastra

Mastra AI framework

Mastra is a TypeScript framework for building AI applications and agents, created by the Gatsby team.

★ 23,853 · TypeScript
Arize-ai/phoenix

AI observability platform

Platform for monitoring and evaluating AI models: quality monitoring, data drift detection, and interactive visualization.

★ 9,652 · Python
AgentOps-AI/agentops

AGENTOPS: AI AGENT MONITORING

Python SDK for monitoring AI agents, tracking LLM costs, and benchmarking. Support for all popular frameworks.

★ 5,960 · Python
Kiln-AI/Kiln

Build AI Systems

Build, evaluate, and optimize AI systems. Includes evals, RAG, agents, fine‑tuning, synthetic data generation, dataset management, MCP, and more.

★ 4,848 · Python
pydantic/logfire

LOGFIRE: LLM OBSERVABILITY

Platform for observing production LLMs and agentic systems: tracing, metrics, and real-time alerts.

★ 4,290 · Python
FrancyJGLisboa/agent-skills-platform

AI AGENT SKILLS PLATFORM

Platform lets you create, test and publish skills for AI agents via a user‑defined marketplace. Manage their lifecycle: discovery, updates, rollback and quarantine, and a

★ 2,414 · Python
ShenSeanChen/waku-agent

WAKU-AGENT — PERSONAL AI AGENT

A personal AI agent in Python: local launch, memory, task loop and evaluation — everything ready in just a few minutes of reading the code.

★ 2,173 · Python
MCPJam/inspector

Inspector - MCP Server Debug

Platform for testing, inspecting, and debugging MCP servers, MCP apps, and ChatGPT apps. 1968 ⭐

★ 1,968 · TypeScript
LilMGenius/paperthin

PAPERTHIN: SKILLS FOR RECURSIVE RSI

OpenAI hackathon winner's project on recursive agent self-improvement. Unpacks NVIDIA's zero-touch proof into engineering practice. Ready skillset for autonomous coding a

★ 1,445 · JavaScript
Jwuthri/Tracely-ai

TRACELY-AI - AUTOMATED AGENT TESTING

Tracely-ai automatically turns AI agent failures into regression tests that block PRs and reproduces them in CI for free.

★ 1,299 · Python
superlinear-ai/raglite

RAGLITE - FAST RAG ON DUCKDB

Simple tool for building RAG pipelines with DuckDB or PostgreSQL in Python.

★ 1,212 · Python
ombharatiya/ai-system-design-guide

GUIDE TO AI SYSTEMS

A practical guide to designing production-ready AI systems and evaluating their quality for engineers.

★ 1,132
benchflow-ai/awesome-evals

AI Agent Evaluation Toolkit

A curated library of top resources for building and evaluating AI agents, including articles, blogs, tools, and benchmarks.

★ 1,039
future-agi/future-agi

AI Application Evaluation

Open platform for evaluating and improving LLM and AI agents with tracing, simulation, and dataset support. Runs locally.

★ 971 · Python
tripleyak/SkillForge

SKILLFORGE — RELIABLE SKILL GENERATOR

SkillForge creates verified skills for Claude Code and Codex with baseline testing and regression assessment.

★ 918 · Python
overmind-core/overmind

OVERMIND: PLATFORM FOR AI AGENTS

Overmind — platform for the continuous improvement of AI agents: automated evaluation, fine‑tuning and integration with LLM operations. It allows tracking metrics, runnin

★ 759 · Python
calmrocks/ai-engineer-notebooks

AI Engineer Notebooks

A practical collection of framework-free Jupyter notebooks for learning AI engineer skills: model APIs, structured output, tool calling, RAG, evaluation, agents, fine-tun

★ 749 · Jupyter Notebook
osolmaz/switch

Decision Model Harness Framework

Framework for building harnesses that guide models through stepwise decision-making by presenting curated action choices until task completion.

★ 444 · TypeScript
heaven999b/hello-agent-system

Enterprise AI Agent Design Course

17 bilingual lessons teaching enterprise AI agent system design with hands-on exercises covering tools, RAG, security, evals, and distributed execution.

★ 316 · Python
edonadei/caliper

EVALUATE AGENTS WITH CALIPER

Caliper runs your real agent with skills, MCPs, and rules and without them, showing what actually helps and how many tokens it costs. Supports Claude Code, Codex, Pi, and

★ 235 · Python

New projects on this topic in Telegram

Telegram →
Evals — 20 open-source GitHub projects | REDDYX AI