kestra-io/kestra
WORKFLOW ORCHESTRATOR FOR DATA AND AI
Open-source orchestration platform for data, AI, and infrastructure. Unifies scheduled and event-driven workflows through declarative YAML with Git versioning and AI Copilot.
// KEY FEATURES
- Declarative YAML with three editors: code, no-code, AI Copilot
- AI agents and RAG as built-in task types, MCP server
- Event-driven and scheduled triggers, scale to millions of executions
- Git versioning, plugins for any languages and systems
// DEEP DIVE: HOW IT WORKS
Kestra works as a control plane: the orchestrator accepts declarative flow definitions in YAML, distributes tasks to workers via gRPC, and persists execution state. Workers can reside in other networks or regions, communicating only with the control plane. Flows are triggered by schedules or events, support subflows, retries, timeouts, quotas, and conditional branching.
It stands out with three synchronized editing modes: YAML editor, visual no-code, and AI Copilot in a persistent sidebar (Ask/Edit/Plan). AI-native capabilities are implemented via built-in task types for agents and RAG, plus an MCP server exposing flows as tools to compatible clients. The ecosystem includes hundreds of plugins for databases, cloud storage, APIs, and running scripts in any language.
Requires Java runtime for the server component. HA and fault tolerance need a clustered deployment. The built-in versioning system allows saving drafts without affecting production revisions. Git and CI/CD integration is configured via UI. Hardware requirements are not specified in README.
// USE CASES
Best for: Data and platform engineers for orchestrating ETL, AI agents, and infrastructure
- Orchestrate ETL pipelines with Kafka events and schedules in a single flow
- Run AI agents and RAG chains as tasks with export via MCP server
- Manage infrastructure as code: deployments, migrations, checks via Git-versioned YAML
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram