R_REDDYX.XYZ
AIPython★ 87,369⑂ 9,116 forksApache-2.0

unclecode/crawl4ai

OPEN SOURCE CRAWLER FOR LLM

Crawl4ai turns any site into clean Markdown ready for RAG and agents. 84k GitHub stars, run locally or via cloud.

// KEY FEATURES

  • Outputs LLM-ready Markdown without noise and ads
  • Supports JS rendering and bypasses protection
  • Integrates with LangChain, LlamaIndex, CrewAI
  • Works via Docker, Python SDK, or Cloud API
#ai#ai-agents#crawler#data-extraction#llm#markdown#mcp#open-source#playwright#python#rag#scraper
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

ONE GATEWAY — 375 AI PROVIDERS★ 87451NextChat — AI Assistant★ 87965Collection of MCP servers★ 86754WorldMonitor: Global Intelligence★ 90701AI Job Search Assistant★ 82638Building LLM from Scratch★ 93387

← FULL CATALOG

unclecode/crawl4ai — Open Source Crawler For LLM | REDDYX AI