AIPython★ 87,369⑂ 9,116 forksApache-2.0
unclecode/crawl4ai
OPEN SOURCE CRAWLER FOR LLM
Crawl4ai turns any site into clean Markdown ready for RAG and agents. 84k GitHub stars, run locally or via cloud.
// KEY FEATURES
- Outputs LLM-ready Markdown without noise and ads
- Supports JS rendering and bypasses protection
- Integrates with LangChain, LlamaIndex, CrewAI
- Works via Docker, Python SDK, or Cloud API
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram