AIRust★ 1,060
fastcrw/crw
FAST ALTERNATIVE FIRECRAWL IN RUST
crw — fast Firecrawl/Tavily alternative in Rust: web scraper, crawler and search API with MCP server for AI agents. Compatible API (/scrape, /crawl, /search), runs 2.3× faster than Tavily and 1.5× faster than Firecrawl on 1 000 URL, uses 6 MB RAM, binary, hosting.
// KEY FEATURES
- Single‑page scrape: /scrape endpoint returns clean markdown/HTML in milliseconds.
- Recursive crawler: /crawl traverses a site to a set depth, obeying robots.txt and sitemap.
- Search API: /search returns ranked results with snippets, compatible with Tavily.
- MCP server for AI agents: provides scrape, crawl, search tools via standard MCP.
#ai#ai-agents#crawler#data-extraction#docker#firecrawl-alternative#html-to-markdown#llm#markdown#mcp#mcp-server#rust
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram