R_REDDYX.XYZ
AIRust★ 272⑂ 37 forksApache-2.0

YOLOW92/pagesift

Rust Web Crawler for AI

High-performance Rust crawler extracting clean Markdown and structured data for AI agents and RAG pipelines. Supports PDF/RSS parsing, browser rendering, and Firecrawl-compatible APIs.

// KEY FEATURES

  • Crawls adaptively with browser rendering for JavaScript-heavy sites
  • Outputs clean Markdown and structured JSON for RAG ingestion
  • Provides Firecrawl-compatible API with Python and Node.js SDKs
#ai-agents#browser-automation#data-extraction#firecrawl#html-to-markdown#markdown#mcp#pdf-extraction#rag#rust#web-crawler#web-scraping
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Aegis: AI Agent Architecture Control★ 272Markra — AI Markdown editor★ 272Google Drive Downloader for Colab★ 272FREECAD AI: Fast 3D Modeling★ 272BEST LLM OPS LIST★ 272KOTLIN AND AI: PRACTICAL EXAMPLES★ 272

← FULL CATALOG

YOLOW92/pagesift — Rust Web Crawler for AI | REDDYX AI