Scraping: open-source projects on GitHub
15 repositories on Scraping, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
Firecrawl Web API
Firecrawl provides a simple API for searching, scraping, and interacting with web pages, optimized for use in AI applications.
★ 118,560 · TypeScriptAdaptive web scraper
An adaptive framework for web scraping that supports both single requests and large-scale crawlers.
★ 49,091 · PythonScrapegraph: AI scraper on Python
Library for web scraping using artificial intelligence. Automatically extracts structured data from any web page.
★ 25,439 · PythonTRAFILATURA - FAST WEB PARSE
Trafilatura extracts clean text and metadata from pages, supporting CSV, JSON, HTML and other formats.
★ 6,055 · PythonCamoFox: Stealth Browser
Stealth headless browser for AI agents, bypassing Cloudflare, bot protection, and anti‑scraping. Fully compatible with Puppeteer and Playwright.
★ 4,734 · JavaScriptOpenOutreach - AI Lead Search
OpenOutreach automates finding and connecting with potential clients on LinkedIn using AI.
★ 3,720 · PythonANY SITE TO LLM DATA
AnyCrawl — a Node.js crawler that turns websites into LLM‑ready data and extracts structured search results from Google, Bing, Baidu, and others.
★ 3,547 · MDXSPIDER: LIGHTNING-FAST WEB CRAWLER
High-speed web data streaming with minimal latency. Ideal for real-time parsing and data collection.
★ 2,849 · RustBRIGHTDATA-MCP: SINGLE SOLUTION FOR WEB
Model Context Protocol server for accessing public web data without complex setup.
★ 2,434 · JavaScriptTHEPIPE: DATA FROM DOCUMENTS FAST
ThePipe uses vision‑language models to extract data from PDF, Word and PowerPoint. It recognises tables, text and images, turning them into ready‑to‑use structures. Suppo
★ 1,530 · PythonCHATGPT-SCRAPER - INTELLIGENT SCRAPER
Get structured responses from ChatGPT scraping API in one line. Enable live search, HTML context, and automate tasks.
★ 1,411LLM READER: WEB TO TEXT
llm-reader turns web pages into LLM-friendly text. Alternative to Firecrawl and Jina Reader API. Simplifies RAG, AI scraping, and link extraction.
★ 316 · PythonSLM EXTRACTOR FOR MAIN HTML CONTENT
MinerU-HTML uses a small language model to extract main content from HTML pages. Outputs clean HTML without clutter. Ideal for RAG, deep research agents, and training dat
★ 302 · PythonSTEALTH CHROMIUM BROWSER CLEARCOTE
clearcote-browser — open-source stealth Chromium with engine-level fingerprint spoofing. It is fully de-Googled, Playwright-compatible as a drop-in replacement and built
★ 263 · Pythonhh-ai-agent - Automate HH.ru Responses
A Python bot automates responses to hh.ru vacancies, using AI to select and send personalized answers.
★ 261 · PythonNew projects on this topic in Telegram
Telegram →