Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
pi-research
SkillsWeb research for your agents with smart and safe tooling + knowledge store
git clone https://github.com/Lincoln504/pi-research.git -
trafilatura
AI repositoriesPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
git clone https://github.com/adbar/trafilatura.git -
google-search
MCP serversA Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.
git clone https://github.com/web-agent-master/google-search.git -
CRW Web Scraper
MCP serversOpen-source web scraper for AI agents with scrape, crawl, and map tools
ghcr.io/us/crw:latest -
oc
Agents & frameworksTurn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.
git clone https://github.com/only-cli/oc.git -
xpay Web Scraping
MCP servers35+ web scraping tools. Firecrawl, Bright Data, Jina, Olostep, ScrapeGraph, Notte, Riveter.
-
firecrawl-mcp-server
MCP serversOfficial Firecrawl MCP server, the context API for AI agents: search, scrape, and interact with the web, plus parse, crawl, map, Agent and Monitor, from any MCP client.
git clone https://github.com/firecrawl/firecrawl-mcp-server.git -
WebAI2API
AI repositoriesWebAI2API: 基于 Camoufox 的网页 AI 转 API 工具,支持 LMArena/Gemini等,多窗口并发与账号隔离。 | Web AI to OpenAI API via Camoufox. Supports LMArena/Gemini and more, multi-window concurrency & account isolation.
git clone https://github.com/foxhui/WebAI2API.git -
camoufox-cli
Agents & frameworksAnti-detect browser automation CLI & Skills for AI agents — Camoufox-powered fingerprint spoofing, no bot-detectable Playwright leaks
git clone https://github.com/Bin-Huang/camoufox-cli.git -
wigolo
AI repositoriesThe go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
git clone https://github.com/KnockOutEZ/wigolo.git -
cli
SkillsFirecrawl CLI and agent skills: search, scrape, and interact with the web from your terminal or any coding agent, plus parse, crawl, map, Agent and Monitor. Firecrawl is the context API for AI agents.
git clone https://github.com/firecrawl/cli.git -
agent-skills
SkillsAgent skills that teach AI coding agents to use Decodo for web scraping
git clone https://github.com/Decodo/agent-skills.git -
web-scraping-mcp-server
MCP serversWebsite crawling, Google search and Wikipedia lookup as agent tools.
-
apitap
MCP serversCLI, MCP server, and npm library that turns any website into an API — no docs, no SDK, no browser.
git clone https://github.com/n1byn1kt/apitap.git -
servo-fetch
SkillsA self-contained browser engine that fetches, renders, and extracts web content as Markdown, JSON, or screenshots — no Chromium, no API key, no setup.
git clone https://github.com/konippi/servo-fetch.git -
Aetheris-MCP — x402 Web Scraping MCP Server
MCP serversx402-gated web scraping MCP server. Pay 0.01 USDC per page on Base. Returns clean Markdown via JSDOM
-
MinerU-HTML
AI repositoriesMinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
git clone https://github.com/opendatalab/MinerU-HTML.git -
Ultimate Web Scraper
MCP serversScrape and crawl websites into queryable tables: products, leads, Shopify catalogs, real estate.
-
agentql-mcp
MCP serversModel Context Protocol server that integrates AgentQL's data extraction capabilities.
git clone https://github.com/tinyfish-io/agentql-mcp.git -
GoAI Moat Web Scrape
MCP serversScrape any URL into clean LLM-ready Markdown, or crawl a whole site.
-
web-scraping
MCP serverssuperagnt: marketing plugin for Claude and ChatGPT. Know your market: scrape and search the web.
-
Web Scraper to Markdown API
MCP serversExtract clean markdown from any URL. Removes boilerplate. For RAG pipelines. x402.
-
HasData Web Scraping
MCP serversFetch any public web page through managed proxies, with optional JS rendering and extraction rules.
@hasdata/web-scraping-mcp@1.0.2 -
web-scraper
MCP serversFetch, crawl, and browse protected pages with anti-bot handling - renders in a real browser and
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.