InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay

    Stale Official MCP Registry, io.github.kriton-dev Updated 14 days ago 1 note

  2. ketch

    Agents & frameworks

    Fast, stateless CLI for web search and scrape. Built for AI agents.

    Indexed GitHub, 1broseidon 748 ★MITUpdated 3 days ago

    git clone https://github.com/1broseidon/ketch.git
  3. nopecha-extension

    Agents & frameworks

    Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.

    Stale GitHub, NopeCHALLC 11.0k ★MITUpdated 45 days ago

    git clone https://github.com/NopeCHALLC/nopecha-extension.git
  4. seo-crawler-mcp

    MCP servers

    Crawl and analyse websites for SEO errors using Crawlee with SQLite storage

    Stale Official MCP Registry, io.github.houtini-ai Updated 9 months ago 1 note

    @houtini/seo-crawler-mcp@2.0.1
  5. Crawl Readiness

    Public APIs

    Check what 50+ AI crawlers can read on a website and whether a page permits an AI use

    Indexed public-apis (GitHub list)

  6. decodo-openclaw-skill

    AI repositories

    OpenClaw skill for scraping any URL using the Decodo Web Scraping API.

    Stale GitHub, Decodo 152 ★Updated 18 days ago

    git clone https://github.com/Decodo/decodo-openclaw-skill.git
  7. donsetch

    Agents & frameworks

    Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.

    Indexed GitHub, dondai44423 1,000 ★AGPL-3.0Updated today

    git clone https://github.com/dondai44423/donsetch.git
  8. crawl-readiness

    MCP servers

    Check what AI crawlers may read and what a page permits, generate llms.txt and schema.

    Stale Official MCP Registry, io.github.Grubbomatic Updated 17 days ago 2 notes

    crawl-readiness-mcp@0.2.9
  9. llm-reader

    Agents & frameworks

    Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

    Indexed GitHub, m92vyas 316 ★MITUpdated 6 days ago

    git clone https://github.com/m92vyas/llm-reader.git
  10. Crawl Census

    MCP servers

    Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?

    Stale Official MCP Registry, io.github.taylorsmithgg Updated 2 months ago 1 note

  11. Crawl a website for links, file download links, HTML tables as rows, and lists, with no AI

    Stale Official MCP Registry, com.mambabuilt Updated 5 days ago 2 notes

    @mambalabsdev/mcp-site-crawler-links-tables@1.0.2
  12. JustHireMe

    AI repositories

    Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.

    Stale GitHub, vasu-devs 2,257 ★AGPL-3.0Updated 38 days ago

    git clone https://github.com/vasu-devs/JustHireMe.git
  13. Crawl a website into clean Markdown for LLMs/RAG: main content only, linked docs converted.

    Stale Official MCP Registry, io.github.prometheus-maker Updated 10 days ago 1 note

  14. reverse-api-engineer

    Agents & frameworks

    Reverse engineer any website into a typed API client. Works with Claude Code, Codex, Cursor and OpenCode.

    Stale GitHub, kalil0321 2,054 ★MITUpdated 18 days ago

    git clone https://github.com/kalil0321/reverse-api-engineer.git
  15. Search or monitor any Flarum-powered forum for discussions, replies, participants, dates, and…

    Stale Official MCP Registry, com.ranovam Updated 49 days ago 1 note

  16. MinerU-HTML

    AI repositories

    MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.

    Stale GitHub, opendatalab 299 ★Apache-2.0Updated 10 days ago

    git clone https://github.com/opendatalab/MinerU-HTML.git
  17. agentql-mcp

    MCP servers

    Model Context Protocol server that integrates AgentQL's data extraction capabilities.

    Indexed GitHub, tinyfish-io 183 ★MITUpdated 2 days ago

    git clone https://github.com/tinyfish-io/agentql-mcp.git
  18. Upwork-AI-jobs-applier

    Agents & frameworks

    AI tool for automating Upwork job applications using AI agents to find and qualify jobs, write personalized cover letters, and prepare for interviews based on your skills and experience.

    Stale GitHub, kaymen99 167 ★Updated 18 months ago

    git clone https://github.com/kaymen99/Upwork-AI-jobs-applier.git
  19. IndexLinks

    MCP servers

    Check how crawlers see a site, send new pages to search engines and AI crawlers, see crawl receipts.

    Stale Official MCP Registry, app.indexlinks Updated 42 days ago 1 note

  20. Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.

    Indexed GitHub, Mahanaicoach 2,430 ★MITUpdated 3 days ago

    git clone https://github.com/Mahanaicoach/google-maps-scraper-kit.git
  21. google-search

    MCP servers

    A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

    Stale GitHub, web-agent-master 623 ★Updated 19 months ago

    git clone https://github.com/web-agent-master/google-search.git
  22. Skill_Seekers

    Skills

    Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection

    Indexed GitHub, yusufkaraaslan 15.3k ★MITUpdated today

    git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
  23. x-reader

    MCP servers

    Universal content reader MCP Server for 10+ platforms

    Stale GitHub, runesleo 988 ★MITUpdated 6 days ago

    git clone https://github.com/runesleo/x-reader.git
  24. xpay Web Scraping

    MCP servers

    35+ web scraping tools. Firecrawl, Bright Data, Jina, Olostep, ScrapeGraph, Notte, Riveter.

    Stale Official MCP Registry, io.github.xpaysh Updated 8 months ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.