InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. CRW Web Scraper

    MCP servers

    Open-source web scraper for AI agents with scrape, crawl, and map tools

    Indexed Official MCP Registry, io.github.fastcrw 1,259 ★Updated today 1 note

    ghcr.io/fastcrw/crw:latest
  2. seo-crawler-mcp

    MCP servers

    Crawl and analyse websites for SEO errors using Crawlee with SQLite storage

    Stale Official MCP Registry, io.github.houtini-ai Updated 9 months ago 1 note

    @houtini/seo-crawler-mcp@2.0.1
  3. Nstdata Crawl

    MCP servers

    Scrape webpages, run crawls, and read Markdown and other results with Nstdata Crawl.

    Stale Official MCP Registry, io.nstdata Updated 3 days ago 1 note

  4. Crawlwise

    MCP servers

    SEO audits and AI crawler checks. Honest findings only; no invented scores.

    Stale Official MCP Registry, site.crawlwise Updated 11 days ago 1 note

  5. Crawlerpeek

    MCP servers

    Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.

    Stale Official MCP Registry, io.github.crawlerpeek Updated 18 days ago 1 note

    crawlerpeek@0.3.1
  6. Crawlie

    MCP servers

    Technical SEO + GEO (AI-search) site audits: hosted crawls, prioritized fixes, report diffs.

    Stale Official MCP Registry, app.crawlie Updated 4 months ago 1 note

  7. Crawl4Agent

    MCP servers

    crawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...

    Stale Official MCP Registry, com.saastemly Updated 44 days ago 1 note

  8. Crawlgraph

    MCP servers

    CrawlGraph MCP — backlink intelligence on the public Common Crawl webgraph (4.4B edges, 120M…

    Stale Official MCP Registry, io.github.pipeworx-io Updated 16 days ago 3 notes

    @pipeworx/mcp-crawlgraph@0.2.1
  9. ai-crawler-policy

    MCP servers

    Cloudflare Workers MCP server: ai-crawler-policy

    Stale Official MCP Registry, io.github.lazymac2x Updated 6 months ago 1 note

  10. Open SEO Crawler

    MCP servers

    Self-hosted SEO crawler: run site audits and AI crawler checks, then read the issues and pages.

    Stale Official MCP Registry, io.github.puneetindersingh Updated 31 days ago 1 note

    https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb@1.0.0
  11. crawlport

    MCP servers

    Firecrawl web data: scrape, map, search, crawl, extract, research, and llms.txt.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 16 days ago 1 note

    @reneturcios/crawlport@0.1.0
  12. Crawlberg

    MCP servers

    Scrape, crawl, and map websites to Markdown or JSON via local CLI.

    Stale Official MCP Registry, io.github.xberg-io Updated 20 days ago 1 note

    ghcr.io/xberg-io/crawlberg:1.12.3
  13. CrawlRaven

    MCP servers

    Read Search Console performance, keyword opportunities and annotations for your sites.

    Stale Official MCP Registry, com.crawlraven Updated 3 months ago 1 note

  14. crawlforge-mcp-server

    MCP servers

    Web scraping, crawling, deep research & autonomous extraction — 31 MCP tools, clean Markdown/JSON

    Stale Official MCP Registry, io.github.mysleekdesigns Updated 4 days ago 2 notes

    crawlforge-mcp-server@6.25.1
  15. Crawlora MCP

    MCP servers

    Hosted MCP: 4684 structured web-data tools across 534 platform groups.

    Stale Official MCP Registry, net.crawlora Updated 6 days ago 1 note

  16. Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay

    Stale Official MCP Registry, io.github.kriton-dev Updated 14 days ago 1 note

  17. Crawl Readiness

    Public APIs

    Check what 50+ AI crawlers can read on a website and whether a page permits an AI use

    Indexed public-apis (GitHub list)

  18. Ultra-FineWeb-L1

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Stale Hugging Face Hub, openbmb 201 ♥apache-2.0Updated 2 months ago

    openbmb/Ultra-FineWeb-L1
  19. CrawlConsole

    MCP servers

    Backlink Analysis MCP for AI Agents: authority, referring domains, and competitor link gaps.

    Stale Official MCP Registry, com.crawlconsole Updated 6 months ago 2 notes

    @crawlconsole/mcp@0.1.4
  20. crawl-readiness

    MCP servers

    Check what AI crawlers may read and what a page permits, generate llms.txt and schema.

    Stale Official MCP Registry, io.github.Grubbomatic Updated 17 days ago 2 notes

    crawl-readiness-mcp@0.2.9
  21. polite-crawler

    MCP servers

    A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

    Indexed Official MCP Registry, io.github.mark1russell7 Updated yesterday 1 note

    polite-crawler-mcp@0.7.6
  22. crawlgraph-mcp

    MCP servers

    Backlink lookups and competitor gap analysis on Common Crawl's open web graph (4.4B links).

    Stale Official MCP Registry, io.github.pucilpet Updated 4 months ago 2 notes

    https://github.com/pucilpet/crawlgraph-mcp/releases/download/v0.2.2/crawlgraph-mcp.mcpb
  23. Eclipse Crawlway

    MCP servers

    Pay-per-crawl licensing receipts: sealed identity+terms+fingerprint binding. LOCAL/fixture only.

    Stale Official MCP Registry, io.github.CYAN-ECLIPSE-TECHNOLOGIES-LLC Updated 20 days ago

  24. crawlio-browser

    MCP servers

    Bridge a live Chrome browser to your agent: 150 tools for capture, extraction, and observation.

    Stale Official MCP Registry, app.crawlio Updated 3 months ago 1 note

    crawlio-browser@1.11.0

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.