InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. crawlport

    MCP servers

    Firecrawl web data: scrape, map, search, crawl, extract, research, and llms.txt.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 16 days ago 1 note

    @reneturcios/crawlport@0.1.0
  2. basemind

    AI repositories

    Full AI context and content layer for coding agents over one MCP server — tree-sitter code-map, document RAG, shared memory, multi-agent comms, web crawl, git history + blame. 300+ languages, 10+ agent harnesses, pure Rust.

    Indexed GitHub, Goldziher 121 ★MITUpdated today

    git clone https://github.com/Goldziher/basemind.git
  3. CRW Web Scraper

    MCP servers

    Open-source web scraper for AI agents with scrape, crawl, and map tools

    Indexed Official MCP Registry, io.github.fastcrw 1,259 ★Updated today 1 note

    ghcr.io/fastcrw/crw:latest
  4. Crawlberg

    MCP servers

    Scrape, crawl, and map websites to Markdown or JSON via local CLI.

    Stale Official MCP Registry, io.github.xberg-io Updated 20 days ago 1 note

    ghcr.io/xberg-io/crawlberg:1.12.3
  5. Nstdata Crawl

    MCP servers

    Scrape webpages, run crawls, and read Markdown and other results with Nstdata Crawl.

    Stale Official MCP Registry, io.nstdata Updated 3 days ago 1 note

  6. skald

    MCP servers

    Self-hosted Google Analytics alternative that reads your server access log: real people, AI assistants, crawlers, bots and the scanners probing your site, which JavaScript analytics never see. Leads and revenue by source. Every figure counted twice. No cookies, no third-party script. Python, SQLite, Docker.

    Indexed GitHub, spcaeo 22 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/spcaeo/skald.git
  7. crawlforge-mcp-server

    MCP servers

    Web scraping, crawling, deep research & autonomous extraction — 31 MCP tools, clean Markdown/JSON

    Stale Official MCP Registry, io.github.mysleekdesigns Updated 4 days ago 2 notes

    crawlforge-mcp-server@6.25.1
  8. crawlbrulee

    Public APIs

    EU-native web scraping for AI agents & developers: turn any page into markdown, screenshots and more

    Indexed public-apis (GitHub list)

  9. Scrape-LE

    MCP servers

    Analyse robots.txt content and report whether a path may be crawled.

    Stale Official MCP Registry, io.github.nolindnaidoo Updated 20 days ago 1 note

    scrape-le-mcp@2.6.0
  10. RealBots

    Public APIs

    Verify whether requests claiming to be GPTBot, ClaudeBot & other AI crawlers are real

    Indexed public-apis (GitHub list)

  11. Website crawling, Google search and Wikipedia lookup as agent tools.

    Stale Official MCP Registry, com.thenextgennexus Updated 9 days ago 1 note

  12. GoAI Moat Web Scrape

    MCP servers

    Scrape any URL into clean LLM-ready Markdown, or crawl a whole site.

    Stale Official MCP Registry, com.goaimoat Updated 53 days ago 1 note

  13. wet

    MCP servers

    Open-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.

    Indexed GitHub, n24q02m 24 ★Apache-2.0Updated today

    git clone https://github.com/n24q02m/wet.git
  14. cli

    Skills

    Firecrawl CLI and agent skills: search, scrape, and interact with the web from your terminal or any coding agent, plus parse, crawl, map, Agent and Monitor. Firecrawl is the context API for AI agents.

    Stale GitHub, firecrawl 678 ★Updated 10 days ago

    git clone https://github.com/firecrawl/cli.git
  15. ProxyCrawl

    Public APIs

    Scraping and crawling anticaptcha service

    Indexed public-apis (GitHub list)

  16. CitedBy

    Public APIs

    Which AI crawlers can read a site: robots.txt and server answer for each

    Indexed public-apis (GitHub list)

  17. skills

    Skills

    Firecrawl agent skills for Claude Code, Codex and Cursor: web search, scraping, crawling and browser interaction. Firecrawl is the most complete web data API for AI agents.

    Indexed GitHub, firecrawl 156 ★ISCUpdated 5 days ago

    git clone https://github.com/firecrawl/skills.git
  18. geo-skills

    Skills

    9 open Claude skills for GEO / AEO: check if AI search can read a page, collect and analyze how often AI engines name a brand (with confidence ranges), plus schema parity, AI crawler log checks and migration guards. From the AI Visibility Census.

    Indexed GitHub, aivisibilitycensus 5 ★MITUpdated 2 days ago

    git clone https://github.com/aivisibilitycensus/geo-skills.git
  19. crawlbrulee

    Public APIs

    EU-native web scraping for AI agents & developers: turn any page into markdown, screenshots and more

    Indexed public-apis (GitHub list) 1 note

  20. visprobe

    MCP servers

    AI-visibility audit: AI crawler access, identity files, page signals, hosted analysis.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 14 days ago 1 note

    @reneturcios/visprobe@0.1.0
  21. seo-ops

    Skills

    SEO foundation checks as an Agent Skill: give it a URL, get a crawler's-eye pass/fail report with evidence. 26 structural checks, zero LLM, deterministic.

    Stale GitHub, tigerless-labs 951 ★Updated 21 days ago

    git clone https://github.com/tigerless-labs/seo-ops.git
  22. CrawlEyes MCP Server

    MCP servers

    Web scraping & search for AI agents: extract, SearXNG search, rerank, robots compliance. No API key.

    Stale Official MCP Registry, io.github.waiky-github Updated 2 months ago 2 notes

    crawleyes@0.2.0
  23. Chad Analytics

    MCP servers

    Read-only website analytics, AI referrals, crawlers, goals, revenue, and funnels.

    Stale Official MCP Registry, com.chadanalytics Updated 3 months ago 1 note

  24. rattlesnakesbymail

    MCP servers

    Dated, sourced claims about AI crawlers and agents, offered as MCP tools.

    Stale Official MCP Registry, com.rattlesnakesbymail Updated 58 days ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.