InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. Crawl4Agent

    MCP servers

    crawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...

    Stale Official MCP Registry, com.saastemly Updated 45 days ago 1 note

  2. olostep-mcp-server

    MCP servers

    MCP server for Olostep — the web scraping, crawling, and search infrastructure used by top AI companies. Gives any MCP-compatible AI agent the ability to scrape, crawl, batch-extract, and search the web in real time.

    Stale GitHub, olostep 25 ★MITUpdated 7 days ago

    git clone https://github.com/olostep/olostep-mcp-server.git
  3. crawl-readiness

    MCP servers

    Check what AI crawlers may read and what a page permits, generate llms.txt and schema.

    Stale Official MCP Registry, io.github.Grubbomatic Updated 18 days ago 2 notes

    crawl-readiness-mcp@0.2.9
  4. Crawl Census

    MCP servers

    Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?

    Stale Official MCP Registry, io.github.taylorsmithgg Updated 2 months ago 1 note

  5. Web Tools and Web Tasks for AI agents. 100-credit MCP trial; API-key credits or x402.

    Stale Official MCP Registry, dev.oassis Updated 26 days ago 1 note

  6. Free, self-hosted MCP server that scrapes the public Facebook Ad Library (no API token) for competitive ad research — advertiser, copy, landing pages, CTAs, any country.

    Stale GitHub, RamsesAguirre777 273 ★MITUpdated 2 months ago

    git clone https://github.com/RamsesAguirre777/facebook-ads-library-mcp.git
  7. Apify MCP server

    MCP servers

    Lets an agent run Apify's ready-made scrapers to pull data from websites, search engines, maps and social platforms.

    Hand-checked InnerVault (hand-checked), Apify 16.0k ★MIT

  8. crawlport

    MCP servers

    Firecrawl web data: scrape, map, search, crawl, extract, research, and llms.txt.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 17 days ago 1 note

    @reneturcios/crawlport@0.1.0
  9. CRW Web Scraper

    MCP servers

    Open-source web scraper for AI agents with scrape, crawl, and map tools

    Indexed Official MCP Registry, io.github.fastcrw 1,259 ★Updated yesterday 1 note

    ghcr.io/fastcrw/crw:latest
  10. Crawlberg

    MCP servers

    Scrape, crawl, and map websites to Markdown or JSON via local CLI.

    Stale Official MCP Registry, io.github.xberg-io Updated 21 days ago 1 note

    ghcr.io/xberg-io/crawlberg:1.12.3
  11. Nstdata Crawl

    MCP servers

    Scrape webpages, run crawls, and read Markdown and other results with Nstdata Crawl.

    Stale Official MCP Registry, io.nstdata Updated 4 days ago 1 note

  12. skald

    MCP servers

    Self-hosted Google Analytics alternative that reads your server access log: real people, AI assistants, crawlers, bots and the scanners probing your site, which JavaScript analytics never see. Leads and revenue by source. Every figure counted twice. No cookies, no third-party script. Python, SQLite, Docker.

    Indexed GitHub, spcaeo 22 ★Custom licence (see repository)Updated 4 days ago

    git clone https://github.com/spcaeo/skald.git
  13. crawlforge-mcp-server

    MCP servers

    Web scraping, crawling, deep research & autonomous extraction — 31 MCP tools, clean Markdown/JSON

    Stale Official MCP Registry, io.github.mysleekdesigns Updated 5 days ago 2 notes

    crawlforge-mcp-server@6.25.1
  14. Scrape-LE

    MCP servers

    Analyse robots.txt content and report whether a path may be crawled.

    Stale Official MCP Registry, io.github.nolindnaidoo Updated 21 days ago 1 note

    scrape-le-mcp@2.6.0
  15. Website crawling, Google search and Wikipedia lookup as agent tools.

    Stale Official MCP Registry, com.thenextgennexus Updated 10 days ago 1 note

  16. GoAI Moat Web Scrape

    MCP servers

    Scrape any URL into clean LLM-ready Markdown, or crawl a whole site.

    Stale Official MCP Registry, com.goaimoat Updated 54 days ago 1 note

  17. wet

    MCP servers

    Open-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.

    Indexed GitHub, n24q02m 24 ★Apache-2.0Updated yesterday

    git clone https://github.com/n24q02m/wet.git
  18. visprobe

    MCP servers

    AI-visibility audit: AI crawler access, identity files, page signals, hosted analysis.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 15 days ago 1 note

    @reneturcios/visprobe@0.1.0
  19. CrawlEyes MCP Server

    MCP servers

    Web scraping & search for AI agents: extract, SearXNG search, rerank, robots compliance. No API key.

    Stale Official MCP Registry, io.github.waiky-github Updated 2 months ago 2 notes

    crawleyes@0.2.0
  20. Chad Analytics

    MCP servers

    Read-only website analytics, AI referrals, crawlers, goals, revenue, and funnels.

    Stale Official MCP Registry, com.chadanalytics Updated 3 months ago 1 note

  21. rattlesnakesbymail

    MCP servers

    Dated, sourced claims about AI crawlers and agents, offered as MCP tools.

    Stale Official MCP Registry, com.rattlesnakesbymail Updated 59 days ago 1 note

  22. geolint

    MCP servers

    ESLint for AI search: audits AI-crawler access, llms.txt, schema and citability.

    Stale Official MCP Registry, io.github.iliasabk Updated 2 months ago 1 note

    @iliasabk/geolint@0.4.0
  23. matchstick

    MCP servers

    Checks a site build: sitemaps, structured data, robots.txt and what crawlers see.

    Stale Official MCP Registry, io.github.Astralchemist Updated 24 days ago 1 note

    @astralchemist/matchstick@0.1.0-alpha.4
  24. Wholetally

    MCP servers

    Cookieless analytics for agents: visitors, pages, sources, AI crawlers, funnels and revenue.

    Stale Official MCP Registry, com.wholetally Updated 7 days ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.