InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. robotstxt-ai

    MCP servers

    Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

    Stale Official MCP Registry, io.github.sharozdawa Updated 8 months ago 1 note

    robotstxt-ai-mcp@1.0.1
  2. geo-tool-check

    MCP servers

    Is this page readable for AI search? Crawler access, structure and citability checks, runs locally.

    Stale Official MCP Registry, io.github.shufflethis Updated 2 months ago 1 note

    geo-tool-check@1.0.1
  3. mcp

    MCP servers

    Run web performance audits, get code-level fixes, crawl for SEO, and read real-user Core Web Vitals.

    Stale Official MCP Registry, io.speedvault Updated 5 months ago 2 notes

    @speedvault/mcp@1.0.3
  4. CitedBy

    MCP servers

    Check a page, AI-crawler access, llms.txt and backlinks of a site. Free, read-only.

    Stale Official MCP Registry, eu.citedby Updated 23 days ago 1 note

  5. QueryRev AI Readiness

    MCP servers

    Checks a site for AI crawler access, robots.txt and llms.txt. Runs locally, no telemetry.

    Stale Official MCP Registry, com.queryrev Updated 22 days ago 1 note

    @queryrev/ai-readiness-mcp@1.0.0
  6. Scrape, crawl, and search the web via Firecrawl’s official hosted MCP.

    Stale Official MCP Registry, co.civai.nova Updated 14 days ago 1 note

  7. Outcrawl

    MCP servers

    Web infrastructure for AI agents: scrape, crawl and search the web, run browsers and agents.

    Stale Official MCP Registry, io.github.outcrawlai Updated 14 days ago 1 note

  8. SEOClave

    MCP servers

    Google keyword data for Latin America plus technical SEO, robots.txt, DNS and AI-crawler checks.

    Stale Official MCP Registry, com.seoclave Updated 4 days ago 1 note

  9. Deterministic Zero-LLM Web Compiler, Scraper, Crawler, and Site Mapper for AI Agents.

    Stale Official MCP Registry, com.atlas-compiler Updated 22 days ago 3 notes

    @atlascompiler/mcp@1.0.5
  10. On-page crawl, tech-stack detection, DNS, SSL and WHOIS as agent tools.

    Stale Official MCP Registry, com.thenextgennexus Updated 9 days ago 1 note

  11. Viral Outliers

    MCP servers

    Find viral outlier posts on TikTok, Instagram and YouTube, pull creator stats, and crawl on demand.

    Stale Official MCP Registry, com.viraloutliers Updated 3 months ago 1 note

  12. PageWire

    MCP servers

    Web pages to Markdown, metadata and small crawls for agents. Free daily calls, then USDC x402/MPP.

    Stale Official MCP Registry, dev.pagewire Updated 41 days ago 1 note

  13. WebLens

    MCP servers

    Scrape, crawl, map and extract the web. Pay per call in USDC, no account or API key.

    Stale Official MCP Registry, dev.weblens Updated 3 months ago 1 note

  14. AIReady

    MCP servers

    Scan and fix your site's AI discoverability: crawler access, llms.txt, JSON-LD. Free.

    Stale Official MCP Registry, io.github.everyai-com Updated 43 days ago 1 note

  15. rasterly

    MCP servers

    Web toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.

    Stale Official MCP Registry, io.github.rasterly-dev Updated 2 months ago 2 notes

    rasterly-mcp@0.2.1
  16. AEO Audit

    MCP servers

    AEO audit: score any website 0-100 for AI visibility. Checks schema, meta, content, AI crawlers.

    Stale Official MCP Registry, no.synligdigital Updated 8 months ago 1 note

  17. Storefront Check

    MCP servers

    Audit any storefront from outside: ad tags, dead-tag detection, AI crawler access, feed readiness.

    Stale Official MCP Registry, com.profitpather.tools Updated 32 days ago 1 note

  18. SEO Records

    MCP servers

    SEO check of any website and AI crawler access test (access pass), and the full Audit.

    Stale Official MCP Registry, com.seorecords Updated 7 days ago 1 note

  19. RobotsGate

    MCP servers

    Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

    Stale Official MCP Registry, dev.workers.mike-tusa.robotsgate Updated 31 days ago 1 note

  20. ContextFetch

    Public APIs

    Web search and any page as Markdown pruned to the query, plus research, crawl and site maps

    Indexed public-apis (GitHub list) 1 note

  21. SiteWide

    MCP servers

    One MCP connector for GA4, Search Console, Google Ads, Ahrefs, a site crawler and Web Vitals.

    Stale Official MCP Registry, io.github.macminibaus Updated 26 days ago 1 note

  22. Pinterest keyword research from our own crawl: saves, competition, related terms, trends, top pins.

    Stale Official MCP Registry, com.pinpinterest Updated 23 days ago 1 note

  23. Geonode MCP Server

    MCP servers

    Web data for agents. Extract web pages, crawl websites, discover URLs, and search the web.

    Stale Official MCP Registry, com.geonode Updated 20 days ago 1 note

  24. x402serps

    MCP servers

    Pay-per-call web search from $0.004, cited answers, crawl, research, SEO and leads via x402.

    Stale Official MCP Registry, com.x402serps Updated 13 days ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.