Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
robotstxt-ai
MCP serversAnalyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
robotstxt-ai-mcp@1.0.1 -
geo-tool-check
MCP serversIs this page readable for AI search? Crawler access, structure and citability checks, runs locally.
geo-tool-check@1.0.1 -
mcp
MCP serversRun web performance audits, get code-level fixes, crawl for SEO, and read real-user Core Web Vitals.
@speedvault/mcp@1.0.3 -
CitedBy
MCP serversCheck a page, AI-crawler access, llms.txt and backlinks of a site. Free, read-only.
-
QueryRev AI Readiness
MCP serversChecks a site for AI crawler access, robots.txt and llms.txt. Runs locally, no telemetry.
@queryrev/ai-readiness-mcp@1.0.0 -
Firecrawl by Nova (CIVAI)
MCP serversScrape, crawl, and search the web via Firecrawl’s official hosted MCP.
-
Outcrawl
MCP serversWeb infrastructure for AI agents: scrape, crawl and search the web, run browsers and agents.
-
SEOClave
MCP serversGoogle keyword data for Latin America plus technical SEO, robots.txt, DNS and AI-crawler checks.
-
Atlas Web Compiler & Scraper
MCP serversDeterministic Zero-LLM Web Compiler, Scraper, Crawler, and Site Mapper for AI Agents.
@atlascompiler/mcp@1.0.5 -
seo-web-analysis-mcp-server
MCP serversOn-page crawl, tech-stack detection, DNS, SSL and WHOIS as agent tools.
-
Viral Outliers
MCP serversFind viral outlier posts on TikTok, Instagram and YouTube, pull creator stats, and crawl on demand.
-
PageWire
MCP serversWeb pages to Markdown, metadata and small crawls for agents. Free daily calls, then USDC x402/MPP.
-
WebLens
MCP serversScrape, crawl, map and extract the web. Pay per call in USDC, no account or API key.
-
AIReady
MCP serversScan and fix your site's AI discoverability: crawler access, llms.txt, JSON-LD. Free.
-
rasterly
MCP serversWeb toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.
rasterly-mcp@0.2.1 -
AEO Audit
MCP serversAEO audit: score any website 0-100 for AI visibility. Checks schema, meta, content, AI crawlers.
-
Storefront Check
MCP serversAudit any storefront from outside: ad tags, dead-tag detection, AI crawler access, feed readiness.
-
SEO Records
MCP serversSEO check of any website and AI crawler access test (access pass), and the full Audit.
-
RobotsGate
MCP serversGenerate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.
-
ContextFetch
Public APIsWeb search and any page as Markdown pruned to the query, plus research, crawl and site maps
-
SiteWide
MCP serversOne MCP connector for GA4, Search Console, Google Ads, Ahrefs, a site crawler and Web Vitals.
-
PinPinterest — Pinterest Keyword Research
MCP serversPinterest keyword research from our own crawl: saves, competition, related terms, trends, top pins.
-
Geonode MCP Server
MCP serversWeb data for agents. Extract web pages, crawl websites, discover URLs, and search the web.
-
x402serps
MCP serversPay-per-call web search from $0.004, cited answers, crawl, research, SEO and leads via x402.
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.