Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
ai-crawler-policy
MCP serversCloudflare Workers MCP server: ai-crawler-policy
-
Unrendered: AI Crawler Checker
MCP serversAI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.
unrendered-mcp@0.1.4 -
Atalaya AI crawler check
MCP serversWhich AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.
-
ai-crawler-readiness
MCP serversCan AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index
-
ActableSite AI Crawler Monitor
MCP serversCheck AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
-
ActableSite AI Crawler Monitor
MCP serversCheck AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
-
NM Signals: AI crawler audit
MCP serversAudit webpage access for AI crawlers. API key and Premium or Partner plan required.
-
Site SEO & AI Crawler Audit
MCP serversCheck public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion.
-
robots-txt-ai-crawler-audit
MCP serversChecks robots.txt line by line and says what each AI crawler token actually controls - training, AI
@readystack/robots-txt-ai-crawler-audit@0.1.3 -
Robots.txt & AI Crawler Checker
MCP serversrobots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
-
AI Crawler Access Checker
MCP serversShows if GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can read each site in a list.
-
Check declared robots.txt permissions for six AI crawlers at URL paths
-
AI Crawler Index
MCP servers{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
-
AI Crawler Access Check by Citant.ai
MCP serversCan ChatGPT, Claude and Perplexity crawlers reach a site? Real per-bot fetch, firewall/CDN blocks.
-
One-call audit: robots.txt AI-bot rules, llms.txt, JSON-LD, meta tags, sitemap. Score plus fixes.
-
Siteiz
MCP serversCheck which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
-
crawlproof
MCP serversAI-crawler audits + ads (crawlproof.com)
-
visprobe
MCP serversAI-visibility audit: AI crawler access, identity files, page signals, hosted analysis.
@reneturcios/visprobe@0.1.0 -
rattlesnakesbymail
MCP serversDated, sourced claims about AI crawlers and agents, offered as MCP tools.
-
geolint
MCP serversESLint for AI search: audits AI-crawler access, llms.txt, schema and citability.
@iliasabk/geolint@0.4.0 -
Crawlwise
MCP serversSEO audits and AI crawler checks. Honest findings only; no invented scores.
-
Crawlerpeek
MCP serversCheck whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.
crawlerpeek@0.3.1 -
Wholetally
MCP serversCookieless analytics for agents: visitors, pages, sources, AI crawlers, funnels and revenue.
-
domaintrack
MCP serversLook up domain reputation snapshots and AI crawler policy from Domduck's public store.
@reneturcios/domaintrack@0.1.0
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.