InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. ai-crawler-policy

    MCP servers

    Cloudflare Workers MCP server: ai-crawler-policy

    Stale Official MCP Registry, io.github.lazymac2x Updated 6 months ago 1 note

  2. AI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.

    Stale Official MCP Registry, io.github.missiongrowth Updated 26 days ago 1 note

    unrendered-mcp@0.1.4
  3. Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

    Stale Official MCP Registry, com.atalaya-ai Updated 13 days ago 1 note

  4. ai-crawler-readiness

    MCP servers

    Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index

    Stale Official MCP Registry, com.lumnika Updated 58 days ago 1 note

  5. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 1 note

  6. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 2 notes

  7. Audit webpage access for AI crawlers. API key and Premium or Partner plan required.

    Stale Official MCP Registry, io.github.newmediamadman Updated 54 days ago 1 note

  8. Check public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion.

    Stale Official MCP Registry, app.openkrill Updated 26 days ago 1 note

  9. Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

    Stale Official MCP Registry, io.github.jmshinhwa Updated 58 days ago 1 note

    @readystack/robots-txt-ai-crawler-audit@0.1.3
  10. robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

    Stale Official MCP Registry, dev.workers.quietworklabs.robots-check Updated 29 days ago 1 note

  11. Shows if GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can read each site in a list.

    Stale Official MCP Registry, io.github.Nero-Engine Updated 15 days ago 1 note

  12. Check declared robots.txt permissions for six AI crawlers at URL paths

    Stale Official MCP Registry, io.github.kriton-dev Updated 13 days ago 1 note

  13. AI Crawler Index

    MCP servers

    {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

    Stale Official MCP Registry, dev.workers.pathwren.www Updated 2 months ago 1 note

  14. Can ChatGPT, Claude and Perplexity crawlers reach a site? Real per-bot fetch, firewall/CDN blocks.

    Stale Official MCP Registry, io.github.hamzawamiqhq Updated 4 days ago 1 note

  15. One-call audit: robots.txt AI-bot rules, llms.txt, JSON-LD, meta tags, sitemap. Score plus fixes.

    Stale Official MCP Registry, app.rowb.ai-visibility Updated 54 days ago 1 note

  16. Siteiz

    MCP servers

    Check which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.

    Stale Official MCP Registry, com.siteiz Updated 44 days ago 1 note

  17. crawlproof

    MCP servers

    AI-crawler audits + ads (crawlproof.com)

    Stale Official MCP Registry, com.profullstack Updated 5 days ago 1 note

  18. visprobe

    MCP servers

    AI-visibility audit: AI crawler access, identity files, page signals, hosted analysis.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 14 days ago 1 note

    @reneturcios/visprobe@0.1.0
  19. rattlesnakesbymail

    MCP servers

    Dated, sourced claims about AI crawlers and agents, offered as MCP tools.

    Stale Official MCP Registry, com.rattlesnakesbymail Updated 58 days ago 1 note

  20. geolint

    MCP servers

    ESLint for AI search: audits AI-crawler access, llms.txt, schema and citability.

    Stale Official MCP Registry, io.github.iliasabk Updated 2 months ago 1 note

    @iliasabk/geolint@0.4.0
  21. Crawlwise

    MCP servers

    SEO audits and AI crawler checks. Honest findings only; no invented scores.

    Stale Official MCP Registry, site.crawlwise Updated 11 days ago 1 note

  22. Crawlerpeek

    MCP servers

    Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.

    Stale Official MCP Registry, io.github.crawlerpeek Updated 18 days ago 1 note

    crawlerpeek@0.3.1
  23. Wholetally

    MCP servers

    Cookieless analytics for agents: visitors, pages, sources, AI crawlers, funnels and revenue.

    Stale Official MCP Registry, com.wholetally Updated 6 days ago 1 note

  24. domaintrack

    MCP servers

    Look up domain reputation snapshots and AI crawler policy from Domduck's public store.

    Stale Official MCP Registry, io.github.CloudCorpRecords Updated 12 days ago 1 note

    @reneturcios/domaintrack@0.1.0

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.