InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. tdm-reservation-lint

    MCP servers

    Finds the robots.txt lines that reserve nothing — retired crawler tokens, shadowed groups, rules abo

    Stale Official MCP Registry, io.github.jmshinhwa Updated 55 days ago 1 note

    @readystack/tdm-reservation-lint@1.0.5
  2. AI Visibility Checker

    MCP servers

    AI visibility checker: GPTBot vs OAI-SearchBot, robots.txt audit, raw-HTML schema. 165 crawlers.

    Stale Official MCP Registry, io.github.krisdiallo Updated 2 months ago 1 note

    https://github.com/krisdiallo/ecom-agent/releases/download/v1.5.3/aivis.mcpb@1.5.3
  3. polite-crawler

    MCP servers

    A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

    Indexed Official MCP Registry, io.github.mark1russell7 Updated 2 days ago 1 note

    polite-crawler-mcp@0.7.6
  4. robotstxt-ai

    MCP servers

    Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

    Stale Official MCP Registry, io.github.sharozdawa Updated 8 months ago 1 note

    robotstxt-ai-mcp@1.0.1
  5. QueryRev AI Readiness

    MCP servers

    Checks a site for AI crawler access, robots.txt and llms.txt. Runs locally, no telemetry.

    Stale Official MCP Registry, com.queryrev Updated 23 days ago 1 note

    @queryrev/ai-readiness-mcp@1.0.0
  6. Doorman

    MCP servers

    Check which AI bots a website or a pasted robots.txt lets in, grouped by purpose.

    Stale Official MCP Registry, com.clawhard Updated 8 days ago 1 note

  7. SEOClave

    MCP servers

    Google keyword data for Latin America plus technical SEO, robots.txt, DNS and AI-crawler checks.

    Stale Official MCP Registry, com.seoclave Updated 5 days ago 1 note

  8. llms.txt 檢查工具

    MCP servers

    檢查網站 llms.txt 與 AI 爬蟲 robots.txt 設定,並自動產生 llms.txt。

    Stale Official MCP Registry, dev.workers.panda198271.llms-check Updated 51 days ago 1 note

  9. Page Fetch

    MCP servers

    Read one public web page as clean Markdown with its links, after a robots.txt check.

    Stale Official MCP Registry, app.openkrill Updated 27 days ago 1 note

  10. GEO SEO Records

    MCP servers

    Gets a website ready for AI assistants: robots.txt fixed, llms.txt drafted (access pass).

    Stale Official MCP Registry, com.seorecords Updated 8 days ago 1 note

  11. RobotsGate

    MCP servers

    Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

    Stale Official MCP Registry, dev.workers.mike-tusa.robotsgate Updated 32 days ago 1 note

  12. WebRead

    MCP servers

    Turn any public web page into clean Markdown for LLMs; respects robots.txt. By Crypto Money Maker

    Stale Official MCP Registry, co.youraitools Updated 27 days ago 1 note

  13. ai-seo

    MCP servers

    MCP server for AI-SEO auditing: schema, robots.txt, llms.txt, citation scoring, and AEO/GEO rewrites

    Stale Official MCP Registry, io.github.AutomateLab-tech Updated 6 months ago 1 note

    @automatelab/ai-seo-mcp@0.1.1
  14. NetLens

    MCP servers

    Read any web page as clean, ad-stripped Markdown past robots.txt and bot blocks, plus web search.

    Stale Official MCP Registry, io.github.pzalutski-pixel Updated 3 months ago 1 note

    netlens-mcp@0.2.3
  15. Agent Crawl Readiness

    MCP servers

    Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.

    Stale Official MCP Registry, io.github.sadri-dridi Updated 55 days ago 1 note

  16. web-analytics-mcp

    MCP servers

    Web Analitiği MCP sunucusu — in-house on-page SEO denetimi (robots.txt + SSRF korumalı crawl),…

    Stale Official MCP Registry, io.github.sercanmetalore Updated 3 months ago 1 note

  17. Citehound

    MCP servers

    Scan any domain for AI readiness and generate llms.txt, robots.txt and schema. Free, read-only.

    Stale Official MCP Registry, io.github.andacuzel Updated 26 days ago 1 note

  18. Checks robots.txt for AI search vs training crawlers, llms.txt, JSON-LD, canonicals and sitemaps.

    Stale Official MCP Registry, io.github.alicexixi85 Updated 7 days ago 1 note

    ghcr.io/alicexixi85/geo-seo-mcp-pro:0.1.0
  19. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 1 note

  20. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 2 notes

  21. Web Hygiene MCP

    MCP servers

    Live web checks for AI agents: sitemaps, robots.txt, URL status, broken links, feeds, citations.

    Stale Official MCP Registry, io.github.Luca-003 Updated 57 days ago 1 note

  22. MAJR AI SEO

    MCP servers

    Audit a page for search and AI answer engines; generate robots.txt, sitemap, head, llms.txt.

    Stale Official MCP Registry, io.github.majrdotapp Updated 2 months ago 1 note

  23. Agent-readiness check

    MCP servers

    Checks a website for AI agents: robots.txt, llms.txt, MCP, A2A. Ranked fixes, text to paste.

    Stale Official MCP Registry, com.wondaverse Updated 18 days ago 1 note

  24. Free, no-auth tools for AI search visibility: robots.txt, schema, rendering, sitemap, PageSpeed.

    Stale Official MCP Registry, io.github.lazycipher Updated 9 days ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.