InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. xiaobei

    Agents & frameworks

    为OPC/中小微企业量身打造的自媒体获客智能体

    Stale GitHub, TeamWiseFlow 8,711 ★Custom licence (see repository)Updated 5 days ago

    git clone https://github.com/TeamWiseFlow/xiaobei.git
  2. Abrasio

    MCP servers

    Stealth browser for AI agents: navigate, click, type and extract on bot-protected sites.

    Indexed Official MCP Registry, io.github.Scrape-Technology Updated today 2 notes

    abrasio-mcp@0.1.1
  3. Act as a Node.js Automation Script Developer. You are an expert in creating automated scripts using Node.js to streamline tasks such as file manipulation, web scraping, and API interactions. Your task is to: - Write…

    Indexed prompts.chat (CC0 prompt collection), styleyuan5@gmail.com CC0-1.0

  4. create the landing page the i want build and this my project topic name :Development of a Real-time Airfare Price Index for India through Automated Web Scraping of Airline and Online Travel Aggregator Portals for…

    Indexed prompts.chat (CC0 prompt collection), rahulharsha3131@gmail.com CC0-1.0

  5. Leaked-GPTs

    Docs & tutorials

    Leaked GPTs Prompts Bypass the 25 message limit or to try out GPTs without a Plus subscription.

    Stale GitHub, friuns2 2,475 ★Updated 2 years ago

    git clone https://github.com/friuns2/Leaked-GPTs.git
  6. llm-scraper

    AI repositories

    Turn any webpage into structured data using LLMs

    Stale GitHub, mishushakov 6,938 ★MITUpdated 2 months ago

    git clone https://github.com/mishushakov/llm-scraper.git
  7. ai-crawler-policy

    MCP servers

    Cloudflare Workers MCP server: ai-crawler-policy

    Stale Official MCP Registry, io.github.lazymac2x Updated 6 months ago 1 note

  8. Open SEO Crawler

    MCP servers

    Self-hosted SEO crawler: run site audits and AI crawler checks, then read the issues and pages.

    Stale Official MCP Registry, io.github.puneetindersingh Updated 31 days ago 1 note

    https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb@1.0.0
  9. tiktok-mcp

    Skills

    TikTok MCP server for Claude, ChatGPT, Cursor and Codex. Publish and schedule TikTok videos and photo slideshows, or send drafts to your TikTok inbox, through TikTok's official API. No scraping, no developer app.

    Indexed GitHub, postoncehq 5 ★MITUpdated today

    git clone https://github.com/postoncehq/tiktok-mcp.git
  10. polite-crawler

    MCP servers

    A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

    Indexed Official MCP Registry, io.github.mark1russell7 Updated yesterday 1 note

    polite-crawler-mcp@0.7.6
  11. Agent Crawl Readiness

    MCP servers

    Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.

    Stale Official MCP Registry, io.github.sadri-dridi Updated 54 days ago 1 note

  12. AI crawler checker: shows which page content GPTBot and ClaudeBot can't read without JavaScript.

    Stale Official MCP Registry, io.github.missiongrowth Updated 26 days ago 1 note

    unrendered-mcp@0.1.4
  13. Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

    Stale Official MCP Registry, com.atalaya-ai Updated 13 days ago 1 note

  14. ai-crawler-readiness

    MCP servers

    Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index

    Stale Official MCP Registry, com.lumnika Updated 58 days ago 1 note

  15. Hosted Enrichment Crawlers MCP server, with each action switched on or off per account.

    Stale Official MCP Registry, com.insightfulpipe Updated 26 days ago 1 note

  16. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 1 note

  17. Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    Stale Official MCP Registry, com.actablesite Updated 4 months ago 2 notes

  18. Audit webpage access for AI crawlers. API key and Premium or Partner plan required.

    Stale Official MCP Registry, io.github.newmediamadman Updated 54 days ago 1 note

  19. In-memory Bloom filter for URL and document hash deduplication preventing redundant web crawler inge

    Stale Official MCP Registry, io.github.alphaparkinc Updated 26 days ago 1 note

    https://github.com/Alpha-Park/genpark-bloom-filter-url-dedup-crawler-skill/releases/download/v1.0.1/genpark-bloom-filter-url-dedup-crawler-skill-v1.0.1.mcpb
  20. Check public pages for SEO tags, AI crawler rules, AI citation signals and landing page conversion.

    Stale Official MCP Registry, app.openkrill Updated 26 days ago 1 note

  21. Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

    Stale Official MCP Registry, io.github.jmshinhwa Updated 58 days ago 1 note

    @readystack/robots-txt-ai-crawler-audit@0.1.3
  22. robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

    Stale Official MCP Registry, dev.workers.quietworklabs.robots-check Updated 29 days ago 1 note

  23. Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.

    Stale Official MCP Registry, com.insightfulpipe Updated 26 days ago 1 note

  24. Shows if GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can read each site in a list.

    Stale Official MCP Registry, io.github.Nero-Engine Updated 15 days ago 1 note

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.