Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
Douyin_TikTok_Download_API
MCP servers🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API,支持 AI agent 通过 MCP 调用与离线语音转文字,Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
git clone https://github.com/Evil0ctal/Douyin_TikTok_Download_API.git -
Turn a website into Markdown with headings, links and word counts.
-
IndexLinks
MCP serversCheck how crawlers see a site, send new pages to search engines and AI crawlers, see crawl receipts.
-
Crawler IP Verifier - real Googlebot or fake
MCP servers{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
-
Web Tools and Web Tasks for AI agents. 100-credit MCP trial; API-key credits or x402.
-
One-call audit: robots.txt AI-bot rules, llms.txt, JSON-LD, meta tags, sitemap. Score plus fixes.
-
seo-audit-console
MCP serversTechnical SEO in Claude: GSC + crawl + DataForSEO - audit, portfolio, market sizing, content recon.
@houtini/seo-audit-console@1.0.5 -
MeshArc
MCP serversCrawl and scrape websites to clean markdown, and track what changed between crawls.
-
Black SEO Analyzer
MCP serversAudit a page or crawl a whole site for technical SEO issues, locally. Compares crawls.
https://github.com/sethblack/black-seo-analyzer/releases/download/v26.11.3/bseoa-26.11.3-win32.mcpb@26.11.3 -
fastCRW
MCP serversScrape, crawl, map & search the web. Open-source, self-hostable Rust crawler & search for AI agents.
-
Siteiz
MCP serversCheck which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
-
UAParse
MCP serversParse a User-Agent: browser, OS, device, plus bot, search crawler, AI crawler and AI agent flags.
-
Commoncrawl
MCP serversCommon Crawl's public web archive — find every time a URL was crawled since 2008 and read the…
@pipeworx/mcp-commoncrawl@0.2.1 -
wet
MCP serversOpen-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.
git clone https://github.com/n24q02m/wet.git -
skald
MCP serversSelf-hosted Google Analytics alternative that reads your server access log: real people, AI assistants, crawlers, bots and the scanners probing your site, which JavaScript analytics never see. Leads and revenue by source. Every figure counted twice. No cookies, no third-party script. Python, SQLite, Docker.
git clone https://github.com/spcaeo/skald.git -
lilbee
MCP serversA local model engine with a search engine and web crawler built in. Run your coding agents on your own GPUs, and get cited answers from your files, code, and the sites you crawl. One executable.
git clone https://github.com/tobocop2/lilbee.git -
Svipall
MCP serversRead any page past its anti-bot wall, crawl sites, search without an API key, solve captchas locally
svipall@2.1.0 -
firecrawl
MCP serversScrape, crawl, and extract structured data from websites at scale
mcparmory-firecrawl@1.0.3 -
Agent402
MCP serversagent402.tools: 500+ pay-per-call tools, metered models and finished reports for AI agents, paid per call in USDC over x402 and MPP or by card. Open source, self-hostable, MCP-native. The applied layer of agentic finance.
git clone https://github.com/MikeyPetrillo/Agent402.git -
MESSORA
MCP serversScrape, crawl and search the web for AI agents via MCP.
-
deepsearch
MCP serversNeural web search, code context, and page crawling through the Exa API.
@reneturcios/deepsearch@0.1.0 -
visprobe
MCP serversAI-visibility audit: AI crawler access, identity files, page signals, hosted analysis.
@reneturcios/visprobe@0.1.0 -
Chad Analytics
MCP serversRead-only website analytics, AI referrals, crawlers, goals, revenue, and funnels.
-
rattlesnakesbymail
MCP serversDated, sourced claims about AI crawlers and agents, offered as MCP tools.
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.