ActableSite AI Crawler Monitor
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Open the source (opens github.com) https://github.com/unitedideas/actablesite-mcp
Before you use it
- Runs on someone else's server: what you send it leaves your machine.
- Marked deprecated in the MCP Registry.
What it is
MCP servers from com.actablesite.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Remote endpoint (streamable-http)
https://actablesite.com/mcp
From the server's entry in the official MCP Registry
Related
-
ActableSite AI Crawler Monitor
MCP serversCheck AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
-
doc-scraper
MCP serversGo web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data)
git clone https://github.com/Sriram-PR/doc-scraper.git -
Crawlerpeek
MCP serversCheck whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can access a URL. Read-only.
crawlerpeek@0.3.1 -
ai-crawler-policy
MCP serversCloudflare Workers MCP server: ai-crawler-policy
-
graphlit-mcp-server
MCP serversModel Context Protocol (MCP) Server for Graphlit Platform
git clone https://github.com/graphlit/graphlit-mcp-server.git -
Open SEO Crawler
MCP serversSelf-hosted SEO crawler: run site audits and AI crawler checks, then read the issues and pages.
https://github.com/puneetindersingh/open-seo-crawler/releases/download/mcp-v1.0.0/open-seo-crawler-mcp.mcpb@1.0.0