InnerVault

robots.txt Allow count, body discarded

Stale Official MCP Registry, io.github.sadri-dridi

robots.txt Allow count, body discarded

Open the source (opens github.com) https://github.com/sadri-dridi/named-mcp-utilities

Before you use it

  • Runs on someone else's server: what you send it leaves your machine.

What it is

MCP servers from io.github.sadri-dridi.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Remote endpoint (streamable-http)

https://agent-observatory-sensor.nolimit-observatory.workers.dev/s/robots-allow-count/mcp

From the server's entry in the official MCP Registry

Related

  1. robots-txt-checker

    MCP servers

    Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).

    Stale Official MCP Registry, io.github.ryuamarines Updated 18 days ago 1 note

  2. robots.txt group count

    Stale Official MCP Registry, io.github.sadri-dridi Updated 2 months ago 1 note

  3. robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

    Stale Official MCP Registry, dev.workers.quietworklabs.robots-check Updated 29 days ago 1 note

  4. Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

    Stale Official MCP Registry, io.github.jmshinhwa Updated 58 days ago 1 note

    @readystack/robots-txt-ai-crawler-audit@0.1.3
  5. Check declared robots.txt permissions for six AI crawlers at URL paths

    Stale Official MCP Registry, io.github.kriton-dev Updated 13 days ago 1 note

  6. {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

    Stale Official MCP Registry, dev.workers.pathwren.www Updated 2 months ago 1 note