InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

5 results with any of your words Clear search and filters

Looks like a goal we have a starting path for: I want to research potential clients

  1. Crawl4AI

    Public APIs

    An open-source crawler that turns websites into Markdown ready for an AI model.

    Hand-checked InnerVault (hand-checked), unclecode 87.3k ★Apache-2.0

    pip install -U crawl4ai
  2. RealBots

    Public APIs

    Verify whether requests claiming to be GPTBot, ClaudeBot & other AI crawlers are real

    Indexed public-apis (GitHub list)

  3. CitedBy

    Public APIs

    Which AI crawlers can read a site: robots.txt and server answer for each

    Indexed public-apis (GitHub list)

  4. TidyTools

    Public APIs

    Web page to clean Markdown and AI crawler robots.txt checks, free without a key (rate limited)

    Indexed public-apis (GitHub list)

  5. Crawl Readiness

    Public APIs

    Check what 50+ AI crawlers can read on a website and whether a page permits an AI use

    Indexed public-apis (GitHub list)

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.