Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
28 results
Clear search and filters
Looks like a goal we have a starting path for: I want to research potential clients
-
CitedBy
Public APIsWhich AI crawlers can read a site: robots.txt and server answer for each
-
TidyTools
Public APIsWeb page to clean Markdown and AI crawler robots.txt checks, free without a key (rate limited)
-
ScraperApi
Public APIsEasily build scalable web scrapers
-
Crawl Readiness
Public APIsCheck what 50+ AI crawlers can read on a website and whether a page permits an AI use
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.