InnerVault

Firecrawl

Hand-checked InnerVault (hand-checked), Firecrawl

Turns websites into clean Markdown or structured data for AI, through a hosted API or a self-hosted copy.

Open the source (opens github.com) https://github.com/firecrawl/firecrawl

Before you use it

  • The hosted API needs an API key. Self-hosting under AGPL-3.0 has obligations if you offer it to others as a service.

What it is

Public APIs from Firecrawl. Good for:

  • Feed a company's whole site into a research prompt
  • Keep an eye on pages for changes

Level: builder

How to get it

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. Crawl4AI

    Public APIs

    An open-source crawler that turns websites into Markdown ready for an AI model.

    Hand-checked InnerVault (hand-checked), unclecode 87.3k ★Apache-2.0

    pip install -U crawl4ai
  2. SnoopScan

    Public APIs

    Scrape, crawl, map and search the web as clean markdown or JSON, with a free monthly tier

    Indexed public-apis (GitHub list)

  3. Webclaw

    Public APIs

    Web content extraction for LLMs with scrape, crawl, search, and summarize

    Indexed public-apis (GitHub list)

  4. GcrawlAI

    Public APIs

    Web scraping, SERP API, screenshots, URL to Markdown and prebuilt extractors

    Indexed public-apis (GitHub list)

  5. ProxyCrawl

    Public APIs

    Scraping and crawling anticaptcha service

    Indexed public-apis (GitHub list)

  6. crawlbrulee

    Public APIs

    EU-native web scraping for AI agents & developers: turn any page into markdown, screenshots and more

    Indexed public-apis (GitHub list)