Scrapegraph-ai
Python scraper based on AI
Open the source (opens github.com) https://github.com/scrapegraphai/scrapegraph-ai
What it is
AI repositories from ScrapeGraphAI.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/ScrapeGraphAI/Scrapegraph-ai.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
AnyCrawl
AI repositoriesAnyCrawl ๐: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
git clone https://github.com/any4ai/AnyCrawl.git -
fess
AI repositoriesOpen-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.
git clone https://github.com/codelibs/fess.git -
Scrapegraph-demo
AI repositoriesStreamlit demo of Scrapegraph-ai for GPT4-hackaton
git clone https://github.com/ScrapeGraphAI/Scrapegraph-demo.git -
MindSearch
AI repositories๐ An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)
git clone https://github.com/InternLM/MindSearch.git -
trafilatura
AI repositoriesPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
git clone https://github.com/adbar/trafilatura.git -
rag-search
AI repositoriesRAG Search API
git clone https://github.com/thinkany-ai/rag-search.git