Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to research potential clients
-
Scrapegraph-ai
AI repositoriesPython scraper based on AI
git clone https://github.com/ScrapeGraphAI/Scrapegraph-ai.git -
trafilatura
AI repositoriesPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
git clone https://github.com/adbar/trafilatura.git -
AnyCrawl
AI repositoriesAnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
git clone https://github.com/any4ai/AnyCrawl.git -
searxNcrawl
AI repositoriesOpen-source MCP server that gives AI agents web search (SearXNG) and clean Markdown crawling of pages, whole sites and logged-in areas. Self-hosted, no API key. Firecrawl-compatible API planned.
git clone https://github.com/DasDigitaleMomentum/searxNcrawl.git -
wigolo
AI repositoriesThe go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
git clone https://github.com/KnockOutEZ/wigolo.git -
fess
AI repositoriesOpen-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.
git clone https://github.com/codelibs/fess.git -
Crawllama
AI repositoriesCrawlLama 🦙 is an local AI agent that answers questions via Ollama and integrates web- and RAG-based research.
git clone https://github.com/arn-c0de/Crawllama.git -
basemind
AI repositoriesFull AI context and content layer for coding agents over one MCP server — tree-sitter code-map, document RAG, shared memory, multi-agent comms, web crawl, git history + blame. 300+ languages, 10+ agent harnesses, pure Rust.
git clone https://github.com/Goldziher/basemind.git -
JustHireMe
AI repositoriesLocal-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.
git clone https://github.com/vasu-devs/JustHireMe.git -
decodo-openclaw-skill
AI repositoriesOpenClaw skill for scraping any URL using the Decodo Web Scraping API.
git clone https://github.com/Decodo/decodo-openclaw-skill.git -
SurfSense
AI repositoriesAir-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9
git clone https://github.com/MODSetter/SurfSense.git -
MinerU-HTML
AI repositoriesMinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
git clone https://github.com/opendatalab/MinerU-HTML.git -
awesome-agent-apis
AI repositories660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.
git clone https://github.com/Anil-matcha/awesome-agent-apis.git -
WebAI2API
AI repositoriesWebAI2API: 基于 Camoufox 的网页 AI 转 API 工具,支持 LMArena/Gemini等,多窗口并发与账号隔离。 | Web AI to OpenAI API via Camoufox. Supports LMArena/Gemini and more, multi-window concurrency & account isolation.
git clone https://github.com/foxhui/WebAI2API.git -
dingo
AI repositoriesDingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
git clone https://github.com/MigoXLab/dingo.git -
AI-Coding
AI repositoriesAI 编程福利导航:汇总 Claude Code、Codex、Cursor 中转站与公益站,整理免费额度、注册福利、签到活动和评测记录,提供可视化看板与 Python 数据采集工具。
git clone https://github.com/liuda30201/AI-Coding.git -
awesome-ai-tools
AI repositoriesA hand-picked collection of AI tools for content, automation, e-commerce, web building, voice, and growth.
git clone https://github.com/cporter202/awesome-ai-tools.git -
openserp
AI repositoriesSelf-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a real browser: ranked results and clean page Markdown. API, CLI, MCP 🎉
git clone https://github.com/karust/openserp.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.