InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to research potential clients

  1. AnyCrawl

    AI repositories

    AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

    Indexed GitHub, any4ai 3,528 ★MITUpdated 4 days ago

    git clone https://github.com/any4ai/AnyCrawl.git
  2. trafilatura

    AI repositories

    Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

    Indexed GitHub, adbar 7,128 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/adbar/trafilatura.git
  3. Scrapegraph-ai

    AI repositories

    Python scraper based on AI

    Stale GitHub, ScrapeGraphAI 31.8k ★MITUpdated 41 days ago

    git clone https://github.com/ScrapeGraphAI/Scrapegraph-ai.git
  4. searxNcrawl

    AI repositories

    Open-source MCP server that gives AI agents web search (SearXNG) and clean Markdown crawling of pages, whole sites and logged-in areas. Self-hosted, no API key. Firecrawl-compatible API planned.

    Stale GitHub, DasDigitaleMomentum 189 ★MITUpdated 5 days ago

    git clone https://github.com/DasDigitaleMomentum/searxNcrawl.git
  5. wigolo

    AI repositories

    The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.

    Indexed GitHub, KnockOutEZ 5,546 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/KnockOutEZ/wigolo.git
  6. decodo-openclaw-skill

    AI repositories

    OpenClaw skill for scraping any URL using the Decodo Web Scraping API.

    Stale GitHub, Decodo 152 ★Updated 18 days ago

    git clone https://github.com/Decodo/decodo-openclaw-skill.git
  7. JustHireMe

    AI repositories

    Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.

    Stale GitHub, vasu-devs 2,257 ★AGPL-3.0Updated 38 days ago

    git clone https://github.com/vasu-devs/JustHireMe.git
  8. MinerU-HTML

    AI repositories

    MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.

    Stale GitHub, opendatalab 299 ★Apache-2.0Updated 10 days ago

    git clone https://github.com/opendatalab/MinerU-HTML.git
  9. SurfSense

    AI repositories

    Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord: https://discord.gg/ejRNvftDp9

    Indexed GitHub, MODSetter 16.9k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/MODSetter/SurfSense.git
  10. fess

    AI repositories

    Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache-2.0.

    Indexed GitHub, codelibs 1,151 ★Apache-2.0Updated yesterday

    git clone https://github.com/codelibs/fess.git
  11. awesome-agent-apis

    AI repositories

    660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.

    Stale GitHub, Anil-matcha 1,057 ★MITUpdated 33 days ago

    git clone https://github.com/Anil-matcha/awesome-agent-apis.git
  12. WebAI2API

    AI repositories

    WebAI2API: 基于 Camoufox 的网页 AI 转 API 工具,支持 LMArena/Gemini等,多窗口并发与账号隔离。 | Web AI to OpenAI API via Camoufox. Supports LMArena/Gemini and more, multi-window concurrency & account isolation.

    Stale GitHub, foxhui 1,388 ★MITUpdated 4 months ago

    git clone https://github.com/foxhui/WebAI2API.git
  13. awesome-ai-tools

    AI repositories

    A hand-picked collection of AI tools for content, automation, e-commerce, web building, voice, and growth.

    Stale GitHub, cporter202 311 ★Updated 27 days ago

    git clone https://github.com/cporter202/awesome-ai-tools.git
  14. Crawllama

    AI repositories

    CrawlLama 🦙 is an local AI agent that answers questions via Ollama and integrates web- and RAG-based research.

    Indexed GitHub, arn-c0de 109 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/arn-c0de/Crawllama.git
  15. AI-Coding

    AI repositories

    AI 编程福利导航:汇总 Claude Code、Codex、Cursor 中转站与公益站,整理免费额度、注册福利、签到活动和评测记录,提供可视化看板与 Python 数据采集工具。

    Indexed GitHub, liuda30201 610 ★Updated today

    git clone https://github.com/liuda30201/AI-Coding.git
  16. basemind

    AI repositories

    Full AI context and content layer for coding agents over one MCP server — tree-sitter code-map, document RAG, shared memory, multi-agent comms, web crawl, git history + blame. 300+ languages, 10+ agent harnesses, pure Rust.

    Indexed GitHub, Goldziher 121 ★MITUpdated today

    git clone https://github.com/Goldziher/basemind.git
  17. openserp

    AI repositories

    Self-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a real browser: ranked results and clean page Markdown. API, CLI, MCP 🎉

    Stale GitHub, karust 1,571 ★MITUpdated 5 days ago

    git clone https://github.com/karust/openserp.git
  18. llm-scraper

    AI repositories

    Turn any webpage into structured data using LLMs

    Stale GitHub, mishushakov 6,938 ★MITUpdated 2 months ago

    git clone https://github.com/mishushakov/llm-scraper.git
  19. dingo

    AI repositories

    Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

    Stale GitHub, MigoXLab 764 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/MigoXLab/dingo.git
  20. Scrapegraph-demo

    AI repositories

    Streamlit demo of Scrapegraph-ai for GPT4-hackaton

    Indexed GitHub, ScrapeGraphAI 120 ★MITUpdated yesterday

    git clone https://github.com/ScrapeGraphAI/Scrapegraph-demo.git
  21. freehire

    AI repositories

    freehire — the open-source search engine for job seekers

    Indexed GitHub, strelov1 3,543 ★MITUpdated today

    git clone https://github.com/strelov1/freehire.git
  22. Open-source AI sales & marketing OS: find leads, draft outreach in your own voice, approve, close. Self-host or use alatpintar.id

    Indexed GitHub, asiifdev 246 ★AGPL-3.0Updated 2 days ago

    git clone https://github.com/asiifdev/business-leads-ai-automation.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.