Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
Curator
AI repositoriesScalable data pre processing and curation toolkit for LLMs
git clone https://github.com/NVIDIA-NeMo/Curator.git -
data-juicer
AI repositoriesData processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
git clone https://github.com/datajuicer/data-juicer.git -
df0
AI repositoriesDF-0: Data processing Framework with Zero-overhead. Local-first C++20 data-processing infrastructure for deterministic, auditable, on-premise workflows. Supports 100+ formats, OCR, local AI, and opt-in cloud AI pipelines
git clone https://github.com/df0-io/df0.git -
cocoindex
AI repositoriesIncremental engine for long horizon agents 🌟 Star if you like it!
git clone https://github.com/cocoindex-io/cocoindex.git -
docetl
AI repositoriesA system for agentic LLM-powered data processing and ETL
git clone https://github.com/ucbepic/docetl.git -
DataMate
AI repositoriesDataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.
git clone https://github.com/ModelEngine-Group/DataMate.git -
FastGPT
AI repositoriesFastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
git clone https://github.com/labring/FastGPT.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.