deepmark
Deepmark AI enables a unique testing environment for language models (LLM) assessment on task-specific metrics and on your own data so your GenAI-powered solution has predictable and reliable performance.
Open the source (opens github.com) https://github.com/ingestai/deepmark
What it is
Datasets & benchmarks from IngestAI.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/IngestAI/deepmark.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
ollama-benchmark
Datasets & benchmarksLLM Benchmark for Throughput via Ollama (Local LLMs)
git clone https://github.com/aidatatools/ollama-benchmark.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
ollama-benchmark
Datasets & benchmarksOllama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
git clone https://github.com/LarHope/ollama-benchmark.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
AI_Diplomacy
Datasets & benchmarksFrontier Models playing the board game Diplomacy.
git clone https://github.com/GoodStartLabs/AI_Diplomacy.git