AgentBench vs teaql-agent-kit
The two are close in size: 3.8k stars for AgentBench, 2.8k for teaql-agent-kit. Over the days we have tracked them AgentBench moved +4.5% and teaql-agent-kit +0.5%, so AgentBench is growing faster right now.
They split the axes: AgentBench leads on model freedom, teaql-agent-kit on context depth and noise control.
Stars and commit dates come from our own daily tracking. The six axes are read off each project's documentation by our review pipeline, so they describe what a project says about itself, not what we measured in its code.
Where they stand today
Provides an end-to-end, multi-environment benchmark specifically for evaluating LLMs as interactive agents with containerized task workers and function-calling support.
- Stars
- 3.8k
- Tracked growth
- +4.5%
- Maturity
- ●●●●●
- Last commit
- 265d ago
- Language
- Python
- License
- Apache-2.0
- Cost to run
- Your OpenAI API key required
Provides a TEAQL-focused, auditable evaluation harness that measures software-engineering discipline and token-efficiency for coding agents rather than offering general-purpose agent automation.
- Stars
- 2.8k
- Tracked growth
- +0.5%
- Maturity
- ●●●●●
- Last commit
- 5d ago
- Language
- TypeScript
- License
- MIT
- Cost to run
- Your API key or model costs may apply
Six axes, head to head
Each axis runs 0 to 5. The label under a score is what that project's own docs claim, not a category average.
| Axis | AgentBench | teaql-agent-kit |
|---|---|---|
Context depth How much of your codebase it sees before it answers: the open diff, the diff plus related files, or the whole repository. | ●●●●● Env/task only | ●●●●● Diff + related files |
Noise control How it keeps output volume down — severity thresholds, deduplication, incremental runs over new commits only. | ●●●●● None mentioned | ●●●●● Guides & checkpoints |
Customization How far it bends to your team: custom rules, prompts, style guides, per-path config. | ●●●●● Config files | ●●●●● Config & prompts |
Privacy Whether your code stays on your own infrastructure: fully local, self-hostable, or cloud API only. | ●●●●● API key cloud | ●●●●● Cloud via API key |
Model freedom Whether you can point it at any provider, or it is wired to one. | ●●●●● Configurable provider | ●●●●● Single fixed provider |
Setup ease What it takes to get a first useful run out of it. | ●●●●● Config + API key | ●●●●● Manual setup |
Which one to pick
Pick AgentBench if…
Comprehensive evaluation — choose AgentBench when you need an end-to-end, multi-task benchmark to measure and compare LLMs as autonomous agents across diverse interactive environments.
- Model freedom: Configurable provider (3/5 against 1/5)
Pick teaql-agent-kit if…
Evaluation-first — pick this when you need reproducible, auditable benchmarks of coding agents working with TEAQL contracts, explicit guardrails, and token-efficiency measurement.
- Context depth: Diff + related files (3/5 against 1/5)
- Noise control: Guides & checkpoints (3/5 against 1/5)
What people want from each one
THUDM/AgentBench
Hacker News: AgentBench: Evaluating LLMs as Agents drew 2 points and 1 comments.
Questions people ask
Is AgentBench better than teaql-agent-kit?
They split the axes: AgentBench leads on model freedom, teaql-agent-kit on context depth and noise control. AgentBench is worth picking when comprehensive evaluation — choose AgentBench when you need an end-to-end, multi-task benchmark to measure and compare LLMs as autonomous agents across diverse interactive environments.
Which of AgentBench and teaql-agent-kit keeps my code private?
AgentBench: API key cloud (3/5). teaql-agent-kit: Cloud via API key (3/5).
What does each one cost to run?
AgentBench: Your OpenAI API key required. teaql-agent-kit: Your API key or model costs may apply.
Full profiles: THUDM/AgentBench and teaql/teaql-agent-kit. Everything else in Evals & benchmarks.