evals-skills vs PostTrainBench
evals-skills is much bigger: 1.7k stars against 611. Over the days we have tracked them evals-skills moved +268.1% and PostTrainBench +26.2%, so evals-skills is growing faster right now.
They split the axes: evals-skills leads on customization and privacy and setup ease, PostTrainBench on context depth and model freedom.
Stars and commit dates come from our own daily tracking. The six axes are read off each project's documentation by our review pipeline, so they describe what a project says about itself, not what we measured in its code.
Where they stand today
Provides modular, reusable agent 'skills' (not just benchmarks), including an error-discovery skill that builds a dependency-free single-file review app for guided failure-mode analysis.
- Stars
- 1.7k
- Tracked growth
- +268.1%
- Maturity
- ●●●●●
- Last commit
- 6d ago
- Language
- mixed
- License
- Apache-2.0
- Cost to run
- Free, local
Measures autonomous CLI agents' ability to post-train base LLMs within a 10‑hour H100 budget, evaluating agent-driven R&D rather than only inference.
- Stars
- 611
- Tracked growth
- +26.2%
- Maturity
- ●●●●●
- Last commit
- 29d ago
- Language
- Python
- License
- MIT
- Cost to run
- Your API keys + H100 GPU (cluster or rented)
Six axes, head to head
Each axis runs 0 to 5. The label under a score is what that project's own docs claim, not a category average.
| Axis | evals-skills | PostTrainBench |
|---|---|---|
Context depth How much of your codebase it sees before it answers: the open diff, the diff plus related files, or the whole repository. | ●●●●● Single-file analysis | ●●●●● Whole-repo access |
Noise control How it keeps output volume down — severity thresholds, deduplication, incremental runs over new commits only. | ●●●●● Severity prioritization | ●●●●● Judge + rules |
Customization How far it bends to your team: custom rules, prompts, style guides, per-path config. | ●●●●● Custom skills/plugins | ●●●●● Configurable CLI options |
Privacy Whether your code stays on your own infrastructure: fully local, self-hostable, or cloud API only. | ●●●●● Fully local | ●●●●● APIs with your key |
Model freedom Whether you can point it at any provider, or it is wired to one. | ●●●●● Fixed provider | ●●●●● BYO key + local models |
Setup ease What it takes to get a first useful run out of it. | ●●●●● One-command install | ●●●●● Container + cluster setup |
Which one to pick
Pick evals-skills if…
Easy setup — one-command install via npx and a local, dependency-free review app for interactive error discovery and building product-specific evals.
- Customization: Custom skills/plugins (5/5 against 3/5)
- Privacy: Fully local (5/5 against 3/5)
- Setup ease: One-command install (5/5 against 2/5)
Pick PostTrainBench if…
Autonomy-focused — use this to benchmark end-to-end agent-driven post-training of base models on an H100 with built-in evaluation tasks and reward‑hacking mitigations.
- Context depth: Whole-repo access (5/5 against 1/5)
- Model freedom: BYO key + local models (5/5 against 1/5)
What people want from each one
aisa-group/PostTrainBench
Questions people ask
Is evals-skills better than PostTrainBench?
They split the axes: evals-skills leads on customization and privacy and setup ease, PostTrainBench on context depth and model freedom. evals-skills is worth picking when easy setup — one-command install via npx and a local, dependency-free review app for interactive error discovery and building product-specific evals.
Which of evals-skills and PostTrainBench keeps my code private?
evals-skills: Fully local (5/5). PostTrainBench: APIs with your key (3/5).
What does each one cost to run?
evals-skills: Free, local. PostTrainBench: Your API keys + H100 GPU (cluster or rented).
Full profiles: ai-evals-course/evals-skills and aisa-group/PostTrainBench. Everything else in Evals & benchmarks.