Toolbox/Applied agents

ML experiment agents

agents that run training experiments, tune and evolve models

17 repositories, most starred first

/

State of the category

SakanaAI/AI-Scientist SakanaAI/AI-Scientist is the de facto leader — the most‑starred, end-to-end autonomous research agent that supports self‑hosting and a wide range of model backends. Before choosing, decide whether you need strict reproducibility/auditing, fully local/on‑prem control, long-lived persistent experiments, low LLM spend, or ML‑specific optimization, because those needs are why alternatives proliferate.

Reproducibility and Audit

Teams that must produce publishable, verifiable results or comply with regulation need deterministic review loops, structured artifacts and audit trails rather than ephemeral agent outputs.

→ Auto-claude-code-research-in-sleep, Agent-Native-Research-Artifact, scholar-loop, Kosmos, Plato-Scientific-Research-Autonomous-Agent

Local-first Privacy

Organizations with sensitive data or strict compliance requirements prefer self‑hostable or fully‑local stacks so experiments can run without cloud API keys or external data leaks.

→ openevolve, EvoScientist, DeepScientist, Kosmos, scholar-loop, Plato-Scientific-Research-Autonomous-Agent, aideml

Long-running Control

Long‑horizon research needs persistent memory, branching experiments and easy human takeover so engineers can inspect, resume and steer multi-day automated science.

→ EvoScientist, DeepScientist, InternAgent, ResearcherSkill, AI-Scientist-v2

Low-cost Operations

Continuous 24/7 experiment automation becomes expensive or brittle if every decision calls an LLM, so projects that minimize LLM usage or add reviewer/audit loops lower cost and operational risk.

→ Auto-claude-code-research-in-sleep, auto-deep-researcher-24x7

Specialized ML Tooling

Many teams need agents tuned for ML workflows—hyperparameter-aware proposals, algorithm discovery, sandboxed execution or interactive sandboxes—rather than a one‑size‑fits‑all autonomous scientist.

→ openevolve, aideml, optim-agent, era, karpathy, ResearcherSkill

Find your fit

Which one matches your setup?

Answer any of the questions — the shortlist updates as you go. Recommendations come from the capability passports below, nothing else.

Where should reviews happen?

Can code leave your infrastructure?

What matters most?

Model access?

Pick at least one answer to get a shortlist.

Side by side

Comparison matrix

Axes are extracted from each project's docs by our review pipeline; the maturity score is computed from stars, growth and commit activity — not an opinion. Click a column to sort.

	Runs in	Models	Context					Cost to run
wanshuiyin/Auto-claude-code-research-in-sleep ↗ ⭐ 15.8k +337/7d	CLICoding-agent plugin	BYOKOpenAIAnthropic	Diff + related files	Severity gating & audits	●●●●●	Cloud APIs (your key)	●●●●●	Your API key or model subscription
algorithmicsuperintelligence/openevolve ↗ ⭐ 7.3k +39/7d	CLI	BYOKOpenAIAnthropicGeminiOllama / local	File + related files	Cascade & novelty filters	●●●●●	Run fully local	●●●●●	Your API key; per-iteration LLM costs (local models nearly free)
EvoScientist/EvoScientist ↗ ⭐ 4.7k +102/7d	CLIWeb app	BYOKOpenAIAnthropicFixed provider	Diff + related files	Basic filtering & gating	●●●●●	Self-hostable	●●●●●	Requires your cloud model API keys (per-use billing)
SakanaAI/AI-Scientist ↗Leader ⭐ 14.5k +34/7d	CLI	BYOKOpenAIAnthropicGeminiFixed provider	Template-level view	Ensemble reviews	●●●●●	Self-hostable (local GPU)	●●●●●	Requires your API keys for cloud models; local GPU needed for open-weight runs.
SakanaAI/AI-Scientist-v2 ↗ ⭐ 7.1k +32/7d	CLI	OpenAIAnthropicGemini	Related files	Config-based controls	●●●●●	Cloud APIs (your keys)	●●●●●	Your API key, per-run (README cites ≈$15–$20 for experiments + ≈$5 for writing with default models)
ResearAI/DeepScientist ↗ ⭐ 3.3k +15/7d	CLIWeb appCoding-agent pluginIDE	OpenAIAnthropicGeminiOllama / localFixed provider	Whole-repo analysis	Human-in-the-loop	●●●●●	Self-hostable	●●●●●	Depends on chosen runner — uses your model provider (API) or local models
WecoAI/aideml ↗ ⭐ 1.5k +11/7d	CLIWeb app	BYOKOpenAIAnthropicGeminiOllama / local	Workspace-wide	Metric-guided pruning	●●●●●	Fully local option	●●●●●	Requires your API key (OpenAI/Anthropic/etc.), or can run with local LLMs.
Optim-Agent/optim-agent ↗ ⭐ 937 +1/7d	CLICoding-agent plugin	OpenAIAnthropicFixed provider	Related files	Validation + fallback	●●●●●	Cloud APIs	●●●●●	Model/agent CLI usage costs (your account)
ARA-Labs/Agent-Native-Research-Artifact ↗ ⭐ 676 +6/7d	CLICoding-agent pluginIDE	OpenAIAnthropicGeminiFixed provider	Manifest + on-demand	Verification & provenance	●●●●●	Cloud APIs (user keys)	●●●●●	Uses your agent/provider API (may be paid).
K-Dense-AI/karpathy ↗ ⭐ 1.5k +6/7d	CLIWeb app	BYOK	Sandbox workspace	None mentioned	●●●●●	Cloud via API key	●●●●●	Your API key; cloud model usage charges.
InternScience/InternAgent ↗ ⭐ 1.4k +10/7d	CLI	OpenAIAnthropicBYOK	Whole-repo access	Configurable filters	●●●●●	Cloud API keys	●●●●●	Your API keys (OpenAI / Anthropic / OpenRouter)
Xiangyue-Zhang/auto-deep-researcher-24x7 ↗ ⭐ 1.3k +8/7d	CLI	BYOKOpenAIAnthropic	Whole-repo analysis	Phase gates & rate-limit	●●●●●	API-key cloud	●●●●●	Your API key; low LLM cost (README reports ~$0.08/day).

Ranked by maturity — 5 more in the full catalog below.

At a glance

Capability profiles

Six axes, 0–5 each. The shape tells you the strategy: a wide hexagon is a generalist, a spike is a specialist. Showing the 8 most established — the rest are in the full catalog.

wanshuiyin/Auto-claude-code-research-in-sleep⭐ 15.8k

Provides a lightweight, skill-based autonomous research workflow with built-in cross-model reviewer loops and deterministic integrity audits (Anti-Autoresearch), targeted at reproducible ML research rather than generic agent tasks.

algorithmicsuperintelligence/openevolve⭐ 7.3k

Combines MAP-Elites, island-based evolution and LLM ensembles to autonomously discover novel, hardware-optimized algorithms with reproducible scientific pipelines.

EvoScientist/EvoScientist⭐ 4.7k

Auto-distills a growing memory/knowledge graph into reusable AutoSkills and a multi-agent research loop, enabling long-running self-evolving AI scientists.

SakanaAI/AI-Scientist⭐ 14.5k

Automates end-to-end scientific discovery by generating hypotheses, running experiments, and producing full LaTeX papers from templates.

SakanaAI/AI-Scientist-v2⭐ 7.1k

End-to-end autonomous scientific discovery: it uses progressive agentic tree search to ideate, run experiments, analyze results, and draft papers without human-authored templates.

ResearAI/DeepScientist⭐ 3.3k

A local-first, long-horizon autonomous research studio that preserves full experiment state, branches, artifacts and lets humans inspect and take over—unlike one-shot cloud chat agents.

WecoAI/aideml⭐ 1.5k

Implements LLM-guided agentic tree-search specifically for ML code optimization, combining iterative patching, built-in evaluation and HTML visualisation.

Optim-Agent/optim-agent⭐ 937

Uses coding agents to produce semantic, context-aware hyperparameter proposals that read code and rationale rather than treating parameters as anonymous coordinates.