Toolbox/Running agents in production

Observability & tracing

tracing, logging and debugging agents

8 repositories, most starred first

/

State of the category

raga-ai-hub/RagaAI-Catalyst raga-ai-hub/RagaAI-Catalyst is the clear leader; if you want a single self‑hosted platform for multi‑agent tracing, timeline analytics, evaluation and guardrails, it’s the most mature choice. Pick other projects only when you need lighter instrumentation, OTel-native traces, production-to‑CI replay tests, per‑line provenance, or fully local developer UIs.

Lightweight cross‑framework replay

Teams that run many different agent frameworks need minimal instrumentation, session replays and LLM cost tracking without replacing their stack, so lighter, framework‑agnostic tools matter for adoption and fast debugging.

→ agentops

OpenTelemetry‑native tracing

Organizations standardized on OpenTelemetry want AI traces and evaluations to live in the same telemetry stack to reuse existing collectors, dashboards and automation.

→ openlit

Replayable production tests

Promoting real failing traces into hermetic, replayable CI tests prevents regressions and makes debugging reproducible, which is critical for shipping agents safely.

→ Tracely, Tracely-ai

Per‑step provenance & blame

Teams that need auditable, content‑addressed provenance and per‑line ‘blame’ for generated changes require finer-grained VCS‑style tracking that most observability tools don’t provide.

→ re_gent

Local dev UX & plan review

Some teams prioritize fully local, privacy‑first developer experiences and real‑time plan review or playful visualizations over cloud dashboards, so lightweight local UIs remain attractive.

→ pixtuoid, agenttrail

Find your fit

Which one matches your setup?

Answer any of the questions — the shortlist updates as you go. Recommendations come from the capability passports below, nothing else.

Where should reviews happen?

Can code leave your infrastructure?

What matters most?

Model access?

Pick at least one answer to get a shortlist.

Side by side

Comparison matrix

Axes are extracted from each project's docs by our review pipeline; the maturity score is computed from stars, growth and commit activity — not an opinion. Click a column to sort.

	Runs in	Models	Context					Cost to run
openlit/openlit ↗ ⭐ 2.7k +19/7d	Web appCLICoding-agent plugin	BYOKOllama / local	Full-stack traces	Rule engine filters	●●●●●	Self-hostable	●●●●●	Self-hosted; you pay for LLM/provider API usage.
AgentOps-AI/agentops ↗ ⭐ 5.8k +6/7d	CLIWeb app	BYOKOpenAIAnthropicFixed provider	Runtime tracing	None mentioned	●●●●●	Self-hostable	●●●●●	AgentOps API key for hosted dashboard; can self-host
Jwuthri/Tracely ↗ ⭐ 1.2k	GitHub ActionCICLIWeb app	BYOKOpenAIAnthropic	No repo context	Clustering & gating	●●●●●	Self-hostable / local	●●●●●	Self-hosted; CI replays use recorded fixtures (no model spend).
Jwuthri/Tracely-ai ↗ ⭐ 1.2k	GitHub ActionCICLIWeb app	BYOKOpenAIAnthropic	Per-run only	Clustering & gating	●●●●●	Self-hostable	●●●●●	CI replay $0; live judges use your model API key.
sodiumsun/agenttrail ↗ ⭐ 639	CLIWeb app	AnthropicOpenAIFixed provider	Whole-repo analysis	Minimal signals	●●●●●	Fully local	●●●●●	Free, local
IvanWng97/pixtuoid ↗ ⭐ 468 +8/7d	CLI	OpenAIAnthropicFixed provider	Session events only	None mentioned	●●●●●	Fully local	●●●●●	Free, local
raga-ai-hub/RagaAI-Catalyst ↗Leader ⭐ 16.2k +11/7d	CLIWeb appCI	BYOK	Diff/file only	Configurable thresholds	●●●●●	Self-hostable	●●●●●	Requires RagaAI account; external LLM usage billed to your provider.
regent-vcs/re_gent ↗ ⭐ 789 +3/7d	CLIIDECoding-agent plugin	AnthropicOpenAIFixed provider	Workspace snapshot	Ignore & dedupe	●●●●●	Self-hostable	●●●●●	Free local CLI; external model/API usage may incur costs

At a glance

Capability profiles

Six axes, 0–5 each. The shape tells you the strategy: a wide hexagon is a generalist, a spike is a specialist.

openlit/openlit⭐ 2.7k

OpenLIT provides an OpenTelemetry-native, open-source AI observability platform combining traces, built-in evaluations, a rule engine, prompt management and guardrails in one stack.

AgentOps-AI/agentops⭐ 5.8k

Provides end-to-end agent session replays, LLM cost tracking and debugging across many agent frameworks with minimal instrumentation.

Jwuthri/Tracely⭐ 1.2k

Promotes real failing production traces into hermetic, replayable regression tests that run offline in CI and block PRs.

Jwuthri/Tracely-ai⭐ 1.2k

Promotes real failing production traces into hermetic, replayable regression cases that run offline in CI and block PRs, instead of relying on hand-authored datasets.

sodiumsun/agenttrail⭐ 639

Provides a local, real-time observability map of coding agents' plans, tool calls, and file edits without any cloud service, account, or telemetry.

IvanWng97/pixtuoid⭐ 468

Visualizes multiple coding agents as a whimsical, glanceable pixel-art terminal 'office' with animated coworkers and per-agent session telemetry.

raga-ai-hub/RagaAI-Catalyst⭐ 16.2k

Combines agent/LLM tracing, evaluation, guardrails and red‑teaming with a self‑hosted dashboard and execution-timeline analytics for multi-agent debugging.

regent-vcs/re_gent⭐ 789

Provides content-addressed, per-step audit trails and per-line 'blame' tied to the exact prompt and conversation that produced a change—something traditional VCS and most tools do not track.