eval

/hub:eval — Evaluate Agent Results

Rank all agent results for a session. Supports metric-based evaluation (run a command), LLM judge (compare diffs), or hybrid.

Usage

/hub:eval                           # Eval latest session using configured criteria
/hub:eval 20260317-143022           # Eval specific session
/hub:eval --judge                   # Force LLM judge mode (ignore metric config)

What It Does

Metric Mode (eval command configured)

Run the evaluation command in each agent's worktree:

python {skill_path}/scripts/result_ranker.py \
  --session {session-id} \
  --eval-cmd "{eval_cmd}"

Output:

LLM Judge Mode (no eval command, or --judge flag)

For each agent:

1. Get the diff: git diff {base_branch}...{agent_branch}
2. Read the agent's result post from .agenthub/board/results/agent-{i}-result.md
3. Compare all diffs and rank by:
   • Correctness — Does it solve the task?
   • Simplicity — Fewer lines changed is better (when equal correctness)
   • Quality — Clean execution, good structure, no regressions

Present rankings with justification.

Example LLM judge output for a content task:

Hybrid Mode

1. Run metric evaluation first
2. If top agents are within 10% of each other, use LLM judge to break ties
3. Present both metric and qualitative rankings

After Eval

1. Update session state:

2. Tell the user:
   • Ranked results with winner highlighted
   • Next step: /hub:merge to merge the winner
   • Or /hub:merge {session-id} --agent {winner} to be explicit