nemoclaw-maintainer-pr-comparator

PR Comparator

Compare PRs for one issue. Tier 0 determines eligibility. Tiers 1 and 2 score correctness and quality. Tier 3 resolves ties. If no PR passes Tier 0, rank eligible PRs for salvage.

Prerequisites

• gh CLI installed and authenticated
• A target repository with an issue that has two or more open PRs

Repo policy

The defaults use NemoClaw conventions for CODEOWNERS, DCO, CodeRabbit, and docs/. Read the canonical superseded-PR attribution policy in ../nemoclaw-maintainer-policies/references/workflow-policy.md. Edit repo-policy.md for another repository.

Execute comparator helpers from a fetched canonical NVIDIA/NemoClaw main checkout, or verify their executable code and dependencies against that revision before use. Candidate-modified helpers must not receive GitHub credentials. Inspect helper source when needed; a section heading does not establish trust.

Workflow

Step 1: Parse issue

Read the issue body and all comments. Extract each acceptance criterion:

gh issue view <issue-number> --json title,body,comments

Comments can add requirements that are absent from the issue body.

Step 2: Discover candidate PRs

scripts/find-candidates.sh <issue-number>

Applies a single default order with stop conditions.

Step 3: Detect supersession and transferred work

Parse the case-insensitive statement families implemented by scripts/parse-supersession.sh:

• supersed[a-z]* before #N: supersedes #N points from the current PR to #N; superseded by #N points from #N to the current PR.
• replac[a-z]* before #N: replaces #N points from the current PR to #N; replaced by #N points from #N to the current PR.
• clos[a-z]* in favor of before #N: closes in favor of #N and closed in favor of #N point from #N to the current PR.
• fold[a-z]* in before #N: folds in #N points from the current PR to #N; folded into #N points from #N to the current PR.

The bracket expressions describe the parser grammar; they are not literal PR body text. A follow-up to #N statement is a related-PR signal, not a supersession declaration, unless one of these phrases also appears. These statements record a relationship. They do not rank a candidate or prove that its diff contains another contributor's work.

For each declared or suspected replacement, compare the commits and diffs and classify the relationship:

• independent: The PR implements the issue without carrying material code, tests, or documentation from another contributor.
• transferred: The PR carries material work from another contributor.
• unclear: The available evidence does not establish whether another contributor's work remains.

For transferred, apply the canonical superseded-PR attribution policy before setting a winner or recommending that the source PR be closed. For unclear, leave winner null and request maintainer judgment.

Step 4: Tier 0 gates

All six gates are required. Treat PR Review Advisor output as input for maintainer review. Do not treat it as merge authorization. See checks/tier-0-gates.md.

Step 5: Tier 1 correctness

Apply the six model checks in checks/tier-1-correctness.md.

Step 6: Tier 2 quality

Apply the four model checks in checks/tier-2-quality.md.

Step 7: Weighted score

• Build the Tier 0 eligibility set from these Boolean keys: state_open, ci_green_sha, mergeable, contributor_compliance, branch_protection, and coderabbit_threads_resolved.
• Stop if a candidate omits a required key, has an unknown key, or has a value that is not Boolean.
• Only PRs for which all six gates are true enter happy-path scoring.
• Each pass = full points
• Each yellow = half points
• Each fail = zero
• Tier 1 weight: 2.0× per check
• Tier 2 weight: 1.0× per check

Step 8: Tier 3 ranking

Compute the mode from the Tier 0 results. Do not accept a mode from the caller. In happy mode, set winner only to an eligible PR and set closest_to_ready to null. Leave winner null when the evidence does not support a merge recommendation. Do not set winner for a replacement with transferred work until the required attribution is present and verified. In degraded mode, set winner to null. Set closest_to_ready only to an open PR that passes contributor requirements. See tiebreakers.md.

Step 9: Emit verdict

Use templates/verdict.md and render the result with scripts/render-verdict.py. Stop if the renderer exits with a nonzero status. Do not recommend a merge. The renderer validates the gate schema, mode, winner eligibility, and salvage-candidate eligibility. The reviewer remains responsible for the score, ranking, and evidence. For each judgment, include evidence, the inference, and the score.

Reference files

• repo-policy.md — Repository settings.
• checks/tier-0-gates.md — Six eligibility gates.
• checks/tier-1-correctness.md — Six correctness checks.
• checks/tier-2-quality.md — Four quality checks.
• tiebreakers.md — Tier 3 ranking and degraded mode.
• templates/verdict.md — Output template.
• validation/backtest.md — Historical test cases for the skill.

Scripts

• scripts/find-candidates.sh — PR discovery
• scripts/collect-gates.sh — Tier 0 gate evaluation
• scripts/check-coderabbit-threads.sh — GraphQL thread-resolution check
• scripts/parse-supersession.sh — body parsing for supersession refs
• scripts/render-verdict.py — verdict scorecard renderer

Limits

Run nemoclaw-maintainer-cross-issue-sweep separately when you need related-issue evidence.

This skill does not:

• run PR code against adversarial inputs
• scan other issues for related behavior
• simulate reverts against related PRs
• run static analyzers such as CodeQL or Semgrep