triage

triage

Adversarial triage of raw security-scanner output. Does four jobs: verify each finding is real, deduplicate across runs and scanners, rank survivors by derived exploitability rather than the scanner's claimed severity, and route each to a component owner. Output is a short, ranked, owned list instead of a raw dump.

Invoke with /triage <findings-path> [--auto] [--votes N] [--repo PATH] [--fp-rules FILE].

Arguments (parse from $ARGUMENTS; positional $1/$2 expansion is not stable across runtimes):

• findings path (first positional, required): a JSON file, a directory of JSON files, a VULN-FINDINGS.json, an INCIDENTS.json, a pipeline results/<target>/<ts>/ directory, or a markdown report.
• --auto: skip the interview and use defaults. Default mode is interactive.
• --votes N: verifier votes per finding (default 3; use 1 for a quick pass, 5 for high-stakes batches).
• --repo PATH: path to the target codebase, read-only (default cwd). Verification needs source access; the skill stops with an error if the cited files aren't reachable.
• --fp-rules FILE: append the contents of FILE to the verifier's exclusion-rule list (Phase 3a). Use for org-specific precedents: "we use Prisma ORM everywhere — raw-query SQLi only", "k8s resource limits cover DoS", etc. Plain text, one rule per line or paragraph.
• --fresh: ignore any existing checkpoint in ./.triage-state/ and start from Phase 0. Without this flag the skill resumes from the last completed phase if a checkpoint is present.

Tools: Read, Glob, Grep, Write, Task, AskUserQuestion. Bash is permitted only for git, find, wc, ls, jq, and python3 .claude/skills/_lib/checkpoint.py (checkpoint I/O).

Do not execute target code. No building, running, installing dependencies, or sending requests. A proof-of-concept that accidentally works against something real is unacceptable, and "couldn't write a working PoC" is weak evidence of non-exploitability. Every conclusion comes from reading source. This applies to the orchestrator and every subagent; include the constraint in every Task prompt. For high-confidence HIGH findings, recommend a human-built PoC as a follow-up instead.

Do not reach the network. No package-registry lookups, CVE-database queries, or upstream-commit fetches.

Checkpointing (runs before Phase 0 and after every phase)

On large finding batches a full run can exhaust context or hit rate limits mid-way — particularly Phase 3, which spawns candidates × votes verifiers. Phase state persists to ./.triage-state/ so a fresh /triage session can resume without re-asking the interview or re-spawning verifiers.

All checkpoint I/O goes through python3 .claude/skills/_lib/checkpoint.py (atomic writes, JSON-validated). Never use the Write tool for progress.json directly. Never pass payload via heredoc or stdin; target-derived strings could collide with the heredoc delimiter and break out to shell. The Write→--from pattern keeps repo-derived bytes out of Bash argv.

State files in ./.triage-state/:

• progress.json — single source of truth for resume position: {"status": "running"|"complete", "phase_done": N, "shards_done": [...]}. Resume decisions read ONLY this file, never a glob of phase*.json or shard files (stale files from a prior run must not be trusted).
• phaseN.json — data payload for phase N (schemas at the tail of each phase section below).
• _chunk.tmp — transient payload buffer; overwritten before every save/shard/append call.

Start of run — resume check. Bash: python3 .claude/skills/_lib/checkpoint.py load ./.triage-state

• status == "absent" OR "complete", OR --fresh in $ARGUMENTS → fresh start. Bash: python3 .claude/skills/_lib/checkpoint.py reset ./.triage-state, then proceed to Phase 0.
• status == "running" with phase_done == N → resume. Read ./.triage-state/phase0.json through phaseN.json in order (and any shard_*.json files listed in shards_done), merging keys into working state (later files override earlier — checkpoints may be deltas). Print Resuming from checkpoint: Phase N complete (./.triage-state/phaseN.json), and skip directly to Phase N+1.

End of every phase N. Two tool calls:

1. Write tool → ./.triage-state/_chunk.tmp containing the phase's output JSON (schema at the tail of each phase section).
2. Bash → python3 .claude/skills/_lib/checkpoint.py save ./.triage-state <N> <name> --from ./.triage-state/_chunk.tmp

End of run. After writing TRIAGE.json and TRIAGE.md, Bash: python3 .claude/skills/_lib/checkpoint.py done ./.triage-state 6

Phase 0: Mode select and interview

0a. Parse arguments

From $ARGUMENTS: extract the findings path (first positional), --auto flag, --votes N (default 3), --repo PATH (default .), --fp-rules FILE (default none). If no findings path was given, ask for one and stop. If --fp-rules was given, Read the file now and carry its contents as context.extra_fp_rules for injection into the Phase 3a verifier prompt.

0b. Interactive mode (default): interview the user

Unless --auto was passed, use AskUserQuestion to gather context that shapes verification and ranking. Batch into one or two calls of up to four questions. Expect free-text answers via "Other"; the multiple-choice options are prompts, not constraints.

Round 1 (single AskUserQuestion call):

1. Environment & trust boundary (header Environment, single-select) What kind of system are these findings from, and where does untrusted input enter it? Options: Internet-facing web service (HTTP is untrusted), Internal service (callers are authenticated peers), Library / SDK (caller is the trust boundary), CLI / batch tool (operator inputs trusted, file inputs not), Embedded / firmware (physical access in scope). Reachability is judged against this boundary; "command injection from env var" is a true positive in a multi-tenant web service and a rule-8 false positive in an operator CLI.

2. Threat model (header Threat model, multi-select) What does a worst-case attacker look like for this system, and what must never happen? Free text is best. Options: Unauthenticated remote code execution, Tenant-to-tenant data leakage, Privilege escalation to admin, Supply-chain compromise of downstream users, Denial of service against a paid SLA, Compliance-scoped data exposure (PII / PCI / PHI). Phase 4 boosts findings that map onto a stated threat.

3. Scoring standard (header Scoring, single-select) How should severity be expressed in the output? Options: Derived HIGH/MEDIUM/LOW from preconditions (default), CVSS v3.1 vector + base score, CVSS v4.0 vector + base score, OWASP Risk Rating (likelihood x impact), Organization bug-bar (describe in Other). The precondition rule is always computed; this controls what severity_label additionally shows.

4. Noise tolerance (header Noise tolerance, single-select) When verifiers disagree, which way should ties break? Options: Precision: drop anything not majority-confirmed (fewer FPs, may miss real bugs), Recall: keep split votes as needs_manual_test (more to review, fewer misses), Ask me per-finding when it happens.

Round 2 (conditional): if the threat-model answer was empty or generic, or the scoring answer was Organization bug-bar, ask one targeted follow-up.

Record the answers as a context dict carried through every phase and echoed in the output under triage_context.

0c. Auto mode defaults

When --auto is set, do not call AskUserQuestion. Use:

• Environment: Unknown. Treat any externally-reachable entry point as untrusted; flag trust-boundary assumptions explicitly in rationale.
• Threat model: empty (no boost).
• Scoring: derived HIGH/MEDIUM/LOW.
• Noise tolerance: precision.

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

{"phase": 0, "context": {mode, environment, threat_model, scoring, noise_tolerance, votes_per_finding,

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 0 interview --from ./.triage-state/_chunk.tmp On resume past Phase 0, the interview is not re-asked; context is restored from this file.

Phase 1: Ingest and normalize

Turn the input into a flat findings[] list with stable ids, regardless of source format.

1a. Detect input shape

Inspect the findings path:

• Directory: Glob for **/*.json and **/*.jsonl. Recognized containers, in priority order:
  • VULN-FINDINGS.json (a {findings: [...]} container): read .findings[].
  • INCIDENTS.json (the /dnr-hunt / /dnr-respond output): read .incidents[]; each incident is one finding. Map the nested fields — vuln.file → file, vuln.line → line, vuln.type → category, vuln.function → function, description ← vuln.mechanism plus impact — and keep id, title, verdict, and poc; the 1b aliases route poc → exploit_scenario, and a confirmed_exploited verdict means the incident arrived execution-verified, not statically guessed — verifiers should weight that. The top-level ruled_out array is not findings — don't ingest it. Incidents with no vuln.file follow the 1b missing-file rule like any other finding.
  • reports/bug_*/report.json or reports/manifest.jsonl (this repo's pipeline output): one finding per bug_NN. Map crash.crash_type → category, verdict.severity_rating → severity, the prose report → description, crash file from the ASAN top frame → file/line.
  • found_bugs.jsonl: one finding per line.
  • Any other *.json whose top level is a list of objects, or an object with a findings/results/issues/vulnerabilities array: that array.
• Single .json / .jsonl file: same recognition as above.
• Markdown / text: split on level-2/3 headings or --- rules; for each section, extract file, line, category, severity, description by pattern (File:, Line:, Severity: labels or path:NN spans). Best-effort; mark source_format: "markdown_heuristic".

If nothing parseable is found, stop and report what was seen.

1b. Normalize fields

For each raw record, build a finding dict. Pull what's present; never guess what's absent. Field map (source-key aliases → canonical):

Attach to every finding:

• id: f001, f002, ... in ingest order. If scanner_confidence is present on most findings, order ingest by it descending so high-signal findings get verified (and surface in partial output) first; otherwise keep source order. This is a scheduling prior only — it does not affect verdicts.
• source: relative path of the file it came from, plus source format.
• missing_fields: list of canonical fields that were absent. If file is missing or does not resolve under --repo, the finding is unlocatable: it skips dedup and verification and is emitted directly with verdict: false_positive, verify_verdict: needs_manual_test, confidence: 0, refute_reasons: ["doesnt_exist"], rationale: "no source location in input; cannot verify statically; human review required". Never emit a confident verdict on a finding you could not locate, and never let it absorb or be absorbed by dedup.

1c. Locate the target codebase

Resolve --repo (default cwd). For the first 5 findings with a file, check the path resolves under the repo. Try, in order: (a) repo/file as-given; (b) file as an absolute or cwd-relative path; (c) repo/file with common prefixes stripped from file (src/, app/, ./, or the repo's own basename, e.g. harness/grade.py with --repo harness). Record which resolution worked and apply it to every finding. If none resolve, stop: tell the user verification needs source access and the cited files aren't reachable, and suggest a --repo value based on the longest common suffix you can see.

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 1 ingest --from ./.triage-state/_chunk.tmp

Phase 2: Deduplicate (before verification)

Collapse repeats so duplicate findings don't each burn N verifiers.

2a. Deterministic pass (inline, no subagent)

Cluster findings where all of:

• same file (after path normalization), AND
• same category (case-insensitive, punctuation stripped), AND
• line numbers within 10 of each other. Both-missing matches; one-side- missing does NOT (a line-less record must not absorb a located one).

Within each cluster, the canonical is the record with the fewest missing_fields; ties break to lowest id. Every other member gets verdict: duplicate, duplicate_of: <canonical id>, and is removed from the working set. Record duplicate ids on the canonical as absorbed: [...].

2b. Semantic pass (one subagent, only if >1 cluster survives)

Spawn ONE Task with subagent_type: "general-purpose" and this prompt:

Parse GROUP: lines. For each, mark the listed dup ids with verdict: duplicate, duplicate_of: <canonical>, append them to the canonical's absorbed, and drop them from the working set.

Carry forward candidates[] = the surviving canonicals.

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 2 dedup --from ./.triage-state/_chunk.tmp

Phase 3: Verify

For each candidate, N independent adversarial verifiers re-derive the claim from the code and vote. Each verifier's stance is "find any reason this is wrong." Each starts from the code at the cited location, not the scanner's description, and never sees the other verifiers' reasoning (shared context propagates blind spots).

3a. Verifier prompt (assemble once, reuse for every spawn)

3b. Spawn N verifiers per candidate, all in one message

For each finding in candidates[], build N Task calls (N = --votes, default 3) with subagent_type: "general-purpose" and description: "verify {id} vote {k}/{N}".

Always set subagent_type; never fork. Omitting subagent_type forks the orchestrator, and a fork inherits the full conversation context: every other finding's description, the scanner's prose, and any prior verifier results. That defeats verifier independence and re-introduces the inherited-framing failure mode this phase exists to prevent. Each verifier must start with a fresh, empty context and receive only the 3a prompt plus the single finding under review. The same applies to the ranking subagents in 4a.

Each prompt is the verifier prompt from 3a with this block appended:

Put all verifier Task calls in a single assistant message so they run concurrently. Do not set run_in_background; you need the final text, not an async handle. If len(candidates) * N exceeds ~40, shard into sequential batches of ~40, but keep each batch a single message.

Prompt size at scale. The 3a prompt is ~1200 words. When candidates * votes > ~50, use this compact form instead (same procedure and output contract, prose stripped):

Findings with a file but no line get one verifier vote regardless of --votes (a file-level sweep is expensive and doesn't benefit from voting).

If any Task call returns status: "async_launched" instead of the verifier's text, the runtime backgrounded it (some runtimes do this automatically for large parallel batches). Pick one recovery and use it for the whole batch:

• If completion notifications arrive in your conversation: parse each verifier's VERDICT block from its notification result as it lands. Do not end your turn until every vote is accounted for.
• If notifications do not arrive: do not poll transcript files. Re-spawn the missing verifiers in a fresh Task batch (smaller shard size, e.g. 10) and use the synchronous results. The same recovery applies to the dedupe subagent in 2b and the ranking subagents in 4a.

3c. Tally votes

For each candidate, parse the trailing block from each of its N verifiers (tolerate code fences and whitespace). If a verifier errored, timed out, or produced no parseable VERDICT block, re-spawn it once. If the retry also fails, count that vote as cannot_verify with confidence: 0 and note "verifier_error" in refute_reasons. The remaining N-1 votes still decide.

Build:

• vote_breakdown: {"true_positive": x, "false_positive": y, "cannot_verify": z}
• confidence: mean CONFIDENCE across votes that agree with the majority, rounded to one decimal.
• exclusion_rule: the modal EXCLUSION_RULE among FALSE_POSITIVE votes, else null.
• refute_reasons: sorted unique REFUTE_REASON values from FALSE_POSITIVE votes.
• first_links: unique FIRST_LINK values across all votes (reachability audit trail).
• rationale: the RATIONALE from the highest-confidence vote on the winning side, verbatim.

Decide verdict:

• Majority TRUE_POSITIVE → verdict: true_positive. Proceeds to Phase 4.
• Majority FALSE_POSITIVE → verdict: false_positive. Skips Phase 4.
• No majority (tie, or majority CANNOT_VERIFY):
  • Noise tolerance precision → verdict: false_positive; append "(split vote, dropped under precision policy)" to rationale.
  • Noise tolerance recall → verdict: true_positive with verify_verdict: needs_manual_test. Proceeds to Phase 4.
  • Noise tolerance ask → collect all split findings and present them in one AskUserQuestion call at the end of Phase 3 (header: id + title, options: keep / drop), then apply the user's choices.

Build confirmed[] = candidates with verdict == true_positive.

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 3 verify --from ./.triage-state/_chunk.tmp

This is the most expensive checkpoint. When len(candidates) * votes exceeds ~40 and verifier spawns are sharded into sequential batches, additionally checkpoint per candidate as its votes are tallied:

1. Write tool → ./.triage-state/_chunk.tmp = that finding's post-tally dict.
2. Bash: python3 .claude/skills/_lib/checkpoint.py shard ./.triage-state <id> --from ./.triage-state/_chunk.tmp

On resume at phase_done == 2, the Phase-3 entry point reads progress.json:shards_done (default [] — do not glob shard files on disk; stale shards from a prior run may exist), loads the corresponding shard_{id}.json files, and spawns verifiers only for candidates[] ids from phase2.json that are NOT in shards_done. Once every candidate is in shards_done, write the consolidated phase3.json checkpoint as above.

Phase 4: Rank by exploitability (confirmed findings only)

Recompute severity from preconditions and reachability rather than category name, and judge the scanner's claimed severity separately. Verification and severity are independent judgments; "this is real" must not inflate into "this is critical."

4a. Ranking prompt

Spawn one Task per confirmed finding (subagent_type: "general-purpose", all in one message) with:

4b. Merge

For each confirmed finding, parse the block and attach preconditions (replacing any scanner-supplied list), access_level, severity (recomputed), severity_label, threat_match, severity_alignment, verify_verdict, and append RANK_RATIONALE to rationale (separated by a blank line from the Phase-3 rationale).

For findings that did NOT reach Phase 4 (false_positive, duplicate, unlocatable): set severity: null, verify_verdict: null, severity_alignment: null, preconditions: [].

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 4 rank --from ./.triage-state/_chunk.tmp

Phase 5: Route

Tag each confirmed true-positive with the most specific component or owner inferable. For each finding in confirmed[], stop at the first hit:

1. CODEOWNERS / OWNERS. Grep --repo for CODEOWNERS, OWNERS, .github/CODEOWNERS, docs/CODEOWNERS. If found, match the finding's file against its patterns (last match wins). Hint: "CODEOWNERS: <pattern> -> <owner(s)>".
2. git log. If --repo is a git checkout, run git -C {REPO} log --format='%an' -n 50 -- "{file}" | sort | uniq -c | sort -rn | head -3. Hint: "top committer: <name> (<n>/<total> recent commits); no CODEOWNERS entry".
3. Module fallback. Hint: "component: <top-level dir of file>/; no CODEOWNERS or git history".

Attach as owner_hint. State the source so confidence is clear; a bare username is less useful than "component: auth/; no CODEOWNERS entry; top committer jsmith (14/20 recent commits)". For non-true-positive findings, set owner_hint: null.

Checkpoint: Write tool → ./.triage-state/_chunk.tmp:

Then Bash: python3 .claude/skills/_lib/checkpoint.py save ./.triage-state 5 route --from ./.triage-state/_chunk.tmp

Phase 6: Output

6a. Sort

Order all findings by:

1. verdict: true_positive, then duplicate, then false_positive.
2. Within true positives: severity HIGH > MEDIUM > LOW, then confidence descending, then severity_alignment descending.
3. Within others: original id.

6b. Write ./TRIAGE.json

Every input finding appears exactly once (duplicates reference their canonical via duplicate_of). Do not silently drop anything. Do not print this JSON to the terminal; write to file only.

6c. Write ./TRIAGE.md

Reviewer-facing report. Build it incrementally. Do NOT emit the whole file in one Write. One chunk per finding; a stalled chunk loses that one section, not the file.

Step 1 — header. Write tool → ./TRIAGE.md (clobbers any prior file) containing only the title block, summary, and ## Act on these heading:

Step 2 — per finding. For each true_positive in severity order:

1. Write tool → ./.triage-state/_chunk.tmp containing ONE finding's section:

2. Bash: python3 .claude/skills/_lib/checkpoint.py append ./TRIAGE.md --from ./.triage-state/_chunk.tmp

Repeat for each true_positive.

Step 3 — footer. Write tool → ./.triage-state/_chunk.tmp containing the Dropped table, then checkpoint.py append it the same way:

Checkpoint (final): Bash: python3 .claude/skills/_lib/checkpoint.py done ./.triage-state 6 The next invocation's resume check sees status == "complete" and starts fresh.

6d. Terminal summary

Under ~12 lines:

Testing this skill

Smoke test (five-finding fixture: 2 real, 1 dup, 2 FP):

Expected: f001 and f003 confirmed; f002 duplicate of f001; f004 dropped (misread_code: it's a read buffer, not a randomness source); f005 dropped (already_handled: there is a null check at line 68).

Or against pipeline output:

Hand-check a sample of TRUE_POSITIVE/HIGH results (the first_links should point at real call sites) and a sample of FALSE_POSITIVE rejects (the exclusion_rule or refute_reasons should be defensible).

Design notes

• Checkpoints are per-phase JSON, not conversation state. The pipeline's --resume <session_id> (docs/pipeline.md) restores transcript history but doesn't help when the orchestrator's context window itself fills; file-backed checkpoints let a brand-new session pick up from the last completed phase. ./.triage-state/ is scratch — add to .gitignore.
• Dedupe runs before verify to cut verifier spend by the duplication factor (often 2-4x on multi-scanner input) at the cost of one cheap subagent.
• Semantic dedupe is one agent, given only id/file/line/category/title: enough to cluster, not enough to leak one scanner's reasoning into another finding's verification.
• Bash is allowed narrowly for git log (owner hints), jq/find (ingest), and python3 .claude/skills/_lib/checkpoint.py (state I/O). The actual safety property is "no execution of target code," which is preserved.
• CANNOT_VERIFY exists so verifiers aren't forced into a false binary. It maps to needs_manual_test under recall policy and to a drop under precision policy.
• Threat-model boost is capped at one step so a stated threat can't re-inflate a LOW back to HIGH and defeat the precondition rule.
• severity_label is separate from severity. Sorting always uses the precondition-derived HIGH/MEDIUM/LOW; the label is presentation-layer for whatever standard the reviewer's tooling expects.
• Pipeline report.json ingest is best-effort. Those reports describe ASAN crashes with prose exploitability analysis rather than the file/line/category shape static verifiers expect. Expect more needs_manual_test verdicts on that input than on static-scanner JSON.
• Sharding at ~40 parallel Tasks is a conservative ceiling for typical agent-spawn limits; tune up if your runtime allows.
• No network, deliberately. CVE-database enrichment and upstream-fix checks would help ranking but break the air-gapped-review property.