mantis-researcher

Mantis Researcher (/mantis-researcher)

System Goal

Resilience Code Auditor. Performs rapid triage and deep-dive reviews of source files to identify boundary checks, preconditions, missing sanitization, and interface violations.

Command Definition

• Command: /mantis-researcher
• Description: Audits production source code files based on the strategy in workspace/plan.json.
• Arguments (optional; supplied by the orchestrator, consumed by Block A):
  • --snapshot_root / SNAPSHOT_ROOT: absolute path to the pinned, read-only code snapshot for this pass. This is the CODE_ROOT that all snapshot-relative path fields resolve against (Block A step 1b).
  • --snapshot_id / SNAPSHOT_ID: the pass snapshot identifier. Used for the sentinel check (Block A step 2) and stamped verbatim into every finding's discovery_commit.
  • --state_root: absolute path to the workspace/ state directory (plan.json, .mantis_state.json, findings/, kb/). State paths are STATE-RELATIVE and are NEVER prefixed with CODE_ROOT (Block A step 3).
  • --target_root (authoritative override, Block A step 1a) is also honored if supplied.
  • All flags absent -> DEGRADED/legacy mode: CODE_ROOT falls back to the current directory, snapshot_pinned is treated as false, and behavior matches today's exactly (no discovery_commit is written).

Input/Output Contract

• Reads:
  • workspace/plan.json (falls back to codebase sweep if missing/empty).
  • workspace/.mantis_state.json (to track current loop pass).
  • referenced Markdown files in "kb_references" (e.g. workspace/kb/entities/*.md).
  • Target source code files.
  • workspace/kb/structural_index/manifest.json (to check structural index availability/status).
  • workspace/helpers/query_structural_index.py (to invoke bounded structural-index queries).
• Writes:
  • Raw finding files to workspace/findings/<uuid>.json (creates workspace/findings/ if missing).
• Preconditions:
  • Target files must be accessible.
• Idempotency Guarantee:
  • Writes new findings as separate files with unique UUIDs. Rely on mantis-dedupe to cluster and merge duplicate findings on subsequent steps.

Instructions

Step 0: Locator Resolution (Snapshot-Aware Path Handling)

Run this BEFORE the numbered research steps below. It fixes the single CODE_ROOT that every target_files / code_paths reference in this stage resolves against, so all sub-agents audit the same pinned snapshot.

Skill-specific notes for the researcher:

• The researcher is a CODE-READING stage, so Block A step 0's findings-only skip does NOT apply here — you MUST resolve CODE_ROOT and honor the sentinel.
• workspace/plan.json target_files and finding code_paths are SNAPSHOT-RELATIVE: resolve them under CODE_ROOT (Block A step 3).
• kb_references, workspace/plan.json, workspace/.mantis_state.json, and everything under workspace/findings/ are STATE-RELATIVE: read/write them under --state_root, NEVER under CODE_ROOT.
• Never write, compile, or generate anything under CODE_ROOT when pinned (Block A step 4).

Perform a thorough memory-safety, logical-correctness, and robustness review of the targeted codebase.

Execute the research stage as follows:

1. Load Reviewing Plan & Context: Read the active pass number from workspace/.mantis_state.json and resolve the current ISO 8601 timestamp. Read the workspace/plan.json file to retrieve the target investigations. If workspace/plan.json is missing or empty, perform a general list of the directories and review any primary source files. If the investigation contains a "kb_references" array, explicitly read those Markdown files (e.g., workspace/kb/entities/auth.md) to gain compounded historical context before you begin auditing the "target_files". Also read active_snapshot from workspace/.mantis_state.json (root, snapshot_id, snapshot_pinned). Hold active_snapshot.snapshot_id in memory: it is the value you will stamp into every finding's discovery_commit (see Findings Schema Format). If active_snapshot is absent or snapshot_pinned is false, you are in DEGRADED/legacy mode — do NOT stop (Block A step 1d); you will simply omit discovery_commit.

2. Sub-Agent Delegation (Wave-Based Swarm Parallelization): If the CLI or agent platform supports spawning sub-agents (e.g., using specialized sub-agent tools or multi-agent orchestrator directives):

   • Do not execute investigations sequentially if sub-agents are supported. Split the investigations in workspace/plan.json into parallel waves to maximize throughput and context efficiency.

   • Wave 1: Lightweight Rapid Triage (Concurrency Peak): Spawn concurrent, lightweight sub-agents (e.g. up to 10-20 in parallel) to sweep all files listed in workspace/plan.json. Each sub-agent should only output a fast classification: {"potentially_flawed": true/false, "reason": "..."}.

   • Wave 2: Deep Security Flaw Hotspot Audits & Parallel Trajectory Search: Collect all files flagged in Wave 1. Spawn a wave of concurrent deep auditor sub-agents (e.g. up to 4-8 in parallel) to focus exclusively on those identified hotspots. For particularly complex files, spawn multiple subagents targeting the same file using either different prompt constraints or a diverse set of less expensive LLMs to explore parallel attack vectors. Rely on the subsequent deduplication stage to merge any overlapping findings.

   • Token Optimization (Distributed Writes): Instruct the Wave 2 sub-agents to generate unique UUIDs and write their findings directly to individual workspace/findings/<id>.json files on disk. Do not ask them to return the full JSON payload in their messages back to you, as aggregating them will blow out your context window. Ask them to only return the list of UUIDs they created.

   • Snapshot Isolation (Wave Pinning) — MANDATORY: Pass the SAME --snapshot_root (CODE_ROOT resolved in Step 0) and the same --snapshot_id value to EVERY Wave-1 and Wave-2 sub-agent, and instruct each of them to obey Block A (resolve target_files/code_paths under that CODE_ROOT, honor the sentinel). Any Wave-2 sub-agent that writes a finding MUST stamp discovery_commit with that snapshot_id, exactly as specified in the Findings Schema Format. Sub-agents MUST NOT run git pull/fetch/ checkout/reset, hg pull/update, repo sync, or any command that changes the working tree or switches revisions — the snapshot is immutable for the whole pass. A sub-agent that cannot see the snapshot must report that, not re-sync.

   • If sub-agents or concurrency are not supported by the current environment, fall back to performing the sweeps and deep-dives sequentially.

   • Structural Index (HINT-only enhancement): When a structural index is available (workspace/kb/structural_index/manifest.json exists), use it to SUPPLEMENT the wave-based swarm above. The structural index decides ORDER, never MEMBERSHIP. It MUST NEVER replace the exhaustive Step-3 call-site sweep.

     • Resolution-first protocol (MANDATORY before any structural query):

       1. Resolve the symbol first: python3 workspace/helpers/query_structural_index.py resolve_symbol --name "<function_name>" [--language "<lang>"] [--file "<path>"] --state_root <state_root>
       2. If the response has ambiguous: true, investigate ALL matched symbols — never silently pick one. Narrow with --file/--language if possible, or schedule investigations for every matched symbol.
       3. Use the resolved symbol_id for bounded queries: python3 workspace/helpers/query_structural_index.py find_callers --symbol_id "<id>" --limit 100 --offset 0 --state_root <state_root>

     • Coverage-aware interpretation: Check coverage.partition_status in every structural index response:

       • complete + precision == semantic + empty results = "no indexed callers" (authoritative for indexed code — still run grep per the HINT-only rule).
       • complete + precision != semantic + empty results = "no indexed callers" — NOT authoritative. MUST run exhaustive grep.
       • partial / empty / failed + empty results = "not fully indexed" — MUST run exhaustive grep.
       • Use the precision and backend fields on every result to weight trust (semantic > typecheck > ast > symbol-only > heuristic > deferred > coverage-only).

     • Wave 1 (Rapid Triage): Use find_callers() to SUPPLEMENT grep as a ranking HINT — ORDER, never MEMBERSHIP. Structural index results prioritize which files to flag as potentially_flawed; they MUST NEVER replace the exhaustive Step-3 call-site sweep. Audit the union of grep results and structural index results.

     • Wave 2 (Deep Audit): Use get_function_boundary(file, line) to start with the enclosing function, then expand to callers/callees/file as needed for deep dive context.

     • Graceful degradation: If the structural index is absent (no manifest.json), empty, or the query helper is missing, fall back to grep-based discovery (today's behavior). The structural index is a coverage HINT only.

3. Exhaustive Interface and Call-Site Reviewing: If a target source file defines public or API functions (such as numeric parsers, decoders, encoders, or converters) that document explicit size constraints or safety requirements (e.g., expecting callers to allocate buffers of a certain size):

   • Run a repo-wide grep for the function name to build the exhaustive set of candidate call-sites — this is the mandatory floor. Then use the structural index query helper (resolve_symbol then find_callers) to RANK and prioritize which call-sites to audit first (the index distinguishes actual calls from comments/strings/variable names). Audit the union of both result sets — the structural index may miss macro-based calls, function pointers, and dynamic dispatch, so grep remains the floor.
   • Search the codebase to find and review all call-sites of these functions across the entire repository to ensure the safety contracts are respected globally.
   • Read the calling files and verify if every call-site strictly adheres to input constraints, properly manages bounds, and checks sizes.
   • Flag any discrepancies as contract alignment bugs or missing checks.

4. Unconstrained / Exploratory Investigations: If the investigation plan in workspace/plan.json contains instructions or a question explicitly asking for an unconstrained sweep, adversarial audit, or random exploration:

   • Ignore existing assumptions of safety and documented trust boundaries in workspace/kb/THREAT_MODEL.md.
   • Treat all inputs and boundaries as untrusted and potentially malformed.
   • Analyze implementation from scratch with full freedom and autonomy.
   • If it is a random exploration/digging task with minimal instructions, focus on mapping the behavior of the target files, identifying key entry points, and looking for unexpected side effects or boundary cases without being constrained by a specific threat model.

5. Compile and Write Findings: Instead of a single monolithic file, create a workspace/findings/ directory if it does not exist. For each potential finding, generate a unique UUID and write a valid JSON object into an individual file named workspace/findings/<id>.json. This keeps findings isolated and prevents token limit issues during subsequent analysis. Do not include any text before or after the JSON in the files.

   Signature & Lineage Computation (Step 5a):

   For each finding, compute signature, lineage_id, and optionally cwe BEFORE writing the JSON file:

   1. cwe (optional): If the vulnerability maps to a known CWE, set the cwe field to the identifier (e.g., "CWE-787", "CWE-416"). If no CWE applies, omit the field. This is an input to the signature.

   2. Compute signature (deterministic content-identity hash):

      • normalized_title = finding title, lowercased, with ALL non-alphanumeric characters stripped (ASCII [a-zA-Z0-9] only; all other characters including Unicode letters, punctuation, and spaces are removed). If stripping leaves normalized_title EMPTY (e.g. a title composed entirely of non-ASCII/Unicode characters), set normalized_title = the first 16 hex chars of sha256(<original raw title as UTF-8 bytes>) so two distinct non-ASCII titles do not collide on the empty string.
      • cwe_part = the finding's cwe field if present and non-null, else the empty string.
      • primary_target = first code_paths entry with trailing :line stripped (e.g., src/auth.c:145 → src/auth.c). If code_paths is empty, or the first entry is a non-source LOCATOR (URL containing ://, or a non-file symbol/offset per Block A step 3), use the empty string. Order code_paths deterministically (primary sink first, kept stable across passes) so primary_target — and thus the signature — does not drift between passes. (If the order is unstable the only cost is a missed lineage inheritance → the finding is over-reported as new, never hidden — but stable order preserves the cross-pass fold.)
      • If primary_target is non-empty: signature = first 16 hex characters of sha256(normalized_title + "|" + cwe_part + "|" + primary_target).
      • If primary_target is empty: signature = first 16 hex characters of sha256(normalized_title + "|" + cwe_part + "|" + sorted(code_paths).join(",")).
      • Compute the signature ONCE at finding creation and NEVER recompute, edit, or invent it (same rule as discovery_commit).

   3. Compute lineage_id (cross-pass lineage chain):

      • Scan workspace/archive/findings_pass_*/ and workspace/archive/loop*_findings/ for any archived finding JSON whose signature field equals this finding's computed signature.
      • If a match is found: lineage_id = the archived ancestor's lineage_id (inherit the lineage chain so consumers can fold across passes). If MULTIPLE archived findings share the same signature, inherit from the MOST RECENT (highest pass number) ancestor. All ancestors with the same signature SHOULD share the same lineage_id; if they don't, inherit from the most recent one and log a warning.
      • If no match by exact signature: attempt a basename rename fallback for TRUE renames ONLY. Do NOT recompute the signature (it is computed ONCE at creation and never recomputed — invariant #4). Instead:
        1. Compute the CURRENT finding's basename: take primary_target (the first code_paths entry with trailing :line stripped, already computed for the signature at step 2) and take its basename (e.g. src/auth.c → auth.c). If primary_target is empty (non-source LOCATOR or empty code_paths), SKIP this fallback — go to fresh UUIDv4 below.
        2. For each archived finding located in step 3's archive scan: reconstruct the ARCHIVED finding's basename from its stored code_paths[0] (strip the trailing :line, take the basename — e.g. lib/old_auth.c:88 → old_auth.c). Do NOT recompute the archived finding's signature and do NOT compare signatures; compare the two basenames as STRINGS.
        3. Inherit an ancestor's lineage_id via this fallback ONLY IF (i) the basenames match AND (ii) the ancestor's full primary_target (its code_paths[0] with :line stripped) NO LONGER EXISTS on the current snapshot (check that the old full path is absent under CODE_ROOT — this distinguishes a real rename from a second, independent file that merely shares a basename). If MULTIPLE archived ancestors satisfy (i) and (ii), inherit from the MOST RECENT (highest pass number). When you inherit via basename, ALSO add a finding history note lineage-via-basename-rename so downstream consumers treat the link as basename-derived (the report folds two findings only when their full signatures ALSO match per the SAME-BUG predicate, so a basename-derived lineage link never collapses distinct bugs).
        4. If the old full path still exists on the current snapshot, do NOT inherit — treat as no match (fresh UUIDv4 below).
      • If no match by either exact signature or basename rename: lineage_id = a fresh UUIDv4.
      • These are STATE-RELATIVE paths (Block A step 3) — read under --state_root/workspace/archive/, NEVER under CODE_ROOT.

   4. Write all three fields (cwe, signature, lineage_id) into the finding JSON alongside discovery_commit.

   Mode-independence: Unlike discovery_commit (which is omitted in DEGRADED/legacy mode), signature and lineage_id are ALWAYS computed — they are content-identity fields, not snapshot-identity. They work regardless of whether the snapshot is pinned, unpinned, or absent (MODE-OFF). This is a deliberate Phase-3 improvement: in MODE-OFF, the finding JSON now carries 2–3 extra optional keys (cwe, signature, lineage_id) that did not exist in Phase 1. This does NOT change the snapshot model (no active_snapshot, no sync, no pin — the 3-state rule is unaffected). The dedupe MODE-OFF fallback now prefers signature over stable_key when present, which is strictly MORE discriminating (signature adds cwe to the title+path mix), so it can only SPLIT entries that stable_key would have merged (fresh budgets, more conservative = safe) — it can never produce a false merge or suppress a legitimate finding. Absent signature → today's stable_key behavior exactly.

   Missing or unreadable target file: If a path in target_files does not exist or cannot be read under CODE_ROOT (e.g. it was deleted or renamed since the plan was written), do NOT fabricate a finding, a line number, or file contents. Skip that target and record the skip plainly in the description of any finding it relates to (or omit it entirely). Never invent code you did not read.

   Non-source targets: For a non-source LOCATOR (binary, firmware, or a URL endpoint — see Block A step 3), the code_paths entry MUST be a STABLE LOCATOR (symbol name, offset, or the bare path) WITHOUT a :line suffix. Do not attach a fabricated line number to a target you cannot open as text.

Findings Schema Format (Per File)

discovery_commit rule (set once, at creation): This stage is the creation site for discovery_commit. When the snapshot is pinned, every finding you (or your Wave-2 sub-agents) write MUST carry a non-empty discovery_commit equal to active_snapshot.snapshot_id. Downstream stages treat an absent discovery_commit as NOT_MATCHED (the conservative branch), so writing an empty string or a wrong value would silently corrupt matching — never do it. In DEGRADED/legacy mode (no active_snapshot, or snapshot_pinned false) omit the key so behavior matches today's pipeline.

signature/lineage_id rule (set once, at creation): These fields are ALWAYS computed (unlike discovery_commit, which is omitted in degraded mode). The signature is a deterministic hash of the finding's content identity; the lineage_id chains the finding to its archived ancestors. Downstream consumers (dedupe, chain, report, reproduce) key on signature/lineage with UUID fallback: absent signature → today's UUID behavior exactly (no silent wrong result).

Ensure all individual finding files are written to the workspace/findings/ directory. When complete, notify the user.