vulnerability-validation

Vulnerability Validation

Validate security findings by assessing whether they are actually exploitable in the context of this codebase. This skill filters false positives, confirms real vulnerabilities, and generates proof-of-concept exploits.

When to Use This Skill

• After commit-security-scan - Validate findings before creating issues or blocking PRs
• HIGH/CRITICAL findings - Prioritize validation of severe findings
• Before patching - Confirm vulnerability is real before investing in fixes
• Security review - Deep-dive validation of specific findings

Prerequisites

• .factory/threat-model.md must exist (from threat-model-generation skill)
• security-findings.json must exist (from commit-security-scan skill)

Inputs

Instructions

Follow these steps for each finding to validate:

Step 1: Load Context

1. Read security-findings.json from commit-security-scan
2. Read .factory/threat-model.md for system context
3. Identify which findings to validate based on inputs

Step 2: Reachability Analysis

For each finding, determine if the vulnerable code is reachable:

1. Trace entry points

   • Can external users reach this code path?
   • What HTTP endpoints, CLI commands, or event handlers lead here?
   • Is authentication required to reach this code?

2. Map the call chain

   • Starting from the entry point, trace the path to the vulnerable code
   • Document each function call in the chain
   • Note any branching conditions that must be satisfied

3. Classify reachability

   • EXTERNAL - Reachable from unauthenticated external input
   • AUTHENTICATED - Requires valid user session
   • INTERNAL - Only reachable from internal services
   • UNREACHABLE - Dead code or blocked by conditions

Step 3: Control Flow Analysis

Determine if an attacker can control the vulnerable input:

1. Identify the source

   • Where does the tainted data originate?
   • HTTP parameter, file upload, database query, environment variable?

2. Trace data flow

   • Follow the data from source to sink (vulnerable function)
   • Document each transformation or validation step
   • Note any sanitization, encoding, or type conversion

3. Assess attacker control

   • Can the attacker fully control the input?
   • Are there length limits, character restrictions, or format validation?
   • Does the data pass through any sanitization?

Step 4: Mitigation Assessment

Check if existing security controls prevent exploitation:

1. Input validation

   • Is the input validated before reaching the vulnerable code?
   • What validation rules are applied?

2. Framework protections

   • Does the framework provide automatic protection? (e.g., ORM parameterization, React XSS escaping)
   • Is the protection enabled and properly configured?

3. Security middleware

   • Are there WAF rules, rate limiting, or other controls?
   • Do CSP headers or other browser protections apply?

4. Reference threat model

   • Check the "Existing Mitigations" section for this threat type
   • Verify mitigations are actually in place

Step 5: Exploitability Assessment

Determine how difficult it is to exploit:

Consider:

• Attack complexity
• Required privileges
• User interaction needed
• Scope of impact

Step 6: Generate Proof-of-Concept

For confirmed vulnerabilities, create a proof-of-concept:

1. Craft exploit payload

   • Create a minimal payload that demonstrates the vulnerability
   • Use benign payloads (no actual damage)

2. Document the request

   • HTTP method, URL, headers, body
   • Or CLI command, file input, etc.

3. Describe expected vs actual behavior

   • What should happen (secure behavior)
   • What actually happens (vulnerable behavior)

Example PoC structure:

{
  "payload": "' OR '1'='1",
  "request": "GET /api/users?search=' OR '1'='1",
  "expected_behavior": "Returns users matching search term",
  "actual_behavior": "Returns all users due to SQL injection"
}

Step 7: Calculate CVSS Score

Assign a CVSS 3.1 score based on:

Example: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N = 9.1 (Critical)

Step 8: Classify Finding

Based on analysis, classify each finding:

Step 9: Generate Output

Create validated-findings.json:

Success Criteria

The skill is complete when:

• All specified findings have been analyzed
• Each finding has a status (CONFIRMED, LIKELY, FALSE_POSITIVE, NEEDS_MANUAL_REVIEW)
• Confirmed findings have exploitation paths documented
• Confirmed findings have proof-of-concept exploits
• False positives have clear reasoning
• validated-findings.json is valid JSON
• CVSS scores are calculated for confirmed findings

Verification

Run these checks before completing:

Example Invocations

Validate all findings:

Validate specific findings:

Validate only HIGH/CRITICAL:

Validate with specific files:

References

• CVSS 3.1 Calculator: https://www.first.org/cvss/calculator/3.1
• OWASP Testing Guide: https://owasp.org/www-project-web-security-testing-guide/
• Examples: validation-examples.md (in this skill directory)
• Upstream: commit-security-scan skill