signal-postmortem

Signal Postmortem

Overview

Signal Postmortem records and analyzes the outcomes of trading signals generated by the edge pipeline, screeners, and other skills. It compares predicted edge direction against 5-day and 20-day realized returns, categorizes outcomes (true positive, false positive, missed opportunity, regime mismatch), and generates feedback for edge-signal-aggregator weight adjustments and skill improvement backlog entries.

When to Use

• After a trade has been closed and you want to record the outcome
• When reviewing a batch of signals that have reached their holding period (5 or 20 days)
• To identify systematic false positive patterns from specific skills
• To generate feedback for edge-signal-aggregator weight calibration
• When building a skill improvement backlog from decision quality metrics
• For periodic (weekly/monthly) signal quality audits

Prerequisites

• Python 3.9+
• FMP API key (optional, for fetching realized returns if not provided manually)
• Standard library + requests for API calls
• Input: signal records in JSON format (from edge-signal-aggregator or screener outputs)

API Key Setup (Optional)

If you want to automatically fetch price data for return calculations, set up the FMP API key:

export FMP_API_KEY=your_api_key_here

Alternatively, pass the key via command line with --api-key YOUR_KEY. Without an API key, you can still record outcomes manually by providing --exit-price and --exit-date.

Workflow

Step 1: Prepare Signal Records

Gather closed or matured signal records. Each record should include:

• signal_id: Unique identifier
• ticker: Stock symbol
• signal_date: Date signal was generated
• predicted_direction: LONG or SHORT
• source_skill: Which skill generated the signal
• entry_price: Price at signal generation (optional, for manual override)

# Example: List signals ready for postmortem (5+ days old)
python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --list-ready \
  --signals-dir state/signals/

Step 2: Record Outcomes

Run the postmortem recorder to fetch realized returns and classify outcomes.

For manual outcome recording (when price data is already available):

Step 3: Classify Outcomes

The recorder automatically classifies each signal into one of four categories:

Classification rules are documented in references/outcome-classification.md.

Step 4: Generate Feedback Files

Generate feedback for downstream consumers:

Step 5: Review Summary Statistics

Generate aggregate statistics by skill, by ticker, and by time period:

Output Format

Postmortem Record (JSON)

Weight Feedback (JSON)

Skill Improvement Backlog Entry (YAML)

Summary Report (Markdown)

Reports are saved to reports/ with filenames postmortem_summary_YYYY-MM-DD.md.

Resources

• scripts/postmortem_recorder.py -- Records individual signal outcomes
• scripts/postmortem_analyzer.py -- Generates feedback and summary statistics
• references/outcome-classification.md -- Classification rules and edge cases
• references/feedback-integration.md -- How to integrate feedback with downstream skills

Key Principles

1. Honest Attribution -- Every outcome is attributed to its source skill for accountability
2. Regime Awareness -- Regime context is recorded to distinguish skill failure from market regime shifts
3. Minimum Sample Size -- Weight adjustments require 20+ signals for statistical validity
4. Feedback Loop Closure -- Results flow back to improve both signal aggregation and skill quality