medtech-model-evidence-export

Medtech Model Evidence Export to MLflow

Purpose

Mirror an existing medical-inference result or evidence pack into MLflow after the run and emit the export_result JSON contract. Keep the original evidence pack as the source of truth. Training skills should add MLflow inside their training loops instead.

Instructions

1. Run scripts/export_evidence_pack.py in the default dry-run mode.
2. Inspect params, metrics, artifact_plan, and mlflow.note.content.
3. Choose --mode local or --mode databricks only after checking the target.
4. Keep --artifact-policy metadata unless the target is approved for images.
5. For preview or all in a live mode, also pass --confirm-medical-artifact-upload.
6. Keep --source-ref, --note, config filenames, and artifact filenames free of patient or secret identifiers; always review the dry-run output first.

Hosts with a script helper can use run_script("scripts/export_evidence_pack.py", args=["PACK_OR_RESULT", "--mode", "dry-run"]).

Available Scripts

Prerequisites

• Python 3.10+.
• mlflow>=2.10,<4 for local or databricks mode.
• numpy>=1.24,<3 and nibabel>=4,<6 for NIfTI quality metrics and previews.
• MLFLOW_TRACKING_URI may select a caller-managed tracking server.
• Databricks mode uses the caller's DATABRICKS_HOST, DATABRICKS_TOKEN, or configured Databricks profile. The declared network endpoint is https://<caller-provided-mlflow-or-databricks-workspace>; Docker and GPU are not required.
• Local mode may write the MLflow store under <current-working-directory>/mlruns.

Examples

Preview the export without contacting MLflow:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
  runs/inference_pack --mode dry-run --artifact-policy metadata

Export a direct NV-Generate result with reproducibility metadata:

python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py

Log downsampled slice previews, but not raw NIfTI files:

--artifact-policy all additionally uploads discovered or explicitly supplied NIfTI images and masks, subject to --max-artifact-mb. Use --image and --mask when paths are not present in the result JSON.

The exporter logs:

• scalar run and quality metrics, including sampled HU mean/std/min/max for CT (generic intensity statistics otherwise), a documented intensity-SNR heuristic, mask foreground percentage, and mapped tumor volume percentage when a tumor label mapping is available;
• generation parameters, model/checkpoint identity, RNG seed, and recipe hash;
• source config digest or --source-ref, plus a prompt digest when present;
• mlflow.note.content with a short human-readable run summary;
• a sanitized metadata bundle by default, optional PNG slice previews, and raw image/mask artifacts only under the explicit all policy.

Limitations

• This is post-hoc inference export, not live training-curve tracking.
• Global intensity SNR and downsampled volume statistics are engineering checks, not image-quality or clinical-performance claims.
• Preview and raw artifacts may contain sensitive medical information. The caller must approve the destination and data policy before upload.
• The exporter does not evaluate model quality, register models, or alter the source evidence pack.

Troubleshooting