scorevision: push artifact

detect_crime_ops/README.md

@@ -0,0 +1,77 @@
+# Detect Crime Ops Toolkit
+
+Personal operations toolkit for improving your own `Detect-crime` miner.
+
+This folder implements the 8 focus areas:
+
+1. Live challenge intelligence
+2. Per-class score attribution
+3. Runtime/latency telemetry
+4. Gold personal eval set
+5. A/B inference harness
+6. Data provenance + quality tracking
+7. Confidence calibration table
+8. Deploy journal
+
+## Folder layout
+
+- `templates/challenge_log.csv` - one row per challenge (you + king + score + failure tags)
+- `templates/deploy_journal.md` - deployment decision log
+- `config/confidence_thresholds.yaml` - class thresholds to tune weekly
+- `config/failure_tags.yaml` - canonical failure taxonomy
+- `gold_eval/manifest.csv` - fixed personal eval set manifest
+- `gold_eval/images.txt` - image list consumed by A/B script
+- `scripts/log_challenge.py` - append normalized challenge rows
+- `scripts/summarize_challenges.py` - aggregate trends and regressions
+- `scripts/ab_compare.py` - compare two ONNX models on same image list
+
+## Quick start
+
+```bash
+cd /root/turbovision_crime/scratch/crime_miner/detect_crime_ops
+```
+
+### 1) Log challenge outcomes (items 1,2,3,6)
+
+```bash
+python3 scripts/log_challenge.py \
+  --csv templates/challenge_log.csv \
+  --challenge-id "2026-05-06T06:00Z_abcd" \
+  --scene "night_outdoor" \
+  --my-model-id "crime_a_from_best_ep29" \
+  --my-score 0.58 \
+  --king-model-id "alfred8995/crime001" \
+  --king-score 0.56 \
+  --my-preds 8 \
+  --king-preds 6 \
+  --p50-ms 95 --p95-ms 142 --p99-ms 181 \
+  --tags missed_balaclava_small,hoodie_fp_dark \
+  --data-sources "manako_distilled,roboflow_balaclava" \
+  --notes "dim frame, glove missed near bat"
+```
+
+### 2) Weekly summary (items 1,2,3,6)
+
+```bash
+python3 scripts/summarize_challenges.py --csv templates/challenge_log.csv
+```
+
+### 3) A/B check before deploy (item 5)
+
+```bash
+python3 scripts/ab_compare.py \
+  --a /root/turbovision_crime/scratch/crime_miner/weights.onnx \
+  --b /root/turbovision_crime/king_models/alfred8995_crime001/weights.onnx \
+  --images-file gold_eval/images.txt \
+  --imgsz 1280 --device 0
+```
+
+### 4) Gold set maintenance (item 4)
+
+- Add hard/rare examples to `gold_eval/manifest.csv`
+- Keep labels manually verified
+
+### 5) Confidence tune + deploy journal (items 7,8)
+
+- Update `config/confidence_thresholds.yaml`
+- Record each deploy in `templates/deploy_journal.md`