DAYUN_CALIBRATION.md

Da Yun blind calibration

This round adds a measurement harness and a separate broad trend signal. The detailed judgment.score remains the structure/rule score; the free trend preview and life curve prefer judgment.trend_score, which is a support-aware, deduplicated ordinal heuristic. Neither 0..100 value is a probability.

Protocol

1. Put pseudonymous chart inputs in data/accuracy/dayun_charts.template.json. Do not put names, emails, or other identifiers in the file.

2. Export scores before labels are collected:

   PYTHONDONTWRITEBYTECODE=1 python dayun_prediction_export.py \
     --charts data/accuracy/dayun_charts.template.json \
     --output /tmp/dayun-predictions.json
   

3. Copy the period keys into a private label file based on dated evidence. Use data/accuracy/dayun_labels.template.json as the format. Keep the scores hidden from the person assigning the labels. Each label needs at least one dated evidence item and a confidence level.

4. Evaluate only after the labels are sealed:

   PYTHONDONTWRITEBYTECODE=1 python dayun_calibration.py \
     --labels /path/to/private/dayun-labels.json \
     --predictions /tmp/dayun-predictions.json
   

Reported metrics

• Spearman rank correlation: whether higher model scores order periods better.
• Pairwise ranking accuracy: among periods with different observed labels, how often the score ordering agrees; a prediction tie counts as half credit.
• Ordinal MAE: distance after mapping labels 0..4 and scores 0..100 to 0..1.
• Constant-50 baseline: the minimum comparator for a supposedly informative score.

The prediction exporter includes both score and trend_score so they can be evaluated separately. A trend score is intended to rank broad ten-year windows; it must not be read as an event probability or a verdict on a particular traditional school.

The evaluator reports warnings for small samples and unmatched periods. It does not declare a rule “accurate” from a threshold. Any weight change must be pre-registered, evaluated out-of-sample, and compared with the baseline.

Test resolution

The focused fixtures and disposable mutation runner exercise stem alignment, weak-storehouse penalty, relationship weight, and score clamping:

PYTHONDONTWRITEBYTECODE=1 python mutation_test_dayun.py

Each mutation must be detected. An intentionally changed weight that leaves the test suite green is a test failure, not evidence that the rule is safe.

Data boundary

The repository contains only empty templates and synthetic unit-test labels. Real labels should remain private or be anonymized before sharing. The label document rejects common input/identity fields and the prediction exporter does not write birth details to its output.

Cross-system discrepancy diagnostic

When another system appears to rate a period differently, run the validation-only diagnostic before changing the production score:

PYTHONDONTWRITEBYTECODE=1 python dayun_cross_system_diagnostic.py \
  --birth-date 1981-10-04 --birth-time 20:40 --gender female \
  --current-age 44 --num-dayun 8 --target 壬寅 --target 癸卯 \
  --output /tmp/dayun-cross-system.json

The report keeps four tracks separate: the production baseline, a diagnostic that counts an identical natal branch signature once, a weak-body 扶抑 support track, and both counterfactuals combined. They are hypotheses, not replacement outputs. For the 1981-10-04 chart the baseline is 壬寅=62 and 癸卯=20; the diagnostic exposes 癸卯=35 after relation de-duplication and 75 on the support track. This spread is evidence of rule sensitivity, not evidence that any counterfactual is correct. Use the blind label protocol above to decide whether either change improves out-of-sample ranking.