Spaces:
Running
Running
| # Da Yun blind calibration | |
| This round adds a measurement harness and a separate broad trend signal. The | |
| detailed `judgment.score` remains the structure/rule score; the free trend | |
| preview and life curve prefer `judgment.trend_score`, which is a support-aware, | |
| deduplicated ordinal heuristic. Neither `0..100` value is a probability. | |
| ## Protocol | |
| 1. Put pseudonymous chart inputs in | |
| `data/accuracy/dayun_charts.template.json`. Do not put names, emails, or | |
| other identifiers in the file. | |
| 2. Export scores before labels are collected: | |
| ```bash | |
| PYTHONDONTWRITEBYTECODE=1 python dayun_prediction_export.py \ | |
| --charts data/accuracy/dayun_charts.template.json \ | |
| --output /tmp/dayun-predictions.json | |
| ``` | |
| 3. Copy the period keys into a private label file based on dated evidence. Use | |
| `data/accuracy/dayun_labels.template.json` as the format. Keep the scores | |
| hidden from the person assigning the labels. Each label needs at least one | |
| dated evidence item and a confidence level. | |
| 4. Evaluate only after the labels are sealed: | |
| ```bash | |
| PYTHONDONTWRITEBYTECODE=1 python dayun_calibration.py \ | |
| --labels /path/to/private/dayun-labels.json \ | |
| --predictions /tmp/dayun-predictions.json | |
| ``` | |
| ## Reported metrics | |
| - Spearman rank correlation: whether higher model scores order periods better. | |
| - Pairwise ranking accuracy: among periods with different observed labels, how | |
| often the score ordering agrees; a prediction tie counts as half credit. | |
| - Ordinal MAE: distance after mapping labels 0..4 and scores 0..100 to 0..1. | |
| - Constant-50 baseline: the minimum comparator for a supposedly informative | |
| score. | |
| The prediction exporter includes both `score` and `trend_score` so they can be | |
| evaluated separately. A trend score is intended to rank broad ten-year windows; | |
| it must not be read as an event probability or a verdict on a particular | |
| traditional school. | |
| The evaluator reports warnings for small samples and unmatched periods. It | |
| does not declare a rule “accurate” from a threshold. Any weight change must be | |
| pre-registered, evaluated out-of-sample, and compared with the baseline. | |
| ## Test resolution | |
| The focused fixtures and disposable mutation runner exercise stem alignment, | |
| weak-storehouse penalty, relationship weight, and score clamping: | |
| ```bash | |
| PYTHONDONTWRITEBYTECODE=1 python mutation_test_dayun.py | |
| ``` | |
| Each mutation must be detected. An intentionally changed weight that leaves | |
| the test suite green is a test failure, not evidence that the rule is safe. | |
| ## Data boundary | |
| The repository contains only empty templates and synthetic unit-test labels. | |
| Real labels should remain private or be anonymized before sharing. The label | |
| document rejects common input/identity fields and the prediction exporter does | |
| not write birth details to its output. | |
| ## Cross-system discrepancy diagnostic | |
| When another system appears to rate a period differently, run the | |
| validation-only diagnostic before changing the production score: | |
| ```bash | |
| PYTHONDONTWRITEBYTECODE=1 python dayun_cross_system_diagnostic.py \ | |
| --birth-date 1981-10-04 --birth-time 20:40 --gender female \ | |
| --current-age 44 --num-dayun 8 --target 壬寅 --target 癸卯 \ | |
| --output /tmp/dayun-cross-system.json | |
| ``` | |
| The report keeps four tracks separate: the production baseline, a diagnostic | |
| that counts an identical natal branch signature once, a weak-body 扶抑 | |
| support track, and both counterfactuals combined. They are hypotheses, not | |
| replacement outputs. For the 1981-10-04 chart the baseline is `壬寅=62` and | |
| `癸卯=20`; the diagnostic exposes `癸卯=35` after relation de-duplication and | |
| `75` on the support track. This spread is evidence of rule sensitivity, not | |
| evidence that any counterfactual is correct. Use the blind label protocol | |
| above to decide whether either change improves out-of-sample ranking. | |