Improve model card and calibration visual

README.md

@@ -15,11 +15,29 @@ thumbnail: https://huggingface.co/flydexo/bad-laya/resolve/main/assets/overview.
 
 # bad-laya
 
-**A research checkpoint for typed decisions across 21 evaluation sources.** It combines a fine-tuned ModernBERT-large encoder with a two-layer decision head that scores the available options. This release is the **partial stage-16 checkpoint** of an ordered, full-split curriculum—not the best checkpoint of the run.
+**An open-weight research checkpoint for typed decisions.** Give it a state and a question with explicit options; its ModernBERT encoder and decision head score those options without generating text. This is the **partial stage-16 checkpoint** of an ordered curriculum, not the best checkpoint of the run.
 
 ![Evaluation overview](https://huggingface.co/flydexo/bad-laya/resolve/main/assets/overview.png)
 
-> **Research use.** The model is often confident when wrong, especially on review classification. Validate on your own data before using its decisions. It has not been evaluated as a general-purpose assistant or on a held-out final test.
+**Explore:** [Results](#at-a-glance) · [Quickstart](#load-and-score-a-request) · [Calibration](#temperature-calibration) · [Provenance](#training-and-evaluation-provenance) · [Limitations](#limitations-and-use)
+
+> **Research use.** The model is weak on some review tasks, even after calibration. The 21-source comparison uses validation samples; the calibration check uses separate evaluation partitions. An official Decision Index 0.3 score has not yet been completed.
+
+## Model details
+
+| | |
+|:--|:--|
+| Developer | [flydexo](https://huggingface.co/flydexo) |
+| Backbone | Fine-tuned `answerdotai/ModernBERT-large` at pinned revision `45bb4654a4d5aaff24dd11d4781fa46d39bf8c13` |
+| Decision head | Two transformer layers with one learned score per supplied option marker |
+| Input | A text or JSON state, plus named `choice`, `score`, or `noul` questions |
+| Output | A probability for each option; selected choice, expected score, or probability of true |
+| Context | Up to 1,024 tokens per question under the saved preprocessing configuration |
+| Weights | 895 MB Safetensors: BF16 encoder and FP32 decision head |
+| Calibration | One post-training temperature, **T = 12.60**, stored in `calibration.json` |
+| License | Not declared for this combined checkpoint; the ModernBERT base is Apache 2.0 |
+
+The question’s options define the answer space at inference time. `choice` returns the most likely named option, `score` returns the probability-weighted level index, and `noul` returns the probability of true. The custom [`decisions` code](https://github.com/Flydexo/decisions) handles preprocessing and option-marker scoring. **This repository is not a drop-in `transformers.AutoModel.from_pretrained` or text-generation model.**
 
 ## At a glance
 
@@ -56,7 +74,7 @@ Yelp and Amazon are below random expectation in this small validation sample. Th
 | `tokenizer/` | Tokenizer files pinned to the ModernBERT-large base revision |
 | `assets/overview.png` | The visual score summary above |
 
-The model uses the custom [`decisions` implementation](https://github.com/Flydexo/decisions), including typed request preprocessing and option-marker scoring. **It is not a drop-in `transformers.AutoModel.from_pretrained` model or a text-generation model.** The weights are provided in Safetensors so they can be inspected and loaded without unpickling the original training checkpoint.
+The weights are provided in Safetensors so they can be inspected and loaded without unpickling the original training checkpoint.
 
 ## Load and score a request
 
@@ -130,4 +148,11 @@ The original 91.8% figure above is normalized entropy confidence from the valida
 
 The curriculum comparison above is **validation**, not a held-out final benchmark. The calibration check uses separate evaluation partitions but is also small. Small per-source samples, nonuniform option counts, and the difference in training budgets limit what the comparisons establish. Accuracy and confidence can change substantially across stages: the partial stage-16 checkpoint is weaker overall than stage 15. Sensitive domains represented among the sources include finance, customer support, phishing, and safety; do not use this model to make consequential decisions without a separate domain-specific evaluation and human oversight.
 
+## Limitations and use
+
+- **Useful experiments:** English text classification, routing, triage, and typed decision research where the allowed answers are supplied with each question. Refit or verify calibration on a labelled sample of the actual workload before choosing confidence thresholds.
+- **Poor fit:** Open-ended generation, fact retrieval, image or video inputs, and fully automated decisions with medical, legal, financial, employment, or safety consequences.
+- **Known failure modes:** Review sentiment collapsed in the partial stage-16 checkpoint: Yelp Review Full scored 6.3% and Amazon Reviews 18.8% on their small validation samples. Temperature reduces reported certainty but does not fix these answers. A 1,024-token context can also refuse or truncate long requests, depending on preprocessing settings.
+- **Evaluation limits:** The 21-source validation set influenced checkpoint selection. The separate calibration evaluation has 832 questions and is too small to establish reliability within every source or on shifted domains. The 0.3 Decision Index run is pending; do not compare these local percentages to its leaderboard.
+
 For the per-source results and training trajectory, see the [results page source](https://github.com/Flydexo/decisions/blob/main/reports/curriculum-results.html) and its [data snapshot](https://github.com/Flydexo/decisions/blob/main/reports/curriculum-results-data.json). The base encoder is [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) (Apache 2.0). A license for this combined checkpoint is not declared here.