aliasfox commited on
Commit
5a83a34
Β·
verified Β·
1 Parent(s): 5ab0961

Restore full model card (results, training, usage, license frontmatter)

Browse files
Files changed (1) hide show
  1. README.md +66 -70
README.md CHANGED
@@ -1,98 +1,94 @@
1
  ---
2
- base_model: Qwen3.5-2B
3
  library_name: peft
 
4
  pipeline_tag: text-generation
5
  tags:
6
- - base_model:adapter:Qwen3.5-2B
7
  - lora
8
- - verdict-slot
9
- - opencodifier
10
  - decision-model
11
- inference: false
12
  ---
13
 
14
- # OpenCodifier E1-E 2B Decision Model
15
-
16
- A verdict-slot LoRA adapter on Qwen3.5-2B for typed decision-making (choice, boolean, score).
17
- Part of the OpenCodifier decision runtime: opencodifier.pages.dev.
18
-
19
- ## Model Details
20
 
21
- - **Model type:** Decision classifier β€” verdict-slot LoRA on Qwen3.5-2B
22
- - **Base model:** Qwen3.5-2B (Apache-2.0)
23
- - **Adapter architecture:** LoRA r=16, alpha=32, dropout=0.05
24
- - **License:** Apache-2.0
25
- - **Developed by:** TaskWizerAI
26
- - **Released by:** TaskWizerAI
27
 
28
- ## Gates Passed
29
 
30
- | Benchmark | Result | Gate | Status |
31
- |---|---|---|---|
32
- | Suite holdout (231 items) | 0.8083 accuracy | β‰₯ 0.75 | βœ… PASS |
33
- | JevBench public 231 | 0.6840 accuracy | β‰₯ 0.6494 | βœ… PASS |
34
 
35
- Full evaluation evidence: `benchmarks/decision-model/results/CALIBRATION.md` and
36
- `benchmarks/decision-model/results/e1e-slot-231/results.jsonl`.
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
  ## Training
39
 
40
- - **Corpus:** 187,822 decision items (e1-softlabel-v2), teacher distributions from MiMo
41
- - **Teacher-gold agreement:** 96.1–97.7% argmax match
42
- - **Hyperparameters:** lr=1e-4, batch=2, accum=8, max_len=2048, fp16, sdpa, seed=42, 1 epoch
43
- - **Platform:** T5500 (2Γ— V100, 120W GPU cap)
44
-
45
- ## Evaluation Results
46
-
47
- ### JevBench Public 231 (231 items, authors' harness)
48
 
49
- | Metric | Value |
50
- |---|---|
51
- | Accuracy | 0.6840 |
52
- | Macro | 0.6840 |
53
- | ECE | 0.127 (calibrated T=3.362) |
54
- | Brier | 0.332 |
55
- | p50 latency | ~1.7 s (CPU, single decision) |
56
- | Replay determinism | 231/231 |
57
-
58
- ### Suite Holdout (231 items, internal)
59
 
60
- | Metric | Value |
61
- |---|---|
62
- | Accuracy | 0.8083 |
 
63
 
64
- ## Uses
65
 
66
- ### Direct Use
 
 
67
 
68
- Use via the OpenCodifier binary:
69
 
70
- ```bash
71
- # Engine-only (no model needed)
72
- opencodifier decide --input request.json
73
 
74
- # With E1-E model (requires llama-server running the decision fork)
75
- opencodifier decide --input request.json --ladder-url http://127.0.0.1:8080
76
  ```
77
 
78
- ### Out-of-Scope Use
79
-
80
- - Not a general-purpose language model
81
- - Not trained for prose generation
82
- - Not calibrated for non-decision classification
83
 
84
- ## Calibration
85
 
86
- Temperature fitted on the suite holdout: T = 3.362.
87
- Apply the calibration artifact from `results/calibration/qwen3.5-2b-q4_k_m/fit.json`
88
- before evaluating on any surface other than the holdout.
89
 
90
- ## How to Cite
91
 
92
- OpenCodifier decision runtime: https://github.com/TaskWizerAI/OpenCodifier
93
-
94
- ## Remarks
95
-
96
- - Corpus withheld (per OpenCodifier model release policy)
97
- - Calibration artifacts published separately
98
- - Explicit opt-in pull only β€” never auto-downloaded at build time
 
 
1
  ---
2
+ base_model: Qwen/Qwen3.5-2B
3
  library_name: peft
4
+ license: apache-2.0
5
  pipeline_tag: text-generation
6
  tags:
 
7
  - lora
8
+ - transformers
 
9
  - decision-model
10
+ - opencodifier
11
  ---
12
 
13
+ # OpenCodifier E1-E β€” Qwen3.5-2B verdict-slot decision LoRA
 
 
 
 
 
14
 
15
+ A LoRA adapter that turns [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B)
16
+ into a Jev-style **decision model**: probabilities are read from the
17
+ log-probs of verdict-slot label tokens at the answer position β€” nothing is
18
+ generated, nothing is parsed. It is the decision rung of the
19
+ [OpenCodifier](https://github.com/aliasfoxkde/OpenCodifier) ladder, and the
20
+ first own-trained arm to pass both release-gate legs.
21
 
22
+ ## Results
23
 
24
+ All numbers from the OpenCodifier evaluation records
25
+ ([BENCHMARKS.md](https://github.com/aliasfoxkde/OpenCodifier/blob/main/docs/BENCHMARKS.md),
26
+ TRAINING.md Β§9.9.1). Public 231-item JevBench split, deterministic
27
+ readout, fp16, single V100.
28
 
29
+ | metric | value |
30
+ |---|---|
31
+ | JevBench-231 accuracy | **0.6840** (publish bar 0.6494 β€” clear) |
32
+ | Internal holdout suite | **0.8083** (gate β‰₯ 0.75 β€” GATE_PASS; seed-replicated 0.8000) |
33
+ | Raw board ECE | 0.207 |
34
+ | Holdout-fitted temperature (T=3.362, analysis-only on board) | ECE 0.207 β†’ 0.127 |
35
+ | Auto-decidable @5% risk | 0.100 (23/231) |
36
+ | Brier | 0.504 |
37
+ | p50 latency | 0.58 s |
38
+
39
+ Per-family shape: routing / tool-selection / ordinal families at 1.00;
40
+ relational/compositional families are the weakness (multi_hop 0.389,
41
+ judge_hard 0.412, long_policy 0.368). The board is lost on compositional
42
+ families, not on routing β€” the candidate lever is relational-heavy
43
+ soft-label data, not parameters.
44
 
45
  ## Training
46
 
47
+ - **Base**: Qwen3.5-2B, LoRA verdict-slot readout
48
+ - **Corpus**: merged-v3 (173,452 records; composition and join rates in
49
+ TRAINING.md). The corpus itself is **not** released with this adapter.
50
+ - **Schedule**: 11,009 steps, single V100 (T5500), fp16
51
+ - **Calibration**: temperature T=3.362 fitted on a labelled holdout
52
+ (holdout ECE 0.179 β†’ 0.117); artifact ships in the OpenCodifier repo
53
+ at `calibration/e1e-qwen3.5-2b-holdout.json`
 
54
 
55
+ ## Intended use
 
 
 
 
 
 
 
 
 
56
 
57
+ Typed multiple-choice decisions (routing, tool selection, ordinal
58
+ grading) where calibrated token-level probabilities β€” not free text β€”
59
+ are the product. Pairs with the OpenCodifier engine's confidence gate:
60
+ low-confidence reads abstain or escalate to a stronger rung.
61
 
62
+ ## Out of scope
63
 
64
+ Free-form generation; the base model's general chat ability is not
65
+ preserved or claimed. Not evaluated for safety of generated content β€”
66
+ the adapter is a readout head, not a chat model.
67
 
68
+ ## How to use
69
 
70
+ ```python
71
+ from peft import PeftModel
72
+ from transformers import AutoModelForCausalLM, AutoTokenizer
73
 
74
+ base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B", torch_dtype="float16")
75
+ model = PeftModel.from_pretrained(base, "TaskWizerAI/opencodifier-e1e-2b")
76
  ```
77
 
78
+ Scores come from label-token log-probs at the answer position; see the
79
+ OpenCodifier repo for the readout harness and the Jev adapter wire shape.
 
 
 
80
 
81
+ ## License
82
 
83
+ Apache-2.0, matching the OpenCodifier project.
 
 
84
 
85
+ ## Citation
86
 
87
+ ```bibtex
88
+ @software{opencodifier_e1e_2026,
89
+ title = {OpenCodifier E1-E: a gate-passing 2B decision-model LoRA},
90
+ author = {Kinney, Michael},
91
+ year = {2026},
92
+ url = {https://github.com/aliasfoxkde/OpenCodifier}
93
+ }
94
+ ```