aliasfox commited on
Commit
5ab0961
·
verified ·
1 Parent(s): bb17b39

Initial upload: E1-E 2B verdict-slot adapter (opencodifier-e1e-2b)

Browse files
README.md ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen3.5-2B
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:Qwen3.5-2B
7
+ - lora
8
+ - verdict-slot
9
+ - opencodifier
10
+ - decision-model
11
+ inference: false
12
+ ---
13
+
14
+ # OpenCodifier E1-E 2B Decision Model
15
+
16
+ A verdict-slot LoRA adapter on Qwen3.5-2B for typed decision-making (choice, boolean, score).
17
+ Part of the OpenCodifier decision runtime: opencodifier.pages.dev.
18
+
19
+ ## Model Details
20
+
21
+ - **Model type:** Decision classifier — verdict-slot LoRA on Qwen3.5-2B
22
+ - **Base model:** Qwen3.5-2B (Apache-2.0)
23
+ - **Adapter architecture:** LoRA r=16, alpha=32, dropout=0.05
24
+ - **License:** Apache-2.0
25
+ - **Developed by:** TaskWizerAI
26
+ - **Released by:** TaskWizerAI
27
+
28
+ ## Gates Passed
29
+
30
+ | Benchmark | Result | Gate | Status |
31
+ |---|---|---|---|
32
+ | Suite holdout (231 items) | 0.8083 accuracy | ≥ 0.75 | ✅ PASS |
33
+ | JevBench public 231 | 0.6840 accuracy | ≥ 0.6494 | ✅ PASS |
34
+
35
+ Full evaluation evidence: `benchmarks/decision-model/results/CALIBRATION.md` and
36
+ `benchmarks/decision-model/results/e1e-slot-231/results.jsonl`.
37
+
38
+ ## Training
39
+
40
+ - **Corpus:** 187,822 decision items (e1-softlabel-v2), teacher distributions from MiMo
41
+ - **Teacher-gold agreement:** 96.1–97.7% argmax match
42
+ - **Hyperparameters:** lr=1e-4, batch=2, accum=8, max_len=2048, fp16, sdpa, seed=42, 1 epoch
43
+ - **Platform:** T5500 (2× V100, 120W GPU cap)
44
+
45
+ ## Evaluation Results
46
+
47
+ ### JevBench Public 231 (231 items, authors' harness)
48
+
49
+ | Metric | Value |
50
+ |---|---|
51
+ | Accuracy | 0.6840 |
52
+ | Macro | 0.6840 |
53
+ | ECE | 0.127 (calibrated T=3.362) |
54
+ | Brier | 0.332 |
55
+ | p50 latency | ~1.7 s (CPU, single decision) |
56
+ | Replay determinism | 231/231 |
57
+
58
+ ### Suite Holdout (231 items, internal)
59
+
60
+ | Metric | Value |
61
+ |---|---|
62
+ | Accuracy | 0.8083 |
63
+
64
+ ## Uses
65
+
66
+ ### Direct Use
67
+
68
+ Use via the OpenCodifier binary:
69
+
70
+ ```bash
71
+ # Engine-only (no model needed)
72
+ opencodifier decide --input request.json
73
+
74
+ # With E1-E model (requires llama-server running the decision fork)
75
+ opencodifier decide --input request.json --ladder-url http://127.0.0.1:8080
76
+ ```
77
+
78
+ ### Out-of-Scope Use
79
+
80
+ - Not a general-purpose language model
81
+ - Not trained for prose generation
82
+ - Not calibrated for non-decision classification
83
+
84
+ ## Calibration
85
+
86
+ Temperature fitted on the suite holdout: T = 3.362.
87
+ Apply the calibration artifact from `results/calibration/qwen3.5-2b-q4_k_m/fit.json`
88
+ before evaluating on any surface other than the holdout.
89
+
90
+ ## How to Cite
91
+
92
+ OpenCodifier decision runtime: https://github.com/TaskWizerAI/OpenCodifier
93
+
94
+ ## Remarks
95
+
96
+ - Corpus withheld (per OpenCodifier model release policy)
97
+ - Calibration artifacts published separately
98
+ - Explicit opt-in pull only — never auto-downloaded at build time
adapter_config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/home/mkinney/t5500-bundle/models/Qwen3.5-2B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 32,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.0",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "v_proj",
36
+ "o_proj",
37
+ "k_proj",
38
+ "q_proj"
39
+ ],
40
+ "target_parameters": null,
41
+ "task_type": "CAUSAL_LM",
42
+ "trainable_token_indices": null,
43
+ "use_bdlora": null,
44
+ "use_dora": false,
45
+ "use_qalora": false,
46
+ "use_rslora": false,
47
+ "velora_config": null
48
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:532be3bdf2ebc560bbbad95c7927937785dddaf0967a7918a5a4c030689d3a9f
3
+ size 5904616
calibration/fit.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "format_version": 1,
3
+ "scheme": "temperature",
4
+ "model_id": "e1e-qwen3.5-2b-holdout",
5
+ "calibration_version": 1,
6
+ "default_temperature": 3.361557,
7
+ "temperatures": {
8
+ "choice": 3.361557
9
+ },
10
+ "fit": {
11
+ "items": 120,
12
+ "ece_before": 0.179237,
13
+ "ece_after": 0.116711,
14
+ "source": "e1 slot-readout eval of /home/mkinney/t5500-bundle/eval/suite_holdout.json (sha256 6e9b6e0fe3ded2bb), single-run arm eval-holdout-best, model /home/mkinney/t5500-bundle/runs/e1-e-qwen2b/adapter-best; winner-vs-rest margin fit, in-sample"
15
+ }
16
+ }