Review V1 Worthify Decision-1 model card

Private review draft with bounded-choice examples, seed-42 held-out results, matched task-LoRA evidence, and serving limits. README only; preserve the verified model release on main.

README.md

@@ -1,39 +1,175 @@
 ---
 license: apache-2.0
 base_model: google/gemma-4-12B-it
-model_name: Worthify Jev Gemma 4 12B full-weight seed-42 preview
+model_name: Worthify Decision-1 (V1 private review)
+language:
+- en
+library_name: transformers
 tags:
-  - provisional
-  - full-weight
+- gemma4
+- decision-making
+- classification
+- routing
+- full-finetuning
 ---
 
-# Worthify Jev Gemma 4 12B: private provisional seed-42 export
-
-This is a **full-weight text model**, not a LoRA adapter. It contains
-11,907,350,272 trainable text parameters from the pinned
-[google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it) revision `707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7`. Its weights
-were updated by one completed seed-42 training run (4,402 steps, two epochs)
-and exported in BF16. The selected checkpoint is training step 3200.
-This run fine-tunes an existing Gemma checkpoint for decision tasks. Training
-used a 2,048-token limit; 256K and 1M context training are unverified. The
-decision interface accepts `state`, `question`, and 2–16 supplied `options`.
-Option probabilities are uncalibrated scores over those supplied choices.
-
-This is an **internal preview**, not the declared two-seed selection.
-Checkpoint selection and reload verification used validation data only.
-No held-out results are included in this snapshot. The checkpoint's recorded
-equal-family validation score is 0.951353. This snapshot makes no claim
-that the model converged, outperforms the base model, or generalizes to new tasks.
-
-The source model is Apache-2.0 licensed. The included `LICENSE` provides the
-license text. This export contains the full text weights, tokenizer, and
-`numeric-policy.json`. Inference must preserve the recorded FP32 final softcap
-of 30; ordinary loader defaults may not implement that numeric policy.
-`training-data-attribution.json` names the source datasets and their recorded
-licenses without bundling raw rows. `provisional-verification.json` records
-the reload check. Every exported tensor was checked against the exact BF16 cast of the FP32 training checkpoint. A separate fresh process then matched its before-save reference on the same A100, using fixed batches of four across 20 validation rows, tolerance 0.05 and unchanged choices. This is a serialization check. The original H200-to-A100 logit comparison did not meet that tolerance; its result is retained separately. Batch grouping also affected logits in diagnostics. Neither equivalence across hardware nor invariance to batching is claimed. `bf16-source-reference.json` contains the corresponding source proof.
-`release-provenance.json` binds the source and local
-export hashes. Raw training, validation, and sealed-test rows are not bundled.
-
-This private repository is for internal evaluation. Recheck model behavior
-for each intended use before making any public performance claim.
+# Worthify Decision-1
+
+**V1 private review draft.** Worthify Decision-1 is a proposed name for a
+single-seed preview. The weights are available in this private repository to
+authorized reviewers; no public model weights are available yet. This is a
+full-weight, decision-task post-training of Gemma 4 12B, with a
+validation-selected checkpoint from seed 42. The planned second full-weight
+training seed and public-release verification remain outstanding.
+
+Give the model a text `state`, a `question`, and **2–16 described options**. It
+scores the offered answers in one model forward pass and returns a selected
+option ID plus conditional option scores. Your application decides whether to
+act, ask for review, or gather more evidence. The scores compare choices
+*within that request*. They are **not calibrated confidence**, safety
+probabilities, or proof that a claim is true.
+
+Evaluation-artifact paths named below refer to Worthify's source repository;
+those artifacts are not bundled with the model weights in this Hub repository.
+
+## See the decision
+
+These examples come from recorded runs of the same full-weight seed-42 preview.
+They show its bounded-choice interface, not independent measures of accuracy.
+
+**Evidence that changes the answer.** A fictional claim says a Harbor gateway
+received a fixed release before a noon audit. At 09:00, the only record is a
+vendor advisory saying release 5.4.2 contains the fix. The model selects
+`insufficient`. After an installation log and an asset-inventory record arrive,
+it selects `supported` for the same claim. The six-stage source-repository
+trace (`results/worthify/private-preview-flows-20260924/v2/evidence/episode.json`)
+matched four of six authored expected routes; it also records disagreements.
+
+| Evidence available | Offered routes | Selected route |
+| --- | --- | --- |
+| Advisory says a fixed release exists | supported · contradicted · insufficient · needs review | **insufficient** |
+| Install log and inventory connect that release to the gateway | same four routes | **supported** |
+
+**Traffic shape without an accusation.** In an authored synthetic window,
+eight roughly 200-byte outbound records reach one destination at roughly
+one-second intervals. Asked to classify the *observable shape*, the model
+selects `beacon_like`. That label says nothing about actor identity or
+malicious intent. The complete source-repository traffic trace
+(`results/worthify/private-preview-flows-20260924/v2/traffic/episode.json`)
+matched four of six authored expected routes; the input is metadata, not
+packet payloads or customer traffic.
+
+**A game that keeps moving.** In one live, text-state falling-block run, the
+same full-weight model selected placement goals and buttons for 30 pieces,
+cleared six lines, and applied 25 rotation buttons. Code enumerated legal
+placements and advanced gravity; the model never saw pixels. This
+single-seed demonstration is recorded in the source repository at
+`results/worthify/smooth-doom-long-tetris-20260926-v1/README.md`. It has no
+matched raw-Gemma control and is not a gameplay benchmark.
+
+## What the held-out comparison measured
+
+Worthify trained all 11,907,350,272 text-model parameters of
+`google/gemma-4-12B-it` revision
+`707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7`. Seed 42 completed two
+epochs over 35,210 training rows. Validation selected update 3,200; a sealed
+3,200-row test was scored after selection and export verification. The
+following arms used the same BF16, batch-four scoring path on one A100.
+
+| Held-out family and metric | Frozen Gemma | Earlier task LoRA | Full-weight preview |
+| --- | ---: | ---: | ---: |
+| Intent routing, macro-F1; 768 rows | 62.21% | 66.38% | **67.29%** |
+| WANLI-derived evidence, macro-F1; 384 rows | 67.24% | **77.01%** | 74.41% |
+| MNLI-derived evidence, macro-F1; 256 rows | 76.61% | **90.66%** | 89.61% |
+| Authored cyber scenarios, accuracy; 256 rows | 83.98% | 98.05% | **100.00%** |
+| Supplied-option menus, accuracy; 1,024 rows | 99.32% | 99.32% | **100.00%** |
+| Authored procedures, accuracy; 256 rows | 91.02% | 99.22% | **100.00%** |
+| Authored uncertainty, accuracy; 256 rows | 85.55% | 100.00% | **100.00%** |
+
+The full-weight preview exceeded frozen Gemma on the primary score for all
+seven evaluated families. The earlier task LoRA had higher evidence point
+estimates than the full-weight preview. These are transformed, supplied-choice
+tasks, not the datasets' original classification benchmarks. In the routing
+slice, the 151-label macro-F1 denominator includes 46 labels without a gold
+test example. The four authored/menu families are bounded fixtures, not
+evidence of real-world operational performance. The source repository records
+the method and full results in `docs/FULL_WEIGHT_SEED42_RESULTS.md` and the
+text-free held-out evidence in
+`results/worthify/full-weight-seed42-heldout-20260924/`.
+
+## Starting point for task LoRAs
+
+The full-weight checkpoint can serve as a base for a separate task adapter.
+The source-repository LoRA recipe (`docs/FOUNDATION_LORA_TUNING.md`) trains
+fresh rank-16 q/k/v/o adapters against the same declared-option loss, with NF4
+base loading, BF16 computation, and FP32 final-softcap scoring. Two matched
+studies trained fresh adapters from original Gemma and from this preview at
+seeds 42 and 43. Within
+each task, paired runs used the same data, options, update budget, and
+validation-only checkpoint selection; initial LoRA tensors matched across
+bases for each seed.
+
+| Downstream test | LoRA on original Gemma | LoRA on Worthify base | Result |
+| --- | ---: | ---: | --- |
+| BANKING77 official test converted to 16-option routing, 3,080 rows; accuracy | 90.42% | **91.62%** | Tuned-base gain: **+1.20 points** |
+| CTU-13 sealed scenario, 407 rows; macro-F1 | **0.432** | 0.115 | Original Gemma ahead: **0.316 F1** |
+
+The BANKING77 result uses the validation-selected seed-42 pair. Both
+predeclared seed pairs favored the tuned base (+1.20 and +0.88 accuracy
+points). Resampling the 77 original intent groups gave a descriptive 95%
+interval of **+0.32 to +2.14 points** for the selected pair. Excluding 28
+official-test rows flagged for lexical overlap with the full-weight model's
+data left a **+1.11-point** gain on 3,052 rows. This was an **exploratory,
+domain-aligned follow-up chosen after the negative CTU-13 result**; the
+full-weight base had already seen related intent training, including CLINC150.
+The task is 16-option routing, not standard 77-way BANKING77 classification.
+The dataset is by Casanueva et al. (2020), published by PolyAI under CC BY 4.0.
+
+These studies show a task-specific LoRA gain and a separate regression. They
+do not establish a general adapter advantage. The tuned-base adapters gained
+more validation accuracy relative to their lower starting score on BANKING77,
+but original Gemma had higher absolute validation-accuracy AUC over the fixed
+training grid and reached its selected checkpoint earlier. The studies do
+**not** establish faster convergence. The source repository has the
+BANKING77 paired result in
+`results/worthify/foundation-banking77-transfer-lora-20260926-v1/`, the
+CTU-13 paired result in
+`results/worthify/foundation-transfer-lora-20260926-v1/`, and their protocols
+in `docs/FOUNDATION_BANKING77_TRANSFER_EVAL.md` and
+`docs/FOUNDATION_TRANSFER_LORA_EVAL.md`.
+
+## Use and limits
+
+- **Appropriate starting points:** bounded routing, evidence triage, and
+  application-defined choices where the application validates the menu and
+  retains human or rule-based review for consequential actions.
+- **Input boundary:** text only, up to 16 options in this interface. Game
+  demonstrations convert engine state to text. The model does not inspect
+  images, packet payloads, or an entire external workflow by itself.
+- **Score boundary:** conditional scores are uncalibrated. Do not interpret a
+  high score as a probability that an action is safe or a fact is true.
+- **Evidence boundary:** the full-weight preview has one completed training
+  seed. The LoRA results cover two downstream tasks; neither estimates
+  performance across all future tasks, model seeds, or deployment settings.
+- **Runtime boundary:** training used a 2,048-token limit. Longer-context
+  behavior is unverified. This BF16 export requires the recorded FP32 final
+  softcap of 30 for option scoring; a generic loader may not implement that
+  numeric policy. The export passed a same-A100, 20-row serialization check,
+  but the original H200-to-A100 logit comparison did not meet its tolerance.
+  Batch grouping affected logits in diagnostics. The repository includes
+  `numeric-policy.json`, `provisional-verification.json`, and
+  `bf16-source-reference.json` for these checks.
+
+The Gemma-derived weights are intended for Apache 2.0 distribution with
+applicable attribution. Worthify's additions to the MIT-licensed
+[`bonsai/openjev`](https://github.com/bonsai/openjev/tree/53e3028363509f8533d90fe82d983770da1f6c02)
+implementation remain MIT. CLINC150 and WANLI keep their separate CC BY
+licenses; MultiNLI has mixed source terms, and BANKING77 is CC BY 4.0. The
+repository includes `training-data-attribution.json` and `LICENSE`; the
+source-repository inventory is `THIRD_PARTY_WORTHIFY.md`.
+This work implements an open bounded-choice interface. It is not TypeSafe's
+Jev model, does not use Jev outputs as training labels, and does not claim
+TypeSafe API compatibility or calibrated behavior.
+
+Public weights, a public release revision, and accessible demo media will be
+linked only after release verification and owner approval.