v0: 512k natural typed sq_o (dev_selected step 4000) + card

1_Dense/config.json

@@ -0,0 +1,7 @@
+{
+    "in_features": 768,
+    "out_features": 1536,
+    "bias": false,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "use_residual": true
+}

1_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:304a390aa5da090c9c22a999ab8fc18194172e97f2098b7bbff9dd1f68d6fa91
+size 9437360

2_Dense/config.json

@@ -0,0 +1,7 @@
+{
+    "in_features": 1536,
+    "out_features": 768,
+    "bias": false,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "use_residual": true
+}

2_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca0a3107481100bbec203c119b39afa6d47345a31dcdc015a763ab1dbd8a256e
+size 9437360

3_Dense/config.json

@@ -0,0 +1,7 @@
+{
+    "in_features": 768,
+    "out_features": 128,
+    "bias": false,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "use_residual": false
+}

3_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad5e8919ffbd769684d58e7e4a5219c14b2834e04dc0101a28a5f65d8eed5e14
+size 393304

README.md

@@ -0,0 +1,85 @@
+---
+tags:
+- ColBERT
+- multi-vector
+- PyLate
+- sentence-transformers
+- typed-decisions
+- zero-shot-classification
+pipeline_tag: text-classification
+library_name: sentence-transformers
+license: apache-2.0
+language:
+- en
+base_model: lightonai/LateOn
+---
+
+# Tasksource-JEV-Nano-v0
+
+Decoupled multi-vector late-interaction decision model: **LateOn (~149M, ModernBERT backbone) fine-tuned on 512k natural heterogeneous Tasksource typed decisions** (choice + noul + score, natural mix, no distillation).
+
+Factorization `sq_o = E(S+Q) ∥ E(O_i)` with PyLate native ColBERT MaxSim scoring. Context representations `E(S+Q)` are computed once and reused across decisions (state caching); candidate options are encoded independently (bit-exact permutation equivariance, verified max diff 0.0).
+
+- **Base model:** [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn)
+- **Training data:** [`tasksource/tasksource-jev-typed-decisions`](https://huggingface.co/datasets/tasksource/tasksource-jev-typed-decisions) @ `071f0cf2201f06b2dea3e55323aa3e264284e139`
+- **Stream:** 512,000 decisions, manifest SHA-256 `997d241e34682f2b8b354103d1bbdc7e873fb45333e09637231f513176969331`, kind mix choice 82.7% / noul 11.4% / score 6.0%, 519 train tasks
+- **Recipe:** 8,000 steps × 64, LR 2e-5, kind-aware loss (choice: soft-target CE, noul: 2-way CE, score: CE + RPS), fail-closed `--canonical_run`
+- **Selection:** Tasksource unseen-dev NLL only → step 4000 of 8000 (best NLL 1.2073 internal / 1.0002 canonical probe). Learned temperature T=0.5347.
+- **Code:** training/eval harness in the `decision-models` repo (`scripts/train_multivector.py`, `scripts/eval_multivector.py`)
+
+## Benchmarks (canonical probes: deterministic 1k slices, seed 13 + full Fast Decisions)
+
+| Benchmark | Accuracy | NLL | Brier | ECE |
+|---|---|---|---|---|
+| Tasksource unseen-dev | 58.9% | 1.0002 | 0.4822 | 0.0435 |
+| Tasksource unseen-test | 53.0% | 0.9676 | 0.5367 | 0.0590 |
+| Tasksource seen-dev | 56.0% | 0.9573 | 0.5317 | 0.0586 |
+| Typed Decisions | 42.6% | 1.3144 | 0.2935 | 0.1294 |
+| AG News | 73.8% | 0.7510 | 0.4017 | 0.0840 |
+| Emotion | 49.0% | 1.4226 | 0.7033 | 0.1309 |
+| Banking77 (K=77) | 44.7% | 2.2661 | 0.7536 | 0.1857 |
+| Fast Decisions (2,900, 17 domains) | 41.31% EM (macro 40.59) | — | — | — |
+| classifier-benchmark v2 (866 cases) | micro 57.51 / macro 57.17 | — | — | — |
+
+Context: same-slice sub-400M peers score AG News 65–78%, Banking77 23–70%, Typed 34–49%, cb-v2 micro 35–60% — v0 is competitive while adding state caching and permutation equivariance, which classifier heads lack. Relative to its choice+KD sibling (50.8% unseen-dev), v0 gains +8.1pp in-domain and +6.9pp Typed at some AG/Banking77 transfer cost (documented specialization tradeoff).
+
+## Usage (minimal inference example)
+
+```python
+from modernjev.multivector.lateon import LateOnDecisionScorer
+
+scorer = LateOnDecisionScorer("tasksource/tasksource-jev-nano-v0", device="cuda")
+
+state = "Customer reports unauthorized international wire transfer of $4,500 on account #4092."
+question = "Select the appropriate fraud mitigation routing:"
+options = [
+    "approve_silent_monitor",
+    "two_factor_push_challenge",
+    "temporary_freeze_outreach",
+    "hard_decline_and_report",
+]
+
+probs = scorer.predict_batch([state], [question], [options], kinds=["choice"])[0]
+for opt, p in zip(options, probs):
+    print(f"  {opt:30s} -> {p:.4f}")
+
+# State caching: encode once, score many candidate sets with zero re-encoding
+cache = scorer.encode_state([state], questions=[question])
+out = scorer.decide_cached(cache, questions=[question], options=[options], kinds=["choice"])
+print("Logits:", out.logits[0, :len(options)])
+```
+
+Heterogeneous kinds are supported per decision: `choice` (softmax), `noul` (2-way), `score` (softmax + ordinal), `multilabel` (independent sigmoids).
+
+## Citation
+
+```bibtex
+@misc{sileo2026jevnanov0,
+  title={Tasksource-JEV-Nano-v0: Decoupled Multi-Vector Late Interaction for Typed Decisions},
+  author={Sileo, Damien},
+  year={2026},
+  howpublished={\url{https://huggingface.co/tasksource/tasksource-jev-nano-v0}},
+}
+```
+
+Base model: LateOn by LightOn (`lightonai/LateOn`, Apache 2.0). Training data: `tasksource/tasksource-jev-typed-decisions`.

config.json

@@ -0,0 +1,78 @@
+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "local_attention": 128,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "repad_logits_with_grad": false,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 160000.0,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "vocab_size": 50370
+}

config_sentence_transformers.json

@@ -0,0 +1,54 @@
+{
+  "model_type": "ColBERT",
+  "__version__": {
+    "sentence_transformers": "5.1.2",
+    "transformers": "5.17.0",
+    "pytorch": "2.7.1+cu126"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "MaxSim",
+  "query_prefix": "[Q] ",
+  "document_prefix": "[D] ",
+  "query_length": 32,
+  "document_length": 300,
+  "attend_to_expansion_tokens": false,
+  "skiplist_words": [
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "do_query_expansion": false
+}

decision_meta.json

@@ -0,0 +1,21 @@
+{
+  "init_temperature": 0.5347228646278381,
+  "path_mode": "sq_o",
+  "pooling_budget": null,
+  "protected_tokens": 1,
+  "config": {
+    "model_name_or_path": "lightonai/LateOn",
+    "embedding_dim": 128,
+    "state_max_length": 512,
+    "query_max_length": 64,
+    "init_temperature": 1.0,
+    "learn_temperature": true,
+    "path_mode": "sq_o",
+    "pooling_budget": null,
+    "protected_tokens": 1,
+    "query_prefix": "[Q] ",
+    "document_prefix": "[D] ",
+    "backend": "torch",
+    "metadata": {}
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82b40602a9ac23a332640fce13e8f9b2ccf8703d9ac5d36b40f6d41107e2a92c
+size 596076280

modules.json

@@ -0,0 +1,26 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Dense",
+    "type": "pylate.models.Dense.Dense"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Dense",
+    "type": "pylate.models.Dense.Dense"
+  },
+  {
+    "idx": 3,
+    "name": "3",
+    "path": "3_Dense",
+    "type": "pylate.models.Dense.Dense"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 31,
+    "do_lower_case": false
+}

tokenizer_config.json

@@ -0,0 +1,24 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "is_local": false,
+  "local_files_only": true,
+  "mask_token": "[MASK]",
+  "max_length": 299,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 299,
+  "pad_to_multiple_of": null,
+  "pad_token": "[MASK]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "tokenizer_class": "TokenizersBackend",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}