clevrpwn commited on
Commit
9d78d68
·
verified ·
1 Parent(s): d0b7ec4

Update model card for DM-JEPA 1.1 (23.16 Index Skill, 16k context)

Browse files
Files changed (1) hide show
  1. README.md +22 -54
README.md CHANGED
@@ -13,15 +13,15 @@ tags:
13
  pipeline_tag: feature-extraction
14
  ---
15
 
16
- # DM-JEPA: Decision-Making Joint Embedding Predictive Architecture
17
 
18
  <div align="center">
19
 
20
  **Developed by Danger Labs**
21
- *Ultra-fast, Non-Autoregressive System 1 Decision Model*
22
 
23
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
24
- [![Decision Index](https://img.shields.io/badge/Decision--Index-0.2.1%20%7C%2018.37-blue)](https://huggingface.co/datasets/DangerLabs/decision-index-results)
25
 
26
  </div>
27
 
@@ -29,12 +29,13 @@ pipeline_tag: feature-extraction
29
 
30
  ## ⚡ Overview
31
 
32
- **DM-JEPA** is a non-autoregressive **System 1 Decision Engine** built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning **entirely within latent thought space**.
33
 
34
- - **Sub-35ms Latency**: Delivers **33.1 ms** median inference latency per decision request on single consumer GPU hardware, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs.
35
- - **Zero Token Hallucination**: Directly predicts compatibility energy between environmental state facts and candidate decision criteria.
 
36
  - **Listwise Joint Decision Space**: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
37
- - **Official Decision Index Full-Panel Score**: **18.37 Balanced Skill** / **37.62 Balanced Raw** across all **150,759 requests** of the full Decision Index suite ([view evaluation dataset](https://huggingface.co/datasets/DangerLabs/decision-index-results)).
38
 
39
  ---
40
 
@@ -42,7 +43,7 @@ pipeline_tag: feature-extraction
42
 
43
  ```mermaid
44
  flowchart TD
45
- State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder"]
46
  Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
47
 
48
  StateEnc --> HState["Contextual Sequence Embeddings h_state"]
@@ -55,10 +56,10 @@ flowchart TD
55
  Scorer --> Probs["Decision Probabilities P(Y_k | X)"]
56
  ```
57
 
58
- 1. **State Context Encoder**: Built upon `answerdotai/ModernBERT-base` (8192-token context window support).
59
  2. **Criteria Target Encoder**: Computes semantic anchor embeddings for each candidate choice.
60
  3. **M-Step Recurrent Latent Predictive Verifier**: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
61
- 4. **Latent Compatibility Scorer**: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.
62
 
63
  ---
64
 
@@ -68,24 +69,24 @@ Evaluated across the complete **150,759-request suite** of the Decision Index (0
68
 
69
  | Metric / Area | Score | Notes |
70
  | :--- | :---: | :--- |
71
- | **Decision Index (Balanced Skill)** | **18.37** | Full 44-benchmark panel (all 38 scored) |
72
- | **Raw Index (Balanced Raw)** | **37.62** | Full panel |
73
- | **Breadth Skill** | **17.86** | Across all 5 domains |
74
- | **Knowledge & Reasoning** | **25.03% Skill** | 43.51% Raw (99.40% coverage) |
75
- | **Tools & Automation** | **23.59% Skill** | 35.02% Raw (66.26% coverage) |
76
- | **Retrieval & Classification** | **15.42% Skill** | 31.45% Raw (93.28% coverage) |
77
- | **Arts & Human Taste** | **13.11% Skill** | 35.09% Raw (92.35% coverage) |
78
- | **Language Understanding** | **12.33% Skill** | 39.31% Raw (95.55% coverage) |
79
 
80
  ### Head-to-Head Benchmark Highlights
81
 
82
- | Benchmark Domain | Metric | DM-JEPA Result | Leaderboard Champion (`jev`) |
83
  | :--- | :--- | :---: | :---: |
84
  | **GSM8K (Arithmetic Reasoning)** | Raw Accuracy / Skill | **100.00% / 100.00%** | 79.87% / 75.65% |
85
  | **Home Appliance Simulator** | Field Accuracy / Field Skill | **91.12% / 82.25%** | 52.27% / 52.27% |
86
  | **OpenJev High-Trust Suite** | Raw Accuracy / Brier | **100.00% / 0.0000** | ~92.00% |
87
  | **Jevbench-Hard** | Raw Accuracy / Trap Avoidance | **85.59% / 92.45%** | ~78.00% |
88
- | **Median Inference Latency** | Time per Request | **33.1 ms** | ~950 ms (~30× faster) |
89
 
90
  ---
91
 
@@ -107,39 +108,6 @@ device = "cuda" if torch.cuda.is_available() else "cpu"
107
  model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)
108
  ```
109
 
110
- ### Running Decision Inference
111
-
112
- ```python
113
- from transformers import AutoTokenizer
114
-
115
- tokenizer = AutoTokenizer.from_pretrained("answerdotai/ModernBERT-base")
116
-
117
- state_text = "The user wants to cancel a bank transfer initiated 5 minutes ago."
118
- options = [
119
- "Cancel pending money transfer",
120
- "Request new plastic card",
121
- "Change ATM PIN code",
122
- "View currency exchange rates"
123
- ]
124
-
125
- state_enc = tokenizer(state_text, return_tensors="pt", max_length=2048, truncation=True).to(device)
126
- opt_enc = tokenizer(options, return_tensors="pt", padding=True, max_length=256, truncation=True).to(device)
127
- opt_ids = opt_enc.input_ids.unsqueeze(0)
128
- opt_mask = opt_enc.attention_mask.unsqueeze(0)
129
-
130
- with torch.no_grad():
131
- out = model(
132
- state_input_ids=state_enc.input_ids,
133
- state_attention_mask=state_enc.attention_mask,
134
- option_input_ids=opt_ids,
135
- option_attention_mask=opt_mask,
136
- )
137
-
138
- probabilities = out["probabilities"][0].cpu().tolist()
139
- for opt, prob in zip(options, probabilities):
140
- print(f"{opt:35s}: {prob*100:.2f}%")
141
- ```
142
-
143
  ---
144
 
145
  ## 🏷️ Citation & Contact
@@ -147,7 +115,7 @@ for opt, prob in zip(options, probabilities):
147
  ```bibtex
148
  @misc{dangerlabs2026dmjepa,
149
  author = {Danger Labs},
150
- title = {DM-JEPA: Non-Autoregressive System 1 Decision Model via Latent Space Predictive Architecture},
151
  year = {2026},
152
  publisher = {Hugging Face},
153
  howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}
 
13
  pipeline_tag: feature-extraction
14
  ---
15
 
16
+ # DM-JEPA 1.1: Decision-Making Joint Embedding Predictive Architecture
17
 
18
  <div align="center">
19
 
20
  **Developed by Danger Labs**
21
+ *Ultra-fast, Non-Autoregressive System 1 Decision Model with 16k Extended Context*
22
 
23
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
24
+ [![Decision Index](https://img.shields.io/badge/Decision--Index-0.2.1%20%7C%2023.16-blue)](https://huggingface.co/datasets/DangerLabs/decision-index-results)
25
 
26
  </div>
27
 
 
29
 
30
  ## ⚡ Overview
31
 
32
+ **DM-JEPA 1.1** is a non-autoregressive **System 1 Decision Engine** built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning **entirely within latent thought space**.
33
 
34
+ - **Sub-35ms Latency**: Delivers **34.3 ms** median inference latency per decision request, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs (~28× faster).
35
+ - **Extended 16k Context**: Supports up to 16,384 tokens of state context and 512 tokens per decision criterion, answering 99.84% (150,518 / 150,759) of the Decision Index full panel.
36
+ - **Zero Token Hallucination**: Directly predicts compatibility energy between environmental state facts and candidate decision criteria without free-form token hallucination.
37
  - **Listwise Joint Decision Space**: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
38
+ - **Official Decision Index Full-Panel Score**: **23.16 Balanced Skill** / **41.92 Balanced Raw** across all **150,759 requests** of the full Decision Index suite ([view evaluation dataset](https://huggingface.co/datasets/DangerLabs/decision-index-results)).
39
 
40
  ---
41
 
 
43
 
44
  ```mermaid
45
  flowchart TD
46
+ State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder (RoPE Scaled to 16k)"]
47
  Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
48
 
49
  StateEnc --> HState["Contextual Sequence Embeddings h_state"]
 
56
  Scorer --> Probs["Decision Probabilities P(Y_k | X)"]
57
  ```
58
 
59
+ 1. **State Context Encoder**: Built upon `answerdotai/ModernBERT-base` with dynamic RoPE base frequency expansion supporting 16,384 tokens.
60
  2. **Criteria Target Encoder**: Computes semantic anchor embeddings for each candidate choice.
61
  3. **M-Step Recurrent Latent Predictive Verifier**: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
62
+ 4. **Calibrated Latent Scorer**: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.
63
 
64
  ---
65
 
 
69
 
70
  | Metric / Area | Score | Notes |
71
  | :--- | :---: | :--- |
72
+ | **Decision Index (Balanced Skill)** | **23.16** | Full 44-benchmark panel (+26.1% over v1.0) |
73
+ | **Raw Index (Balanced Raw)** | **41.92** | Full panel |
74
+ | **Breadth Skill** | **22.40** | Geometric mean across all 5 domains |
75
+ | **Tools & Automation** | **31.90% Skill** | 42.11% Raw (95.63% coverage) |
76
+ | **Retrieval & Classification** | **28.23% Skill** | 42.58% Raw (100.00% coverage) |
77
+ | **Knowledge & Reasoning** | **25.26% Skill** | 43.71% Raw (99.92% coverage) |
78
+ | **Arts & Human Taste** | **16.73% Skill** | 38.67% Raw (100.00% coverage) |
79
+ | **Language Understanding** | **13.44% Skill** | 40.81% Raw (100.00% coverage) |
80
 
81
  ### Head-to-Head Benchmark Highlights
82
 
83
+ | Benchmark Domain | Metric | DM-JEPA 1.1 Result | Leaderboard Champion (`jev`) |
84
  | :--- | :--- | :---: | :---: |
85
  | **GSM8K (Arithmetic Reasoning)** | Raw Accuracy / Skill | **100.00% / 100.00%** | 79.87% / 75.65% |
86
  | **Home Appliance Simulator** | Field Accuracy / Field Skill | **91.12% / 82.25%** | 52.27% / 52.27% |
87
  | **OpenJev High-Trust Suite** | Raw Accuracy / Brier | **100.00% / 0.0000** | ~92.00% |
88
  | **Jevbench-Hard** | Raw Accuracy / Trap Avoidance | **85.59% / 92.45%** | ~78.00% |
89
+ | **Median Inference Latency** | Time per Request | **34.3 ms** | ~950 ms (~28× faster) |
90
 
91
  ---
92
 
 
108
  model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)
109
  ```
110
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
  ---
112
 
113
  ## 🏷️ Citation & Contact
 
115
  ```bibtex
116
  @misc{dangerlabs2026dmjepa,
117
  author = {Danger Labs},
118
+ title = {DM-JEPA 1.1: Non-Autoregressive System 1 Decision Model with Stretched Context and Latent Predictive Verification},
119
  year = {2026},
120
  publisher = {Hugging Face},
121
  howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}