README.md

---
language:
- en
license: mit
tags:
- decision-making
- jepa
- system-1
- decision-intelligence
- non-autoregressive
- fast-inference
- modernbert
pipeline_tag: feature-extraction
---

# DM-JEPA 1.1: Decision-Making Joint Embedding Predictive Architecture

<div align="center">

**Developed by Danger Labs**  
*Ultra-fast, Non-Autoregressive System 1 Decision Model with 16k Extended Context*

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Decision Index](https://img.shields.io/badge/Decision--Index-0.2.1%20%7C%2023.16-blue)](https://huggingface.co/datasets/DangerLabs/decision-index-results)

</div>

---

## ⚡ Overview

**DM-JEPA 1.1** is a non-autoregressive **System 1 Decision Engine** built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning **entirely within latent thought space**.

- **Sub-35ms Latency**: Delivers **34.3 ms** median inference latency per decision request, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs (~28× faster).
- **Extended 16k Context**: Supports up to 16,384 tokens of state context and 512 tokens per decision criterion, answering 99.84% (150,518 / 150,759) of the Decision Index full panel.
- **Zero Token Hallucination**: Directly predicts compatibility energy between environmental state facts and candidate decision criteria without free-form token hallucination.
- **Listwise Joint Decision Space**: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
- **Official Decision Index Full-Panel Score**: **23.16 Balanced Skill** / **41.92 Balanced Raw** across all **150,759 requests** of the full Decision Index suite ([view evaluation dataset](https://huggingface.co/datasets/DangerLabs/decision-index-results)).

---

## 🔬 Architecture

```mermaid
flowchart TD
    State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder (RoPE Scaled to 16k)"]
    Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
    
    StateEnc --> HState["Contextual Sequence Embeddings h_state"]
    CritEnc --> SOpt["Candidate Target Embeddings s_options"]
    
    HState & SOpt --> Predictor["M-Step Latent Predictive Verifier (GRU-Gated Thought Rollout)"]
    Predictor --> SHat["Anticipated Latent Target Vector s_hat"]
    
    SHat & SOpt --> Scorer["Latent Compatibility Scorer (Cosine Similarity * Temperature)"]
    Scorer --> Probs["Decision Probabilities P(Y_k | X)"]
```

1. **State Context Encoder**: Built upon `answerdotai/ModernBERT-base` with dynamic RoPE base frequency expansion supporting 16,384 tokens.
2. **Criteria Target Encoder**: Computes semantic anchor embeddings for each candidate choice.
3. **M-Step Recurrent Latent Predictive Verifier**: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
4. **Calibrated Latent Scorer**: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.

---

## 📊 Decision Index Full-Panel Performance (150,759 Requests)

Evaluated across the complete **150,759-request suite** of the Decision Index (0.2.1 protocol):

| Metric / Area | Score | Notes |
| :--- | :---: | :--- |
| **Decision Index (Balanced Skill)** | **23.16** | Full 44-benchmark panel (+26.1% over v1.0) |
| **Raw Index (Balanced Raw)** | **41.92** | Full panel |
| **Breadth Skill** | **22.40** | Geometric mean across all 5 domains |
| **Tools & Automation** | **31.90% Skill** | 42.11% Raw (95.63% coverage) |
| **Retrieval & Classification** | **28.23% Skill** | 42.58% Raw (100.00% coverage) |
| **Knowledge & Reasoning** | **25.26% Skill** | 43.71% Raw (99.92% coverage) |
| **Arts & Human Taste** | **16.73% Skill** | 38.67% Raw (100.00% coverage) |
| **Language Understanding** | **13.44% Skill** | 40.81% Raw (100.00% coverage) |

### Head-to-Head Benchmark Highlights

| Benchmark Domain | Metric | DM-JEPA 1.1 Result | Leaderboard Champion (`jev`) |
| :--- | :--- | :---: | :---: |
| **GSM8K (Arithmetic Reasoning)** | Raw Accuracy / Skill | **100.00% / 100.00%** | 79.87% / 75.65% |
| **Home Appliance Simulator** | Field Accuracy / Field Skill | **91.12% / 82.25%** | 52.27% / 52.27% |
| **OpenJev High-Trust Suite** | Raw Accuracy / Brier | **100.00% / 0.0000** | ~92.00% |
| **Jevbench-Hard** | Raw Accuracy / Trap Avoidance | **85.59% / 92.45%** | ~78.00% |
| **Median Inference Latency** | Time per Request | **34.3 ms** | ~950 ms (~28× faster) |

---

## 🚀 Quickstart

### Installation

```bash
pip install torch transformers huggingface_hub safetensors
```

### Loading the Model

```python
from modeling_dm_jepa import DMJEPA
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)
```

---

## 🏷️ Citation & Contact

```bibtex
@misc{dangerlabs2026dmjepa,
  author = {Danger Labs},
  title = {DM-JEPA 1.1: Non-Autoregressive System 1 Decision Model with Stretched Context and Latent Predictive Verification},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}
}
```

Published by **Danger Labs** · [Hugging Face Organization](https://huggingface.co/DangerLabs)