Add int8 (Q8_0-style) RT-J checkpoints + model card

README.md

@@ -0,0 +1,93 @@
+---
+license: cc-by-nc-sa-4.0
+base_model: stanford-star/rt-j
+tags:
+- relational-deep-learning
+- relational-databases
+- tabular
+- tabular-classification
+- tabular-regression
+- foundation-model
+- in-context-learning
+- quantized
+- int8
+- relational-transformer
+datasets:
+- stanford-star/the-join
+- stanford-star/relbench
+---
+
+# RT-J — int8 (Q8_0-style) quantized checkpoints
+
+Int8 quantizations of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j),
+the Relational Transformer foundation model for in-context learning over
+relational databases. Produced and consumed by the
+[RelativeDB](https://github.com/RelativeDB) native inference engine
+(`cpp/rt_quantize`); runs on its CPU (Accelerate / portable SIMD), Metal/MPS,
+and CUDA backends.
+
+| File | Task head | Size |
+|---|---|---|
+| `classification/model.q8.safetensors` | classification / ranking (logits — apply sigmoid) | 86 MB |
+| `regression/model.q8.safetensors` | regression / forecasting (normalized values) | 86 MB |
+
+(vs. 171 MB bf16 upstream, 342 MB fp32 in memory.)
+
+## Quantization format
+
+Q8_0-style, plain safetensors — no custom container:
+
+- Every **transformer-block projection** (`wq/wk/wv/wg`, `wo`, `ffn.w1/w2/w3`
+  across all 12 blocks — ~99% of parameters) is stored as an `I8` tensor with
+  a per-output-row symmetric fp32 scale in a `<name>.q_scale` companion
+  tensor: `W[o,i] ≈ q[o,i] * scale[o]`, `scale[o] = max|W[o,:]| / 127`.
+- The value/col-name **encoders, decoder head, norms, biases and mask
+  embeddings stay fp32** (like llama.cpp's embedding/output layers):
+  input-side quantization error would propagate through all 12 blocks for a
+  negligible size win.
+
+Any safetensors reader can load these files; dequantization is one line per
+row. The RelativeDB loader dequantizes at load time and runs its
+golden-verified fp32 pipeline on all backends.
+
+## Accuracy
+
+Measured on the RelativeDB golden batch (B=5, S=16 churn example) against the
+PyTorch reference, identical on CPU and Metal/MPS:
+
+| Metric | fp32 | int8 |
+|---|---|---|
+| `yhat` max abs. error vs. torch | 3.9e-3 | 1.1e-2 |
+| `yhat` mean abs. error | 5.1e-4 | 3.5e-3 |
+| target-score sign / ranking | preserved | preserved |
+
+Worst per-row mean relative weight error: ≈5%.
+
+## Usage (RelativeDB native engine)
+
+```bash
+# direct path
+./build/rt_test testdata classification/model.q8.safetensors --quantized --device mps
+
+# via the Java / Python / Rust bindings: place the .q8 file next to the fp32
+# checkpoint (or point at a directory containing it) and opt in with
+export RELATIVEDB_RT_QUANTIZED=1
+```
+
+The C ABI (`rt_model_load`) accepts these files directly — the format is
+auto-detected from the tensor dtypes.
+
+## Reproduce
+
+```bash
+cmake -B build -S cpp && cmake --build build -j
+./build/rt_quantize <rt-j>/classification/model.safetensors classification/model.q8.safetensors
+./build/rt_quantize <rt-j>/regression/model.safetensors regression/model.q8.safetensors
+```
+
+## License & attribution
+
+Derivative of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j)
+(Stanford STAR lab), redistributed under the same
+**CC-BY-NC-SA-4.0** license. Architecture and training details are described
+in the upstream model card; only the weight storage format differs here.

classification/config.json

@@ -0,0 +1,22 @@
+{
+  "codename": "rt-j-clf",
+  "name": "RT-J Classifier",
+  "task_type": "clf",
+  "checkpoint_file": "model.q8.safetensors",
+  "embedding_model": "all-MiniLM-L12-v2",
+  "d_text": 384,
+  "model": {
+    "num_blocks": 12,
+    "d_model": 512,
+    "d_text": 384,
+    "num_heads": 8,
+    "d_ff": 2048,
+    "materialize_attn_masks": true
+  },
+  "provenance": {
+    "run_id": "pjbmgvu2",
+    "swa_step": 58000,
+    "project": "2026-04-17",
+    "recipe": "real_mixed_v6_5_486t"
+  }
+}

classification/model.q8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c68e3b5e1631434fb646f196bce04cd7e41e8f64f346370280df19cdc7fba52a
+size 88107204

regression/config.json

@@ -0,0 +1,22 @@
+{
+  "codename": "rt-j-reg",
+  "name": "RT-J Regressor",
+  "task_type": "reg",
+  "checkpoint_file": "model.q8.safetensors",
+  "embedding_model": "all-MiniLM-L12-v2",
+  "d_text": 384,
+  "model": {
+    "num_blocks": 12,
+    "d_model": 512,
+    "d_text": 384,
+    "num_heads": 8,
+    "d_ff": 2048,
+    "materialize_attn_masks": true
+  },
+  "provenance": {
+    "run_id": "fsuj4wdl",
+    "swa_step": 18000,
+    "project": "2026-04-17",
+    "recipe": "real_mixed_v6_5_486t"
+  }
+}

regression/model.q8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e5023af00d8410c8e3c394b03538b224cfeccc7703c1825c766f647e15276a
+size 88107204