Initial publish: native C++ port of prajjwal1/bert-tiny (Sajal Labs exp11-14)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,129 @@
+---
+license: mit
+language:
+- en
+tags:
+- bert
+- native-inference
+- sajal-labs
+- transformer
+- feature-extraction
+base_model: prajjwal1/bert-tiny
+pipeline_tag: feature-extraction
+---
+
+# bert-tiny — Native C++ Port (Sajal Labs)
+
+**This is not a new model.** The weights, architecture, and pretraining are
+entirely [`prajjwal1/bert-tiny`](https://huggingface.co/prajjwal1/bert-tiny) by Prajjwal
+Bhargava (MIT license) — a compact pretrained BERT encoder introduced in
+Turc et al. 2019 ("Well-Read Students Learn Better") and ported to
+Hugging Face for Bhargava et al. 2021 ("Generalization in NLI"). **Please
+cite both papers if you use this model** (citations below).
+
+**What Sajal Labs added**: a from-scratch native C++ port of the encoder
+and the WordPiece tokenizer — no PyTorch, no `transformers`, no Python at
+inference time — with rigorous equivalence and benchmark validation against
+the original. See `research/experiments/exp11-real-pretrained-transformer`
+through `exp14-wordpiece-native` in the Sajal Labs repo for full methodology.
+
+## Model details (unchanged from the original)
+
+- **Architecture**: BERT encoder, 2 layers, hidden=128, heads=2, intermediate=512
+- **Vocabulary**: 30522 WordPiece tokens (bert-base-uncased vocab)
+- **Parameters**: 4,385,920
+- **Precision**: fp32
+- **Base model license**: MIT (prajjwal1/bert-tiny)
+- **Port license**: MIT (Sajal Labs' C++ code)
+
+## What was verified (Sajal Labs' contribution)
+
+**Tokenizer**: 17/17 real test sentences — including contractions ("don't"),
+hyphenation ("COVID-19"), an out-of-vocabulary word forcing an 11-piece
+subword split, and an email address — produced **byte-identical token IDs**
+to the original `BertTokenizerFast`. This is an exact-match bar, not a
+tolerance: tokenization is deterministic.
+
+**Encoder**: max absolute error 9.54e-06 (hidden states),
+2.19e-06 (pooled `[CLS]` output), cosine similarity
+~1.0, across 10 real sentences of varying length (4-25 tokens) — fp32-scale
+agreement, consistent with floating-point non-associativity between two
+independent implementations (not a bug; see the repo's `research/papers.md`).
+
+## Benchmark (single request, Apple M4 CPU — full data in `benchmark_results.json`)
+
+True end-to-end cold invocation (process spawn → raw text in → prediction out
+→ process exit, external wall-clock):
+
+| Implementation | Cold invocation p50 |
+|---|---:|
+| **Native C++ (Sajal runtime)** | 11.37ms |
+| ONNX Runtime + lean tokenizer | 95.72ms (8.4x slower) |
+| ONNX Runtime + 🤗 transformers tokenizer | 2495.61ms (219.6x slower) |
+| PyTorch + 🤗 transformers | 4973.91ms (437.6x slower) |
+
+Worth knowing before you read too much into "ONNX Runtime": its own cold-start
+number depends heavily on which tokenizer library it's paired with — using
+`transformers` for convenience costs ~25x more than using the lean, standalone
+`tokenizers` library for the exact same token IDs. Native sidesteps that
+whole dependency-choice question by construction. Full discussion in exp14.
+
+**Honest scope note on the warm-loop numbers**: native's advantage is
+*not* unconditional the way cold-invocation is — Sajal Labs found it
+depends on model width (`hidden_size`), with a measured crossover around
+`hidden≈250` on this hardware (exp12/exp13). This model's `hidden=128`
+sits comfortably below that, so native keeps a real warm-loop edge too —
+but that's a property of this model's size, not a general claim.
+
+## How to use
+
+### Native (Sajal runtime, zero Python)
+
+```bash
+sajal run <this_directory> "The quick brown fox jumps over the lazy dog."
+```
+
+### PyTorch / transformers (the original)
+
+```python
+from transformers import BertModel, BertTokenizerFast
+model = BertModel.from_pretrained("prajjwal1/bert-tiny")
+tokenizer = BertTokenizerFast.from_pretrained("prajjwal1/bert-tiny")
+```
+
+### ONNX Runtime
+
+```python
+import onnxruntime as ort
+session = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
+# feed input_ids/attention_mask/token_type_ids from any WordPiece tokenizer
+```
+
+## Citations (required if you use this model)
+
+```bibtex
+@article{turc2019distillation,
+  title={Well-Read Students Learn Better: On the Importance of Pre-training Compact Models},
+  author={Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
+  journal={arXiv preprint arXiv:1908.08962v2},
+  year={2019}
+}
+
+@misc{bhargava2021generalization,
+  title={Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics},
+  author={Bhargava, Prajjwal and Drozd, Aleksandr and Rogers, Anna},
+  year={2021},
+  eprint={2110.01518},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+```
+
+## Files
+
+- `model.safetensors` — the original weights, HF/PyTorch-ecosystem format
+- `model.onnx` + `model.onnx.data` — ONNX Runtime-compatible export (weights externalized to the `.data` file; both are required together)
+- `word_embeddings.bin`, `position_embeddings.bin`, `token_type_embeddings.bin`, `emb_ln_*.bin`, `layer{i}_*.bin`, `pooler_*.bin` — raw native Sajal runtime format
+- `vocab.txt`, `tokenizer_config.txt` — WordPiece vocabulary + config for the native tokenizer port
+- `config.json` — architecture metadata
+- `benchmark_results.json` — full machine-readable benchmark/equivalence data behind the numbers above

benchmark_results.json

@@ -0,0 +1,28 @@
+{
+  "hardware": "Apple M4, 10 cores, 16GB RAM, macOS (see research/environment.md)",
+  "methodology": "research/experiments/exp11-real-pretrained-transformer/results.md, exp12-warm-latency-width-depth/results.md, exp13-width-threshold/results.md, exp14-wordpiece-native/results.md",
+  "equivalence_native_vs_pytorch": {
+    "note": "native encoder + tokenizer vs. HF BertModel + BertTokenizerFast, 10 real sentences (lengths 4-25 tokens)",
+    "hidden_state_max_abs_error": 9.5367431640625e-06,
+    "pooled_output_max_abs_error": 2.1904706954956055e-06,
+    "mean_cosine_similarity_pooled": 1.000000035762787
+  },
+  "tokenizer_equivalence": "17/17 test sentences (incl. contractions, hyphens, OOV words, emails) produced byte-identical token IDs to HF's real tokenizer \u2014 see exp14",
+  "cold_invocation_ms_p50_precomputed_tokens": {
+    "native": 8.747625000069092,
+    "onnx_runtime_cpu": 87.30802099944412,
+    "pytorch_plus_transformers": 5095.878874999471
+  },
+  "cold_invocation_ms_p50_raw_text_full_pipeline": {
+    "native": 11.36558350026462,
+    "onnx_runtime_cpu_plus_lean_tokenizer": 95.72114549973776,
+    "onnx_runtime_cpu_plus_transformers_tokenizer": 2495.6148125002073,
+    "pytorch_plus_transformers": 4973.907166500794
+  },
+  "warm_loop_ms_p50": {
+    "native": 0.11565,
+    "onnx_runtime_cpu": 0.1342290006505209,
+    "pytorch_eager": 0.27608300115389284
+  },
+  "warm_loop_note": "Unlike cold invocation, native's warm-loop edge is NOT unconditional \u2014 it depends on model width; see exp12/exp13. At this model's hidden_size=128 it's below the measured ~250 crossover, so native keeps a real edge."
+}

config.json

@@ -0,0 +1,20 @@
+{
+  "base_model": "prajjwal1/bert-tiny",
+  "base_model_license": "MIT",
+  "architecture": "bert",
+  "num_hidden_layers": 2,
+  "hidden_size": 128,
+  "num_attention_heads": 2,
+  "intermediate_size": 512,
+  "vocab_size": 30522,
+  "max_position_embeddings": 512,
+  "layer_norm_eps": 1e-12,
+  "num_parameters": 4385920,
+  "precision": "fp32",
+  "original_framework": "pytorch",
+  "sajal_runtime_compatible": true,
+  "sajal_native_port": {
+    "encoder": "native/bert_model.hpp",
+    "tokenizer": "native/wordpiece.hpp (real WordPiece, ASCII-scoped)"
+  }
+}

config.txt

@@ -0,0 +1 @@
+2 128 2 512 30522 512 1e-12

emb_ln_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad4ffafdc100192191cca090b76519c689068fb8b8ee8a5d770bf75fd4f556ce
+size 512

emb_ln_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ebbf221e9922fa7e96a65d83860e14fde877c37901794188dbc0119fb04d78d
+size 512

layer0_attn_ln_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c21382c9a7f96e18dd8c1740f50030e73a52bfedc6185dfb36c4ea309cfc65d
+size 512

layer0_attn_ln_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:326a90ea729d7f1c8dc0b599f49926980d59d73cdcff7c2c80dea2dbf6b094ef
+size 512

layer0_attn_out_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2885546bee40403065c3d4858a17edd09ce84d4df58e06855e5f290b01bae1c4
+size 512

layer0_attn_out_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f22f2bec4a303e4d0c3d326ec101b7042d194c2e1522b5ebd3377dfde2e169b3
+size 65536

layer0_ff1_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0652ab6a4eba5562dcce4be341f4e8bdca1509394b6035c02d27a2697a07eb85
+size 2048

layer0_ff1_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1837abddcee6e6d755e3c5276021a60d8d7be3c568b9d16ce8c3c2fc517794b3
+size 262144

layer0_ff2_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bdc9b2cdec6b03f998f8e7c170415598f70c18b989f6586f567eb33d99e3f06
+size 512

layer0_ff2_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d2afe109364e857354a64679de404820cf3cadf7c5241046c97288db5317a25
+size 262144

layer0_ff_ln_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78d681625f5b35c3eb164d46f4033817992bf1262ab4ff063b03519464862e65
+size 512

layer0_ff_ln_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d9108431697bd6faea8055f0a61ea5ef07dc367c013d1436627d02d86d6e66a
+size 512

layer0_k_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55adb28810a6020320d74a41284b73bb51bf4bf5226198e53cab720f53531f0c
+size 512

layer0_k_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b950f921ee111d9d70682f89ed855e7a2e9d6a6af50555027ce7cbed3757cc7
+size 65536

layer0_q_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c0ae37a6f1e963fedea5b81c732f3230e7f0cddc592938a80c461800f32fe3a
+size 512

layer0_q_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0acebbdf18ed97979c2a5d205d5785fb4bae30c9df08064823ca53150ebfd635
+size 65536

layer0_v_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18c4ec14ca13f7fe4bab91c55eb9b02c012ed073d3c56beaed45ad5f067fcc22
+size 512

layer0_v_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ea5868004c66fc912fe74e5c9238ac6d91cc3ecb83729dfeb1e0f5ab1c187df
+size 65536

layer1_attn_ln_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:106941eea5efc2b3ca11072f55a543bdae2ee6dd592b4b2d4bf6230b70cf28df
+size 512

layer1_attn_ln_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cce4d623046843995bbbc4c582abb76c615ae0da97cc4489ec24effcdb36421
+size 512

layer1_attn_out_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d841b4359077aba819ba8753cf639774d159c18b207e72b1d31997161d748e8b
+size 512

layer1_attn_out_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb091a113ba8dc74d5005b20dc45edebf8a82ff12df7a18532eeb88eb92eaf96
+size 65536

layer1_ff1_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad2916a72a70f197e6252d5df3e1dea61e4c67bd1ec672944360b4ef7c3ab2b1
+size 2048

layer1_ff1_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a08978aa217acb8aa2d956d728a9b5d39d1924af217d3bcce91322a088046606
+size 262144

layer1_ff2_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa08e02a0a4bd1b01b60dcf1194e6552733dc8cbe8a45543a62532f9e96ef268
+size 512

layer1_ff2_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb9898b9df5f9f55e0f40b10bb25c0a14d6f97412c8a87e6b553ea2237dfae50
+size 262144

layer1_ff_ln_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5a0eddf8c9d2d8d2baaf109424fe8233224330347f65bffed0dff0c186fe6d3
+size 512

layer1_ff_ln_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0fcf5d3d8a4dbb1a9ed8b8c3fbf10c568d6d28262d0d69e5bafd1adb482b518
+size 512

layer1_k_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b0eb16ecfae12acae7c2add258a2734db4d46e4ae9a3374a3065e9206a915c6
+size 512

layer1_k_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff567742407f7f8c032754b56f31b718455bdcae21c0b287027f44f07f481769
+size 65536

layer1_q_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebcb022f66703ac7f8187146e9f2051dc01f5ad81629552887784138a8557861
+size 512

layer1_q_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f591ff3f97c84d002964ae49a5c3672ed00320db444e49fa843419d91e69db5b
+size 65536

layer1_v_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8332addbc11ac4437819dbea9b4dd584611017613bf12b8f40c5d5b24a4e8c3
+size 512

layer1_v_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9de7af9f8abd2d730f88d426ed40e1d63a7643dd85fdfac5467137526db10f29
+size 65536

model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b501c2f522a0ab5d6d2297fdd7d3a24bbd5a1b1e84f563dde70fe749df186ef8
+size 23092

model.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9148713852692f5ce42c2ed1c1b62bc7239027f0636056a7fcc5e1e0cacc843b
+size 17593344

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b421978f82848f6538bb99b69036e08f28760dd87116e5121a5764e92f2e8b27
+size 17547888

pooler_bias.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7da0faaef5689a538d0117b0414260814fb91fc3d886db97045e0d5eccd0f60
+size 512

pooler_weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d288a0b6aff69cba192b299e3d12f948cbc5b43393f0198673dc4ea5914e4bb3
+size 65536

position_embeddings.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b17b9d2b78503b0057db042854f79ad83c8eb8a8f022c5e07f9b4a98283bc480
+size 262144

token_type_embeddings.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d0abc942e607e690c4e1229c28ced11c8191b1955fda71c81a196ca07869ecb
+size 1024

tokenizer_config.txt

@@ -0,0 +1 @@
+1 101 102 100 0

word_embeddings.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f49cea2ec944d799022612633ec594eddfb82cd81c54f98be79ef0c58d5cb0ff
+size 15627264