Adds eval/hf/ so the checkpoint loads in a one-liner without any of our
repo on the user's machine:
model = AutoModelForCausalLM.from_pretrained(
"prometheus04/matilda-152m-v15-7.5B",
trust_remote_code=True, dtype="bfloat16").cuda().eval()
Verified bit-exact (max_abs_diff = 0.0, argmax agreement = 100%) against
the native matilda.model.Transformer on the same input.
Files:
- configuration_matilda.py — MatildaConfig(PretrainedConfig) mirroring our
ModelConfig (qk_norm, mlp_activation, final_logit_softcap, ...) with
auto_map populated for AutoConfig/AutoModel/AutoModelForCausalLM.
- modeling_matilda.py — MatildaForCausalLM(PreTrainedModel, GenerationMixin)
+ MatildaModel + MatildaBlock + MatildaAttention + ReLU2/SwiGLU FFNs.
Uses LigerRMSNorm when available, falls back to a plain RMSNorm on CPU.
Eager RoPE (same workaround as the training code).
- convert_to_hf.py — strips _orig_mod prefix, renames keys, clones
embed.weight into lm_head.weig…
instance run 2026-05-30: liger 0.8.0 workarounds + journal
Two patches in src/matilda/model.py to unblock the 152M hero run on the
A100 SXM4 40GB instance:
1. _import_liger() now imports torch.distributed.tensor explicitly.
Liger 0.8.0 RMSNorm calls torch.distributed.tensor.DTensor at runtime,
but torch 2.6 doesn't auto-load that submodule; first forward dies
with AttributeError (eager) or CUDA illegal memory access (compile).
2. _apply_rope() bypasses liger_rotary_pos_emb, falls back to eager.
liger-kernel 0.8.0's Triton RoPE kernel crashes torch.compile autotune
with CUDA illegal memory access at any head_dim (reproduced at 80 and
64). Eager apply_rope is <1% of step time; LigerRMSNorm + LigerFLCE
stay engaged.
Adds docs/instance-run-2026-05-30/ with the full operating journal:
- README.md: overview, what happened, live run progress
- bugs-and-fixes.md: each Liger bug, the python-edu data bug, full diag +
applied patches + why-this-and-not-other reasoning
- calibration-data.md: 363M and 152M BS sweep tables, Liger F…
v1.5 pivot: 152M (18L x 768d) hero config, ReLU2 FFN, final logit soft-cap. 350M kept as reference.
