tokenfold-select-v2 / NOTICE.md
snchimata's picture
Release Tokenfold Select v2 ONNX and Safetensors preview
78736d9 verified
|
Raw History Blame Contribute Delete
3.02 kB

Trained-model change notice

This trained Tokenfold compact12 candidate derives from IBM Granite reranker English R2, revision d09d3d6971b689bf9c23839e45a470874d46e13a. The original provider model card and complete Apache-2.0 text are retained. No IBM endorsement is claimed. Tokenfold pruned to original layers 0–11, added content-range mean pooling and a scalar group head, performed TRAIN-only head warmup, then trained the encoder/head with balanced supervision, hard-pair ranking and original foundation-feature distillation. The checkpoint is trained; the separately retained HISTORICAL-INITIALIZER-NOTICE is initializer evidence, not the final description of this checkpoint or a substitute for this change notice.

Credit: U.S. Geological Survey earthquake event information. CPython source revision 2abcf904b8dac8c999d2b3aac76681abb333798a; complete PSF/historical license text retained. No source corpus, reader outputs or training caches are included.

The maintainer approved distribution of these trained weights/tokenizer under Apache-2.0 and the recorded source applicability. The model LICENSE and retained upstream/CPython texts apply within their respective scopes; this is not a blanket data clearance or upstream endorsement. Runtime source remains Apache-2.0. No ML/CUDA/native dependency wheels or drivers are bundled; caller-provisioned dependency redistribution requires its own applicable notices. Development admission is not quality/performance qualification. Publication authorization is the separate explicit maintainer disposition recorded here.

Distribution format: ONNX FP32 graph with external tensor data, derived from the pinned source checkpoint; source checkpoint bytes are not included. Only absolute-path debug metadata was removed from the distribution graph; numerical operators, tensors, IO and external weights are unchanged. Independent CPU synthetic parity is recorded in evidence/onnx-cpu-validation.json and does not constitute task-quality, performance, rights or release approval.

Safetensors alternative: the full trained FP32 encoder/head state is also serialized as model.safetensors, with a strict custom joint-group CPU loader, canonical architecture, unchanged tokenizer and separately pinned environment. No pickle checkpoint is included. Distribution config corrects the architecture metadata; original source config is retained. Weight-production/parity evidence and its terminal reporting failure are preserved separately, not upgraded into a successful run or new rights/qualification grant. ONNX remains the default.

Independent validation of the existing Safetensors file subsequently passed: all tensor bytes matched the source and both synthetic CPU score/rank cases matched exactly. No reconversion occurred. See evidence/safetensors-cpu-validation.json. Earlier failed histories stay unchanged; this does not confer quality/performance, redistribution or release approval. ONNX remains the default format.