r3lax
/

r3lax ardanila commited on
Commit
ac76798
·
0 Parent(s):

Duplicate from ardanila/sys1mlx

Browse files

Co-authored-by: Artemenko <ardanila@users.noreply.huggingface.co>

.gitattributes ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen3.5-4B-Base
5
+ base_model_relation: merge
6
+ language:
7
+ - en
8
+ - ru
9
+ tags:
10
+ - mlx
11
+ - qwen3_5
12
+ - decision-model
13
+ - multiple-choice
14
+ - vision
15
+ - experimental
16
+ - 4-bit
17
+ ---
18
+
19
+ # Sys1MLX
20
+
21
+ A local decision model for Apple Silicon, based on Qwen3.5-4B with a restored vision tower. Given text, an optional image, and user-supplied choices, it returns a choice and probabilities. Also supports `noul` and `score` question types.
22
+
23
+ **This is a custom decision model, not a text generator.** Use the included Python runtime; standard `mlx_lm.generate`, `mlx_vlm.generate`, Transformers auto-model loading, and hosted inference widgets are not supported. No new visual fine-tuning was performed. Visual decision quality is experimental.
24
+
25
+ ## Contents
26
+
27
+ - `model.safetensors`: fused language weights, MLX affine Q4 (group size 64), plus the original BF16 vision encoder; 3,034,300,803 bytes (2.826 GiB).
28
+ - `head.safetensors`: FP32 pointer head; no vocabulary projection is needed.
29
+ - Tokenizer, processor, config, portable Python runtime, provenance, licenses.
30
+ - `experimental/ane_export.py`: optional fixed-geometry FP16 Core ML export recipe. Precompiled ANE models are **not included**.
31
+
32
+ The weights were converted from base revision `1001bb4d826a52d1f399e183466143f4da7b741b` and adapter revision `70dd4088ebf4eb82d15ef57a863a5b9a98b94d6c`. The base checkpoint is Qwen3.5-4B-Base, not its instruction-tuned variant. Main checkpoint SHA-256: `68efe382da6d702a44f6f9d6ebf97788eacf023eae4ef33c732b763777443580`.
33
+
34
+ ## Run locally
35
+
36
+ Requires an Apple Silicon Mac with working MLX Metal support. Tested with Python 3.11, macOS 26.5.2, M4, 16 GiB unified memory. Smaller machines have not been validated. Allow several GiB for process memory in addition to macOS and other applications; weights alone do not determine RAM use.
37
+
38
+ Download this repository with `hf download ardanila/sys1mlx --local-dir sys1mlx`, then:
39
+
40
+ ```sh
41
+ cd sys1mlx
42
+ python3.11 -m venv .venv
43
+ source .venv/bin/activate
44
+ python -m pip install -r requirements.txt
45
+ python -m sys1mlx.cli < example.jsonl
46
+ ```
47
+
48
+ The supplied example has no image and needs no additional files. For visual decisions add `"image_path": "/path/to/picture.png"` to a request:
49
+
50
+ ```python
51
+ from sys1mlx.vision_loader import load_vision
52
+
53
+ model = load_vision(max_image_tokens=256, cache_mb=64)
54
+ result = model.predict({
55
+ "state": "Answer using the image.",
56
+ "image_path": "/path/to/picture.png",
57
+ "questions": {
58
+ "color": {
59
+ "type": "choice",
60
+ "instructions": "What color is the circle?",
61
+ "criteria": {"red": "red", "blue": "blue", "green": "green"}
62
+ }
63
+ }
64
+ })
65
+ print(result["answers"])
66
+ print(result["probabilities"]) # Unrounded probabilities, in question/option insertion order.
67
+ ```
68
+
69
+ Keep the process alive for repeated requests. Image and optional prefix caches are bounded (64 MiB default); pass `--cache-mb 0` to disable reuse or `--prefix` to enable context reuse. First load includes checksum verification and is excluded from steady-state latency measurements.
70
+
71
+ Limits: one still local image per request; at most 32 megapixels decoded; 16–512 image-token budget (256 default); packed image plus text at most 2,048 tokens. Aspect ratio is preserved by the Qwen processor. The head's probabilities have not been calibrated on image tasks.
72
+
73
+ ## Conversion recipe
74
+
75
+ 1. Load the pinned Qwen base and pinned LoRA adapter. Apply each adapter update in FP32 (`W + (alpha/r) * B @ A`), then cast to BF16.
76
+ 2. Quantize language linear/embedding layers to MLX affine 4-bit, group size 64. Keep the original visual tower in BF16, recurrent `A_log` parameters in FP32, and convert the separate pointer head to FP32 safetensors.
77
+ 3. Sanitize tensor names/layouts through the MLX Qwen3.5 model, retaining all visual weights. Strictly load every tensor. This artifact was converted with Linux CPU MLX 0.32.2; quantization can differ slightly from Metal conversion.
78
+ 4. Use the original option-token encoding and pointer readout. The runtime corrects DeltaNet Q/K normalization to the MLX-LM sum-of-squares epsilon convention; omitting this correction in mlx-vlm 0.7.2 changes outputs.
79
+ 5. Insert Qwen visual features and multimodal RoPE positions into the language stream. No additional training is performed.
80
+
81
+ Upstream training and benchmark results are not measurements of this quantized visual derivative. See [source attribution](ATTRIBUTION.md) for the original releases.
82
+
83
+ ## Local diagnostic results (not a general benchmark)
84
+
85
+ A small diagnostic set contained 18 synthetic color/shape/count/UI/position cases and 6 stock photographs from scikit-image. Native BF16 vision selected 24/24 expected answers, versus 7/24 without images. An image-swap control passed 18/18. These are tiny development checks, not held-out evidence of general vision accuracy; photographs may have appeared in pretraining.
86
+
87
+ Fresh-image native MLX latency was approximately 0.7–1.2 s on an M4, depending on image geometry and request. The observed MLX allocator peak was about 3.54 GiB; that is not total system memory. Repeated-image and prefix-cache timings are different workloads and must not be compared to fresh-image inference.
88
+
89
+ ### Experimental ANE export
90
+
91
+ ```sh
92
+ python -m pip install coremltools==9.0
93
+ python experimental/ane_export.py --grid 24
94
+ python experimental/ane_export.py --grid 32
95
+ ```
96
+
97
+ Outputs are under `coreml_exports/`, for processed 384×384 / 144 visual tokens and 512×512 / 256 tokens respectively. The exporter uses FP16 and fixed geometry. It exports only the visual tower; the language model and decision head still run on MLX. The default CLI continues to use BF16 MLX vision. Each package is approximately 633 MiB; leave ample free disk for Core ML compilation caches. Export/compile performance depends on macOS and hardware.
98
+
99
+ Paired local experiments using CPU_AND_NE for vision and MLX for language:
100
+
101
+ | Processed image | Native median | Hybrid median | Agreement with expected choice |
102
+ |---|---:|---:|---:|
103
+ | 384×384 | 660 ms | 591 ms | 36/36 across 18 cases repeated twice |
104
+ | 512×512 | 1,164 ms | 1,061 ms | 4/4 across 2 cases repeated twice |
105
+
106
+ No image-token reduction was used. Maximum absolute probability change was **0.05315** (5.31 percentage points), so this is not numerically equivalent inference. The device plan preferred ANE for encoder computation; this is not hardware utilization telemetry. Initial compilation (about 26 s for one package) is excluded. Early runs encountered native compilation warnings and disk exhaustion; isolated repeated runs passed, but broad deployment reliability is unproven. Process physical footprint fell from about 3.4 to 2.8 GiB in a separate probe; this excludes memory owned by Core ML/ANE services and is not a demonstrated total-system saving.
107
+
108
+ ## License
109
+
110
+ Apache-2.0. See [LICENSE](LICENSE) and [source attribution](ATTRIBUTION.md).
config.json ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3_5ForConditionalGeneration"
4
+ ],
5
+ "image_token_id": 248056,
6
+ "model_type": "qwen3_5",
7
+ "text_config": {
8
+ "attention_bias": false,
9
+ "attention_dropout": 0.0,
10
+ "attn_output_gate": true,
11
+ "dtype": "bfloat16",
12
+ "eos_token_id": 248044,
13
+ "full_attention_interval": 4,
14
+ "head_dim": 256,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 2560,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 9216,
19
+ "layer_types": [
20
+ "linear_attention",
21
+ "linear_attention",
22
+ "linear_attention",
23
+ "full_attention",
24
+ "linear_attention",
25
+ "linear_attention",
26
+ "linear_attention",
27
+ "full_attention",
28
+ "linear_attention",
29
+ "linear_attention",
30
+ "linear_attention",
31
+ "full_attention",
32
+ "linear_attention",
33
+ "linear_attention",
34
+ "linear_attention",
35
+ "full_attention",
36
+ "linear_attention",
37
+ "linear_attention",
38
+ "linear_attention",
39
+ "full_attention",
40
+ "linear_attention",
41
+ "linear_attention",
42
+ "linear_attention",
43
+ "full_attention",
44
+ "linear_attention",
45
+ "linear_attention",
46
+ "linear_attention",
47
+ "full_attention",
48
+ "linear_attention",
49
+ "linear_attention",
50
+ "linear_attention",
51
+ "full_attention"
52
+ ],
53
+ "linear_conv_kernel_dim": 4,
54
+ "linear_key_head_dim": 128,
55
+ "linear_num_key_heads": 16,
56
+ "linear_num_value_heads": 32,
57
+ "linear_value_head_dim": 128,
58
+ "max_position_embeddings": 262144,
59
+ "mlp_only_layers": [],
60
+ "model_type": "qwen3_5_text",
61
+ "mtp_num_hidden_layers": 1,
62
+ "mtp_use_dedicated_embeddings": false,
63
+ "num_attention_heads": 16,
64
+ "num_hidden_layers": 32,
65
+ "num_key_value_heads": 4,
66
+ "rms_norm_eps": 1e-06,
67
+ "tie_word_embeddings": true,
68
+ "use_cache": true,
69
+ "vocab_size": 248320,
70
+ "mamba_ssm_dtype": "float32",
71
+ "rope_parameters": {
72
+ "mrope_interleaved": true,
73
+ "mrope_section": [
74
+ 11,
75
+ 11,
76
+ 10
77
+ ],
78
+ "rope_type": "default",
79
+ "rope_theta": 10000000,
80
+ "partial_rotary_factor": 0.25
81
+ }
82
+ },
83
+ "tie_word_embeddings": true,
84
+ "transformers_version": "4.57.0.dev0",
85
+ "video_token_id": 248057,
86
+ "vision_config": {
87
+ "deepstack_visual_indexes": [],
88
+ "depth": 24,
89
+ "hidden_act": "gelu_pytorch_tanh",
90
+ "hidden_size": 1024,
91
+ "in_channels": 3,
92
+ "initializer_range": 0.02,
93
+ "intermediate_size": 4096,
94
+ "model_type": "qwen3_5",
95
+ "num_heads": 16,
96
+ "num_position_embeddings": 2304,
97
+ "out_hidden_size": 2560,
98
+ "patch_size": 16,
99
+ "spatial_merge_size": 2,
100
+ "temporal_patch_size": 2
101
+ },
102
+ "vision_end_token_id": 248054,
103
+ "vision_start_token_id": 248053
104
+ }
example-red.png ADDED
example-vision.jsonl ADDED
@@ -0,0 +1 @@
 
 
1
+ {"state": "Answer using the image.", "questions": {"color": {"type": "choice", "instructions": "What color is the circle?", "criteria": {"red": "red", "blue": "blue", "green": "green"}}}, "image_path": "example-red.png"}
example.jsonl ADDED
@@ -0,0 +1 @@
 
 
1
+ {"state":"The circle is red.","questions":{"color":{"type":"choice","instructions":"What color is the circle?","criteria":{"red":"red","blue":"blue","green":"green"}}}}
experimental/ane_export.py ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export the existing Kev vision tower directly to Core ML MIL, fixed grid.
2
+
3
+ Uses the fixed-grid/full-image-attention design in john-rocky/CoreML-LLM,
4
+ but reads our verified MLX weights and needs no PyTorch or model download.
5
+ No resampling or token reduction is performed. FP16 arithmetic differs from
6
+ the original BF16 tower and is explicitly evaluated, not assumed equivalent.
7
+ """
8
+ import argparse,hashlib,json,time
9
+ from pathlib import Path
10
+ import numpy as np
11
+ import mlx.core as mx
12
+ from mlx.utils import tree_flatten
13
+ from mlx_vlm.models.qwen3_vl.vision import VisionModel
14
+ from mlx_vlm.models.qwen3_vl.config import VisionConfig
15
+ import coremltools as ct
16
+ from coremltools.converters.mil import Builder as mb
17
+ from coremltools.converters.mil.mil import types
18
+
19
+ ROOT=Path(__file__).resolve().parents[1];OUT=ROOT/'coreml_exports'
20
+
21
+ def main():
22
+ p=argparse.ArgumentParser();p.add_argument('--grid',type=int,default=24)
23
+ p.add_argument('--attention',choices=['manual','sdpa'],default='manual');p.add_argument('--out');args=p.parse_args()
24
+ dest=OUT/(args.out or f'grid-{args.grid}-{args.attention}');dest.mkdir(parents=True,exist_ok=False)
25
+ config=json.loads((ROOT/'config.json').read_text())['vision_config']
26
+ assert not config['deepstack_visual_indexes']
27
+ checkpoint=ROOT/'model.safetensors'
28
+ h=hashlib.sha256()
29
+ with checkpoint.open('rb') as f:
30
+ for b in iter(lambda:f.read(8*1024**2),b''):h.update(b)
31
+ expected=json.loads((ROOT/'provenance.json').read_text())['sha256'];assert h.hexdigest()==expected
32
+ model=VisionModel(VisionConfig.from_dict(config))
33
+ loaded=mx.load(str(checkpoint));model.load_weights([(k.removeprefix('vision_tower.'),v) for k,v in loaded.items() if k.startswith('vision_tower.')],strict=True)
34
+ del loaded;model.eval();mx.eval(model.parameters());mx.clear_cache()
35
+ weights={k:np.array(v.astype(mx.float32)).astype(np.float16) for k,v in tree_flatten(model.parameters())}
36
+ grid=mx.array([[1,args.grid,args.grid]],dtype=mx.int32);n=args.grid**2;d=config['hidden_size'];heads=config['num_heads'];hd=d//heads
37
+ pos=np.array(model.fast_pos_embed_interpolate(grid).astype(mx.float32)).astype(np.float16)
38
+ freqs=model.rot_pos_emb(grid);cos=np.array(mx.tile(mx.cos(freqs),(1,2))).astype(np.float16)[:,None,:]
39
+ sin=np.array(mx.tile(mx.sin(freqs),(1,2))).astype(np.float16)[:,None,:]
40
+ del model;mx.clear_cache()
41
+ (dest/'manifest.json').write_text(json.dumps({'checkpoint_sha256':expected,'grid':[1,args.grid,args.grid],
42
+ 'image_tokens':n//4,'config':config,'coremltools':ct.__version__,'attention':args.attention,
43
+ 'precision':'FP16 weights converted from original BF16; FP16 Core ML computation',
44
+ 'script_sha256':hashlib.sha256(Path(__file__).read_bytes()).hexdigest()},indent=2)+'\n')
45
+ (dest/'ane_export.py').write_bytes(Path(__file__).read_bytes())
46
+ def linear(x,name):return mb.linear(x=x,weight=weights[name+'.weight'],bias=weights[name+'.bias'],name=name.replace('.','_'))
47
+ def norm(x,name):return mb.layer_norm(x=x,axes=[-1],gamma=weights[name+'.weight'],beta=weights[name+'.bias'],epsilon=np.float16(1e-6))
48
+ def rope(x):
49
+ a,b=mb.split(x=x,num_splits=2,axis=-1)
50
+ rotated=mb.concat(values=[mb.mul(x=b,y=np.float16(-1)),a],axis=-1)
51
+ return mb.add(x=mb.mul(x=x,y=cos),y=mb.mul(x=rotated,y=sin))
52
+ @mb.program(input_specs=[mb.TensorSpec(shape=(n,1536),dtype=types.fp16)],opset_version=ct.target.iOS18)
53
+ def program(pixels):
54
+ # MLX Conv3d stores [out,time,height,width,channel]; input patches are CTHW.
55
+ w=weights['patch_embed.proj.weight'].transpose(0,4,1,2,3).reshape(d,1536)
56
+ x=mb.linear(x=pixels,weight=w,bias=weights['patch_embed.proj.bias'])
57
+ x=mb.add(x=x,y=pos)
58
+ for i in range(config['depth']):
59
+ prefix=f'blocks.{i}';a=norm(x,prefix+'.norm1');qkv=linear(a,prefix+'.attn.qkv')
60
+ qkv=mb.reshape(x=qkv,shape=[n,3,heads,hd]);q,k,v=mb.split(x=qkv,num_splits=3,axis=1)
61
+ q=rope(mb.squeeze(x=q,axes=[1]));k=rope(mb.squeeze(x=k,axes=[1]));v=mb.squeeze(x=v,axes=[1])
62
+ q,k,v=[mb.expand_dims(x=mb.transpose(x=t,perm=[1,0,2]),axes=[0]) for t in [q,k,v]]
63
+ if args.attention=='sdpa':a=mb.scaled_dot_product_attention(query=q,key=k,value=v)
64
+ else:
65
+ scores=mb.matmul(x=mb.mul(x=q,y=np.float16(hd**-.5)),y=k,transpose_y=True)
66
+ a=mb.matmul(x=mb.softmax(x=scores,axis=-1),y=v)
67
+ a=mb.reshape(x=mb.transpose(x=a,perm=[0,2,1,3]),shape=[n,d])
68
+ x=mb.add(x=x,y=linear(a,prefix+'.attn.proj'))
69
+ a=norm(x,prefix+'.norm2');a=mb.gelu(x=linear(a,prefix+'.mlp.linear_fc1'),mode='TANH_APPROXIMATION')
70
+ x=mb.add(x=x,y=linear(a,prefix+'.mlp.linear_fc2'))
71
+ x=mb.reshape(x=norm(x,'merger.norm'),shape=[n//4,4*d])
72
+ x=mb.gelu(x=linear(x,'merger.linear_fc1'),mode='EXACT')
73
+ return mb.identity(x=linear(x,'merger.linear_fc2'),name='features')
74
+ print('MIL built',flush=True);t=time.perf_counter()
75
+ converted=ct.convert(program,convert_to='mlprogram',minimum_deployment_target=ct.target.iOS18,
76
+ compute_precision=ct.precision.FLOAT16,compute_units=ct.ComputeUnit.CPU_AND_NE,skip_model_load=True)
77
+ converted.save(str(dest/'vision.mlpackage'))
78
+ print(json.dumps({'saved':str(dest),'conversion_seconds':time.perf_counter()-t}),flush=True)
79
+
80
+ if __name__=='__main__':main()
head.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81a61de32fed792718b0218366c8a4ae3cd2797276e7d97885966125455f9160
3
+ size 5245247
head_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "head_dim": 256
3
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68efe382da6d702a44f6f9d6ebf97788eacf023eae4ef33c732b763777443580
3
+ size 3034300803
preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "size": {
3
+ "longest_edge": 16777216,
4
+ "shortest_edge": 65536
5
+ },
6
+ "patch_size": 16,
7
+ "temporal_patch_size": 2,
8
+ "merge_size": 2,
9
+ "image_mean": [
10
+ 0.5,
11
+ 0.5,
12
+ 0.5
13
+ ],
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "processor_class": "Qwen3VLProcessor",
20
+ "image_processor_type": "Qwen2VLImageProcessorFast"
21
+ }
provenance.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_revision": "1001bb4d826a52d1f399e183466143f4da7b741b",
3
+ "adapter_revision": "70dd4088ebf4eb82d15ef57a863a5b9a98b94d6c",
4
+ "text_bits": 4,
5
+ "text_group_size": 64,
6
+ "vision_dtype": "bfloat16",
7
+ "format": "kev-vision-mlx-v1",
8
+ "sha256": "68efe382da6d702a44f6f9d6ebf97788eacf023eae4ef33c732b763777443580",
9
+ "bytes": 3034300803,
10
+ "conversion_device": "Linux CPU MLX 0.32.2",
11
+ "conversion_seconds": 439.0967882980001,
12
+ "conversion_spec_sha256": "6fa932b7f75c0c820845b45a9bf4c36eedbec6648ef217552683268e422a8cc6",
13
+ "base_model": "Qwen/Qwen3.5-4B-Base",
14
+ "adapter_model": "jaredpalmer/kev-4b"
15
+ }
requirements.txt ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ mlx==0.32.2
2
+ mlx-lm @ git+https://github.com/ml-explore/mlx-lm.git@a63e24c389382619eb6d9af656e3b46024be217a
3
+ mlx-vlm==0.7.2
4
+ transformers==5.17.0
5
+ pydantic==2.13.5
6
+ numpy==2.4.6
7
+ Pillow>=10
8
+ huggingface-hub==1.32.0
sys1mlx/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """Local MLX inference port of the pinned Kev Qwen3.5 decision model."""
sys1mlx/api.py ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """TypeSafe-compatible request/response shapes (POST /v1/systemone) mapped onto the single pointer primitive.
2
+
3
+ Noul -> 2 options [false, true]; answer = p(true)
4
+ Choice -> options 'name' or 'name: desc'; answer = argmax, probabilities by name, confidence
5
+ Score -> options = ordered level descriptions; answer = expected level, legend, probabilities by index
6
+ """
7
+ import re
8
+ from typing import Any, Literal, Union
9
+ from pydantic import BaseModel, Field, model_validator
10
+
11
+ JSONContent = Union[str, dict, list, int, float, bool, None]
12
+ MAX_OPTIONS = 255
13
+
14
+
15
+ class Noul(BaseModel):
16
+ type: Literal["noul"]
17
+ instructions: JSONContent
18
+ criteria: dict[str, JSONContent] | None = None
19
+
20
+
21
+ class Choice(BaseModel):
22
+ type: Literal["choice"]
23
+ instructions: JSONContent
24
+ criteria: dict[str, JSONContent]
25
+
26
+ @model_validator(mode="after")
27
+ def _check(self):
28
+ if not 1 <= len(self.criteria) <= MAX_OPTIONS: raise ValueError(f"criteria must have 1..{MAX_OPTIONS} options")
29
+ return self
30
+
31
+
32
+ class Score(BaseModel):
33
+ type: Literal["score"]
34
+ instructions: JSONContent
35
+ criteria: list[JSONContent] = Field(min_length=2, max_length=MAX_OPTIONS)
36
+
37
+
38
+ Question = Union[Noul, Choice, Score]
39
+
40
+
41
+ class SystemOneRequest(BaseModel):
42
+ state: JSONContent
43
+ model: str = "kev-latest"
44
+ questions: dict[str, Question] = Field(min_length=1)
45
+
46
+
47
+ def render(v: JSONContent, indent: int = 0) -> str:
48
+ """Flatten str | object | array into text the model sees. Field names are kept as labels."""
49
+ pad = " " * indent
50
+ if v is None: return ""
51
+ if isinstance(v, (str, int, float, bool)): return str(v)
52
+ if isinstance(v, list): return "\n".join(f"{pad}- {render(x, indent + 1).lstrip()}" for x in v)
53
+ return "\n".join(f"{pad}{k}:\n{render(x, indent + 1)}" if isinstance(x, (dict, list)) else f"{pad}{k}: {render(x)}" for k, x in v.items())
54
+
55
+
56
+ def option_text(name: str, desc: JSONContent) -> str:
57
+ return name if desc is None or desc == "" else f"{name}: {render(desc)}"
58
+
59
+
60
+ MONTHS = "January|February|March|April|May|June|July|August|September|October|November|December"
61
+ _DATE = re.compile(rf"\b(?:{MONTHS}) \d{{1,2}}, \d{{4}}\b|\b\d{{4}}-\d{{2}}-\d{{2}}\b")
62
+
63
+
64
+ def date_facts(text: str) -> str:
65
+ """Deterministic date arithmetic for the model: every pair of absolute dates found in `text`, as one sentence each
66
+ ("August 3, 2026 is 12 days after July 22, 2026."). The model cannot subtract dates reliably (issue #8); it can use a
67
+ stated day count. Returns "" when fewer than two dates are found. Dates are listed in order of first appearance."""
68
+ from datetime import datetime
69
+ found = []
70
+ for m in _DATE.finditer(text):
71
+ raw = m.group(0)
72
+ try: d = datetime.strptime(raw, "%B %d, %Y") if "," in raw else datetime.strptime(raw, "%Y-%m-%d")
73
+ except ValueError: continue
74
+ if raw not in [r for r, _ in found]: found.append((raw, d))
75
+ facts = []
76
+ for i in range(len(found)):
77
+ for j in range(i + 1, len(found)):
78
+ n = (found[j][1] - found[i][1]).days
79
+ facts.append(f"{found[j][0]} is {abs(n)} day{'s' if abs(n) != 1 else ''} {'after' if n > 0 else 'before'} {found[i][0]}." if n else f"{found[j][0]} is the same day as {found[i][0]}.")
80
+ return " ".join(facts)
81
+
82
+
83
+ def with_date_facts(state):
84
+ """State with a `date_facts` field (object states) or an appended paragraph (string states) when two or more absolute
85
+ dates appear. Opt-in preprocessing (KEV_DATE_FACTS=1 in kev.serve, --date_facts in kev.benchmark)."""
86
+ facts = date_facts(render(state))
87
+ if not facts: return state
88
+ if isinstance(state, dict): return {**state, "date_facts": facts}
89
+ if isinstance(state, list): return state + [{"date_facts": facts}]
90
+ return f"{state}\n\ndate_facts: {facts}"
91
+
92
+
93
+ def to_record(req: SystemOneRequest):
94
+ """-> internal record for encode(), plus per-question metadata to map probabilities back."""
95
+ qs, meta = [], []
96
+ for qid, q in req.questions.items():
97
+ instr = render(q.instructions)
98
+ if q.type == "noul":
99
+ c = q.criteria or {}
100
+ opts = [option_text("no", c.get("false")), option_text("yes", c.get("true"))]
101
+ meta.append({"id": qid, "type": "noul"})
102
+ elif q.type == "choice":
103
+ opts = [option_text(k, v) for k, v in q.criteria.items()]
104
+ meta.append({"id": qid, "type": "choice", "keys": list(q.criteria.keys())})
105
+ else:
106
+ opts = [render(x) for x in q.criteria]
107
+ meta.append({"id": qid, "type": "score", "legend": {str(i): render(x) for i, x in enumerate(q.criteria)}})
108
+ qs.append({"instr": instr, "options": opts, "label": 0})
109
+ return {"state": render(req.state), "questions": qs}, meta
110
+
111
+
112
+ def choice_confidence(p: list[float]) -> float:
113
+ K = len(p)
114
+ return 1.0 if K == 1 else (max(p) - 1 / K) / (1 - 1 / K)
115
+
116
+
117
+ def score_confidence(p: list[float]) -> float:
118
+ """Approximation of TypeSafe's 'distance from the modal level' statistic (exact formula unpublished):
119
+ 1 - E|level - mode| / (L - 1)."""
120
+ L = len(p); mode = max(range(L), key=lambda i: p[i])
121
+ return 1.0 - sum(pi * abs(i - mode) for i, pi in enumerate(p)) / (L - 1)
122
+
123
+
124
+ def r2(x: float) -> float:
125
+ return round(float(x), 2)
126
+
127
+
128
+ def to_answers(probs: list[list[float]], meta: list[dict]) -> dict[str, Any]:
129
+ out = {}
130
+ for p, m in zip(probs, meta):
131
+ if m["type"] == "noul":
132
+ out[m["id"]] = {"type": "noul", "noul": r2(p[1])}
133
+ elif m["type"] == "choice":
134
+ dist = {k: r2(v) for k, v in zip(m["keys"], p)}
135
+ out[m["id"]] = {"type": "choice", "choice": m["keys"][max(range(len(p)), key=lambda i: p[i])], "confidence": r2(choice_confidence(p)), "probabilities": dist}
136
+ else:
137
+ score = sum(i * pi for i, pi in enumerate(p))
138
+ out[m["id"]] = {"type": "score", "score": r2(score), "legend": m["legend"], "probabilities": {str(i): r2(v) for i, v in enumerate(p)}, "confidence": r2(score_confidence(p))}
139
+ return out
140
+
141
+
142
+ def output_tokens(tok, answers: dict) -> int:
143
+ """Billing-style figure: tokens of the serialised answers. Not a measure of generation (there is none)."""
144
+ import json
145
+ return len(tok(json.dumps(answers), add_special_tokens=False).input_ids)
sys1mlx/cli.py ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """One Kev request and response per JSON line."""
2
+ import argparse
3
+ import json
4
+ import time
5
+ from .vision_loader import load_vision
6
+
7
+
8
+ def main():
9
+ parser = argparse.ArgumentParser()
10
+ parser.add_argument('--model-dir')
11
+ parser.add_argument('--max-image-tokens', type=int, default=256)
12
+ parser.add_argument('--cache-mb', type=int, default=64)
13
+ parser.add_argument('--prefix', action='store_true')
14
+ args = parser.parse_args()
15
+ model = load_vision(args.model_dir, args.max_image_tokens, args.cache_mb)
16
+ import sys
17
+ for line in sys.stdin:
18
+ if not line.strip():
19
+ continue
20
+ try:
21
+ start = time.perf_counter()
22
+ result = model.predict(json.loads(line), use_prefix=args.prefix)
23
+ result['latency_ms'] = 1000 * (time.perf_counter() - start)
24
+ print(json.dumps(result, ensure_ascii=False, allow_nan=False), flush=True)
25
+ except (ValueError, TypeError) as error:
26
+ print(json.dumps({'error': str(error)}, ensure_ascii=False), flush=True)
27
+
28
+
29
+ if __name__ == '__main__':
30
+ main()
sys1mlx/encoding.py ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tokenizer functions extracted unchanged from Kev (Apache-2.0)."""
2
+ import re
3
+
4
+ SPECIAL = ["<|fim_prefix|>", "<|fim_middle|>", "<|box_start|>", "<|box_end|>", "<|fim_suffix|>"]
5
+
6
+ MAX_STATE, MAX_BRANCH = 384, 1024
7
+
8
+ _SPECIAL_RE = re.compile(r"<\|([A-Za-z0-9_]+)\|>")
9
+
10
+ def user_tokens(tok, text):
11
+ """Tokenize caller-supplied text so it can never produce delimiter/control tokens (option boundaries are unforgeable).
12
+ The fast tokenizer ignores split_special_tokens, so `<|name|>` is rewritten to `<¦name¦>` before tokenizing."""
13
+ return tok(_SPECIAL_RE.sub(r"<¦\1¦>", text), add_special_tokens=False).input_ids
14
+
15
+ OPT_NONE, OPT_DECIDE = -1, -2
16
+
17
+ def encode(tok, rec, max_state=MAX_STATE, max_branch=MAX_BRANCH, strict=False, option_isolation=False):
18
+ """Pack one record: [<state> ...] then per-question [<q> instr <opt> o </opt>... <decide>].
19
+
20
+ Returns ids, seg (0 = state, k = question k), pos (branch positions restart after state),
21
+ decide_idx [Q], opt_idx [Q][K] (index of </opt> token for each option), opt (per-token option index within its
22
+ question: OPT_NONE for state/instruction, 0..K-1 for option spans, OPT_DECIDE for <decide>).
23
+
24
+ option_isolation=True: every option span is its own sub-branch (it sees state + instruction + itself only), all
25
+ option spans share the same position ids, and <decide> sits at one fixed position after the longest span. Then the
26
+ per-option representations and <decide>'s attention over them are permutation-invariant by construction.
27
+ """
28
+ state_tokens = user_tokens(tok, rec["state"])
29
+ if strict and len(state_tokens) + 1 > max_state:
30
+ raise ValueError(f"state exceeds {max_state} tokens: {len(state_tokens) + 1}")
31
+ S = [tok.convert_tokens_to_ids(SPECIAL[0])] + state_tokens[: max_state - 1]
32
+ ids, seg, pos, opt = list(S), [0] * len(S), list(range(len(S))), [OPT_NONE] * len(S)
33
+ q_id, o_id, c_id, d_id = (tok.convert_tokens_to_ids(t) for t in SPECIAL[1:])
34
+ decide_idx, opt_idx = [], []
35
+ for k, q in enumerate(rec["questions"], start=1):
36
+ instr = [q_id] + user_tokens(tok, q["instr"])
37
+ spans = [[o_id] + user_tokens(tok, o) + [c_id] for o in q["options"]]
38
+ br = instr + [t for sp in spans for t in sp] + [d_id]
39
+ if len(br) > max_branch - len(S):
40
+ raise ValueError(f"branch too long: {len(br)}")
41
+ base = len(ids); p0 = len(S)
42
+ br_opt = [OPT_NONE] * len(instr) + [j for j, sp in enumerate(spans) for _ in sp] + [OPT_DECIDE]
43
+ if option_isolation:
44
+ longest = max(len(sp) for sp in spans)
45
+ br_pos = list(range(p0, p0 + len(instr))) + [p0 + len(instr) + i for sp in spans for i in range(len(sp))] + [p0 + len(instr) + longest]
46
+ else:
47
+ br_pos = list(range(p0, p0 + len(br)))
48
+ ends, cursor = [], len(instr)
49
+ for sp in spans:
50
+ cursor += len(sp); ends.append(cursor - 1)
51
+ ids += br; seg += [k] * len(br); pos += br_pos; opt += br_opt
52
+ decide_idx.append(base + len(br) - 1); opt_idx.append([base + e for e in ends])
53
+ return {"ids": ids, "seg": seg, "pos": pos, "opt": opt, "option_isolation": option_isolation, "decide_idx": decide_idx, "opt_idx": opt_idx,
54
+ "labels": [q["label"] for q in rec["questions"]], "state_truncated": len(state_tokens) + 1 > max_state}
55
+
56
+ def rows_of(enc):
57
+ """Split a packed encoding into its state and per-question branch rows.
58
+
59
+ Returns (state_ids, state_pos, rows) with rows[k] = {"ids", "pos", "decide", "opts"}: the branch tokens of question
60
+ k with their (already state-continuing) positions, and the readout offsets *within the branch*. Feeding
61
+ state + rows[k] as one causal row is equivalent to the packed block-causal form for that question, on any
62
+ architecture: the row contains exactly the tokens question k may attend to, in the same positions."""
63
+ seg = enc["seg"]; Ls = seg.count(0)
64
+ rows, start = [], Ls
65
+ for k, (d, oi) in enumerate(zip(enc["decide_idx"], enc["opt_idx"]), start=1):
66
+ end = d + 1 # <decide> is the last token of its branch
67
+ if seg[start] != k or seg[end - 1] != k: raise ValueError("branch layout mismatch")
68
+ rows.append({"ids": enc["ids"][start:end], "pos": enc["pos"][start:end], "decide": d - start, "opts": [o - start for o in oi]})
69
+ start = end
70
+ return enc["ids"][:Ls], enc["pos"][:Ls], rows
sys1mlx/runtime.py ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Kev's original encoding and pointer readout on the native MLX backbone.
2
+
3
+ Questions are independent causal rows. No vocabulary projection or generation.
4
+ The optional prefix cache is copied for each branch, including recurrent state.
5
+ """
6
+ import ast
7
+ import copy
8
+ import json
9
+ import math
10
+ from pathlib import Path
11
+ import re
12
+ import sys
13
+ import mlx.core as mx
14
+ import mlx.nn as nn
15
+ from mlx_lm.models.qwen3_5 import Model, ModelArgs
16
+
17
+ from .api import SystemOneRequest, to_record, to_answers
18
+ from .encoding import encode, rows_of
19
+
20
+
21
+ class PointerHead(nn.Module):
22
+ def __init__(self,hidden_size,head_size=256):
23
+ super().__init__()
24
+ self.q=nn.Linear(hidden_size,head_size)
25
+ self.k=nn.Linear(hidden_size,head_size)
26
+ self.scale=1/math.sqrt(head_size)
27
+
28
+ def __call__(self,decide,options):
29
+ return (self.k(options.astype(mx.float32))@self.q(decide.astype(mx.float32)))*self.scale
30
+
31
+
32
+ def backbone(config):
33
+ config=copy.deepcopy(config)
34
+ # Kev discards the language-model head; tying prevents allocating its unused copy.
35
+ config['text_config']['tie_word_embeddings']=True
36
+ return Model(ModelArgs.from_dict(config))
37
+
38
+
39
+ class DecisionModel:
40
+ def __init__(self,model,tokenizer,head):
41
+ self.model,self.tokenizer,self.head=model,tokenizer,head
42
+ model.eval();head.eval()
43
+
44
+ def encoded(self,request):
45
+ request=SystemOneRequest.model_validate(request)
46
+ record,meta=to_record(request)
47
+ enc=encode(self.tokenizer,record,strict=True)
48
+ if len(enc['ids'])>2048:raise ValueError('packed request exceeds frozen 2048-token limit')
49
+ return enc,meta
50
+
51
+ def distributions(self,enc,use_prefix=False):
52
+ state,_,rows=rows_of(enc)
53
+ prefix=None
54
+ if use_prefix:
55
+ prefix=self.model.language_model.make_cache()
56
+ self.model.model(mx.array([state]),cache=prefix)
57
+ mx.eval([c.state for c in prefix])
58
+ result=[]
59
+ for row in rows:
60
+ if prefix is None:
61
+ h=self.model.model(mx.array([state+row['ids']]))[0]
62
+ offset=len(state)
63
+ else:
64
+ cache=copy.deepcopy(prefix)
65
+ h=self.model.model(mx.array([row['ids']]),cache=cache)[0]
66
+ offset=0
67
+ z=self.head(h[offset+row['decide']],h[mx.array([offset+x for x in row['opts']])])
68
+ p=mx.softmax(z)
69
+ result.append(p.tolist())
70
+ return result
71
+
72
+ def predict(self,request,use_prefix=False):
73
+ enc,meta=self.encoded(request)
74
+ probabilities=self.distributions(enc,use_prefix=use_prefix)
75
+ return {'answers':to_answers(probabilities,meta),'probabilities':probabilities,
76
+ 'input_tokens':len(enc['ids'])}
sys1mlx/vision.py ADDED
@@ -0,0 +1,121 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """One-image Kev decisions using Qwen3.5 vision, MRoPE and the trained pointer head."""
2
+ import copy
3
+ import hashlib
4
+ from pathlib import Path
5
+ import numpy as np
6
+ from PIL import Image, ImageOps
7
+ import mlx.core as mx
8
+ from mlx_vlm.models.qwen3_vl.processing_qwen3_vl import Qwen3VLImageProcessor
9
+ from .runtime import DecisionModel, rows_of, to_answers
10
+ from .vision_cache import VisionCache
11
+ from mlx_vlm.models.cache import KVCache
12
+
13
+
14
+ class VisionDecisionModel(DecisionModel):
15
+ def __init__(self,model,tokenizer,head,processor_config,max_image_tokens=256,cache_mb=64):
16
+ super().__init__(model,tokenizer,head)
17
+ if not 16 <= max_image_tokens <= 512:
18
+ raise ValueError('max_image_tokens must be between 16 and 512')
19
+ self.max_image_tokens=max_image_tokens
20
+ self.cache=VisionCache(int(cache_mb*1024**2))
21
+ config=dict(processor_config)
22
+ config.update(min_pixels=32*32*16,max_pixels=32*32*max_image_tokens)
23
+ self.processor=Qwen3VLImageProcessor(**config)
24
+
25
+ def encoded(self,request):
26
+ request=dict(request)
27
+ image_path=request.pop('image_path',None)
28
+ enc,meta=super().encoded(request)
29
+ if image_path is None:return enc,meta
30
+ if not isinstance(image_path,str) or not image_path:
31
+ raise ValueError('image_path must be a nonempty local file path')
32
+ path=Path(image_path).expanduser()
33
+ if not path.is_file():raise ValueError('image_path is not a local file')
34
+ try:
35
+ with path.open('rb') as file, Image.open(file) as source:
36
+ if getattr(source,'n_frames',1)!=1:raise ValueError('Only one still image is supported')
37
+ if source.width*source.height>32_000_000:
38
+ raise ValueError('Image exceeds the 32-megapixel decode limit')
39
+ source_size=list(source.size)
40
+ # Hash file contents on every request: paths and timestamps can stay
41
+ # unchanged when a screenshot is overwritten. Decode only on a miss.
42
+ digest=hashlib.sha256();file.seek(0)
43
+ for block in iter(lambda:file.read(1024**2),b''):digest.update(block)
44
+ image_key=('image',digest.hexdigest(),self.max_image_tokens)
45
+ cached=self.cache.get(image_key) if self.cache.max_bytes else None
46
+ if cached is not None:
47
+ if len(enc['ids'])+cached['tokens']+2>2048:
48
+ raise ValueError('Image and text exceed packed 2048-token limit')
49
+ enc['vision']=dict(cached,pixels=None,image_key=image_key)
50
+ return enc,meta
51
+ file.seek(0)
52
+ picture=ImageOps.exif_transpose(source).convert('RGB')
53
+ data=self.processor(picture)
54
+ except (OSError,Image.DecompressionBombError) as error:
55
+ raise ValueError('Cannot decode image: '+str(error)) from error
56
+ grid=data['image_grid_thw']
57
+ count=int(np.prod(grid))//self.processor.merge_size**2
58
+ if count>self.max_image_tokens:raise ValueError('Image token budget exceeded')
59
+ if len(enc['ids'])+count+2>2048:raise ValueError('Image and text exceed packed 2048-token limit')
60
+ enc['vision']={'image_key':image_key,'pixels':mx.array(data['pixel_values']), 'grid':mx.array(grid,dtype=mx.int32),
61
+ 'tokens':count,'source_size':source_size,
62
+ 'processed_size':[int(grid[0,2])*16,int(grid[0,1])*16]}
63
+ return enc,meta
64
+
65
+ def distributions(self,enc,use_prefix=False):
66
+ state,_,rows=rows_of(enc);visual=enc.get('vision');features=None;prefix=None;prefix_key=None
67
+ model=self.model;lm=model.language_model
68
+ if visual:
69
+ c=model.config
70
+ block=[c.vision_start_token_id]+[c.image_token_id]*visual['tokens']+[c.vision_end_token_id]
71
+ state=state[:1]+block+state[1:]
72
+ if use_prefix and self.cache.max_bytes and 'image_key' in visual:
73
+ prefix_key=('prefix',visual['image_key'],tuple(state))
74
+ cached=self.cache.get(prefix_key)
75
+ if cached is not None:prefix,features=cached['cache'],cached['features']
76
+ if features is None:features=visual.get('features')
77
+ if features is None:
78
+ pixels=visual['pixels'].astype(model.vision_tower.patch_embed.proj.weight.dtype)
79
+ features,_=model.vision_tower(pixels,visual['grid']);mx.eval(features)
80
+ if self.cache.max_bytes and 'image_key' in visual:
81
+ cached={k:visual[k] for k in ('grid','tokens','source_size','processed_size')}
82
+ self.cache.put(visual['image_key'],dict(cached,features=features))
83
+ def embeddings(ids):
84
+ if visual:
85
+ return model.get_input_embeddings(ids,pixel_values=(visual['pixels'] if visual['pixels'] is not None else mx.zeros((0,))),
86
+ image_grid_thw=visual['grid'],cached_image_features=features)
87
+ return model.get_input_embeddings(ids)
88
+ if use_prefix and prefix is None:
89
+ ids=mx.array([state]);embedded=embeddings(ids);prefix=lm.make_cache()
90
+ lm.model(ids,inputs_embeds=embedded.inputs_embeds,position_ids=embedded.position_ids,cache=prefix)
91
+ mx.eval([c.state for c in prefix])
92
+ if prefix_key is not None:
93
+ # A saved prefix needs only used positions, not MLX's 256-token
94
+ # allocation reserve. This keeps a 256-token image within 64 MiB.
95
+ for cache in prefix:
96
+ if isinstance(cache,KVCache):
97
+ cache.keys=mx.contiguous(cache.keys[..., :cache.offset, :])
98
+ cache.values=mx.contiguous(cache.values[..., :cache.offset, :])
99
+ mx.eval(cache.keys,cache.values)
100
+ self.cache.put(prefix_key,{'cache':prefix,'features':features})
101
+ result=[]
102
+ for row in rows:
103
+ ids=mx.array([state+row['ids']]);embedded=embeddings(ids)
104
+ offset=len(state)
105
+ if prefix is None:
106
+ h=lm.model(ids,inputs_embeds=embedded.inputs_embeds,position_ids=embedded.position_ids)[0]
107
+ else:
108
+ h=lm.model(ids[:,offset:],inputs_embeds=embedded.inputs_embeds[:,offset:],
109
+ position_ids=embedded.position_ids[...,offset:],cache=copy.deepcopy(prefix))[0]
110
+ offset=0
111
+ z=self.head(h[offset+row['decide']],h[mx.array([offset+x for x in row['opts']])])
112
+ result.append(mx.softmax(z).tolist())
113
+ return result
114
+
115
+ def predict(self,request,use_prefix=False):
116
+ enc,meta=self.encoded(request)
117
+ probabilities=self.distributions(enc,use_prefix)
118
+ visual=enc.get('vision');extra=visual['tokens']+2 if visual else 0
119
+ return {'answers':to_answers(probabilities,meta),'probabilities':probabilities,
120
+ 'input_tokens':len(enc['ids'])+extra,
121
+ 'vision':None if not visual else {k:visual[k] for k in ('tokens','source_size','processed_size')}}
sys1mlx/vision_backbone.py ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Qwen3.5 vision with the same DeltaNet normalization as the verified text port."""
2
+ from mlx_lm.models.gated_delta import normalize_qk
3
+ from mlx_vlm.models.qwen3_5 import Model
4
+ from mlx_vlm.models.qwen3_5.language import Qwen3_5GatedDeltaNet
5
+
6
+
7
+ class Sys1GatedDeltaNet(Qwen3_5GatedDeltaNet):
8
+ def _normalize_qk(self,q,k):
9
+ # HF adds epsilon to SUM(x^2); rms_norm adds it to MEAN(x^2).
10
+ # mlx-vlm 0.7.2 passes 1e-6 directly, unlike the verified MLX-LM path.
11
+ return normalize_qk(q,k,inv_scale=self.head_k_dim**-0.5,eps=1e-6)
12
+
13
+
14
+ class Sys1VisionBackbone(Model):
15
+ def __init__(self,config):
16
+ super().__init__(config)
17
+ for layer in self.language_model.model.layers:
18
+ if layer.is_linear:
19
+ corrected=Sys1GatedDeltaNet(config.text_config)
20
+ corrected.update(layer.linear_attn.parameters())
21
+ layer.linear_attn=corrected
sys1mlx/vision_cache.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Byte-bounded, process-local LRU for immutable image features and prefix snapshots."""
2
+ from collections import OrderedDict
3
+ import mlx.core as mx
4
+
5
+
6
+ def retained_bytes(value):
7
+ if isinstance(value,mx.array):return value.nbytes
8
+ if isinstance(value,dict):return sum(retained_bytes(v) for v in value.values())
9
+ if isinstance(value,(list,tuple)):return sum(retained_bytes(v) for v in value)
10
+ # MLX cache.nbytes counts allocated KV capacity, not only its used slice.
11
+ return int(getattr(value,'nbytes',0))
12
+
13
+
14
+ class VisionCache:
15
+ def __init__(self,max_bytes=64*1024**2,max_entries=8):
16
+ if max_bytes<0:raise ValueError('cache budget must not be negative')
17
+ self.max_bytes=max_bytes;self.max_entries=max_entries
18
+ self.entries=OrderedDict();self.bytes=0;self.hits=0;self.misses=0;self.evictions=0
19
+
20
+ def get(self,key):
21
+ if key not in self.entries:
22
+ self.misses+=1;return None
23
+ self.hits+=1;self.entries.move_to_end(key);return self.entries[key][0]
24
+
25
+ def put(self,key,value):
26
+ size=retained_bytes(value)
27
+ if key in self.entries:self.bytes-=self.entries.pop(key)[1]
28
+ if not self.max_bytes or size>self.max_bytes:return False
29
+ while self.entries and (self.bytes+size>self.max_bytes or len(self.entries)>=self.max_entries):
30
+ _,(_,old_size)=self.entries.popitem(last=False);self.bytes-=old_size;self.evictions+=1
31
+ self.entries[key]=(value,size);self.bytes+=size;return True
32
+
33
+ def clear(self):
34
+ self.entries.clear();self.bytes=0
35
+
36
+ def info(self):
37
+ return {'bytes':self.bytes,'limit_bytes':self.max_bytes,'entries':len(self.entries),
38
+ 'hits':self.hits,'misses':self.misses,'evictions':self.evictions}
sys1mlx/vision_loader.py ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Load the published checkpoint locally, without upstream conversion/downloads."""
2
+ import hashlib
3
+ import json
4
+ from pathlib import Path
5
+ import mlx.core as mx
6
+ import mlx.nn as nn
7
+ from transformers import AutoTokenizer
8
+ from mlx_vlm.models.qwen3_5 import ModelConfig
9
+ from .vision_backbone import Sys1VisionBackbone
10
+ from .runtime import PointerHead
11
+ from .vision import VisionDecisionModel
12
+
13
+
14
+ def load_vision(model_dir=None, max_image_tokens=256, cache_mb=64):
15
+ root = Path(model_dir) if model_dir else Path(__file__).resolve().parents[1]
16
+ checkpoint = root / 'model.safetensors'
17
+ metadata = json.loads((root / 'provenance.json').read_text())
18
+ digest = hashlib.sha256()
19
+ with checkpoint.open('rb') as stream:
20
+ for block in iter(lambda: stream.read(8 * 1024**2), b''):
21
+ digest.update(block)
22
+ if digest.hexdigest() != metadata['sha256']:
23
+ raise ValueError('Checkpoint checksum mismatch')
24
+ mx.set_cache_limit(128 * 1024**2)
25
+ config = ModelConfig.from_dict(json.loads((root / 'config.json').read_text()))
26
+ model = Sys1VisionBackbone(config)
27
+ nn.quantize(model, group_size=64, bits=4, class_predicate=lambda path, m:
28
+ path.startswith('language_model.') and isinstance(m, (nn.Linear, nn.Embedding)))
29
+ model.load_weights(list(mx.load(str(checkpoint)).items()), strict=True)
30
+ head_config = json.loads((root / 'head_config.json').read_text())
31
+ head = PointerHead(config.text_config.hidden_size, head_config['head_dim'])
32
+ head.load_weights(list(mx.load(str(root / 'head.safetensors')).items()), strict=True)
33
+ tokenizer = AutoTokenizer.from_pretrained(root, trust_remote_code=False, local_files_only=True)
34
+ processor = json.loads((root / 'preprocessor_config.json').read_text())
35
+ mx.eval(model.parameters(), head.parameters())
36
+ mx.clear_cache()
37
+ return VisionDecisionModel(model, tokenizer, head, processor, max_image_tokens, cache_mb)
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927
3
+ size 12807196
tokenizer_config.json ADDED
@@ -0,0 +1,305 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "248044": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "248045": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "248046": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "248047": {
29
+ "content": "<|object_ref_start|>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "248048": {
37
+ "content": "<|object_ref_end|>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "248049": {
45
+ "content": "<|box_start|>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "248050": {
53
+ "content": "<|box_end|>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "248051": {
61
+ "content": "<|quad_start|>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "248052": {
69
+ "content": "<|quad_end|>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "248053": {
77
+ "content": "<|vision_start|>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "248054": {
85
+ "content": "<|vision_end|>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "248055": {
93
+ "content": "<|vision_pad|>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "248056": {
101
+ "content": "<|image_pad|>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "248057": {
109
+ "content": "<|video_pad|>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "248058": {
117
+ "content": "<tool_call>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": false
123
+ },
124
+ "248059": {
125
+ "content": "</tool_call>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": false
131
+ },
132
+ "248060": {
133
+ "content": "<|fim_prefix|>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": false
139
+ },
140
+ "248061": {
141
+ "content": "<|fim_middle|>",
142
+ "lstrip": false,
143
+ "normalized": false,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": false
147
+ },
148
+ "248062": {
149
+ "content": "<|fim_suffix|>",
150
+ "lstrip": false,
151
+ "normalized": false,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": false
155
+ },
156
+ "248063": {
157
+ "content": "<|fim_pad|>",
158
+ "lstrip": false,
159
+ "normalized": false,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": false
163
+ },
164
+ "248064": {
165
+ "content": "<|repo_name|>",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": false
171
+ },
172
+ "248065": {
173
+ "content": "<|file_sep|>",
174
+ "lstrip": false,
175
+ "normalized": false,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": false
179
+ },
180
+ "248066": {
181
+ "content": "<tool_response>",
182
+ "lstrip": false,
183
+ "normalized": false,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": false
187
+ },
188
+ "248067": {
189
+ "content": "</tool_response>",
190
+ "lstrip": false,
191
+ "normalized": false,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": false
195
+ },
196
+ "248068": {
197
+ "content": "<think>",
198
+ "lstrip": false,
199
+ "normalized": false,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": false
203
+ },
204
+ "248069": {
205
+ "content": "</think>",
206
+ "lstrip": false,
207
+ "normalized": false,
208
+ "rstrip": false,
209
+ "single_word": false,
210
+ "special": false
211
+ },
212
+ "248070": {
213
+ "content": "<|audio_start|>",
214
+ "lstrip": false,
215
+ "normalized": false,
216
+ "rstrip": false,
217
+ "single_word": false,
218
+ "special": true
219
+ },
220
+ "248071": {
221
+ "content": "<|audio_end|>",
222
+ "lstrip": false,
223
+ "normalized": false,
224
+ "rstrip": false,
225
+ "single_word": false,
226
+ "special": true
227
+ },
228
+ "248072": {
229
+ "content": "<tts_pad>",
230
+ "lstrip": false,
231
+ "normalized": false,
232
+ "rstrip": false,
233
+ "single_word": false,
234
+ "special": true
235
+ },
236
+ "248073": {
237
+ "content": "<tts_text_bos>",
238
+ "lstrip": false,
239
+ "normalized": false,
240
+ "rstrip": false,
241
+ "single_word": false,
242
+ "special": true
243
+ },
244
+ "248074": {
245
+ "content": "<tts_text_eod>",
246
+ "lstrip": false,
247
+ "normalized": false,
248
+ "rstrip": false,
249
+ "single_word": false,
250
+ "special": true
251
+ },
252
+ "248075": {
253
+ "content": "<tts_text_bos_single>",
254
+ "lstrip": false,
255
+ "normalized": false,
256
+ "rstrip": false,
257
+ "single_word": false,
258
+ "special": true
259
+ },
260
+ "248076": {
261
+ "content": "<|audio_pad|>",
262
+ "lstrip": false,
263
+ "normalized": false,
264
+ "rstrip": false,
265
+ "single_word": false,
266
+ "special": true
267
+ }
268
+ },
269
+ "additional_special_tokens": [
270
+ "<|im_start|>",
271
+ "<|im_end|>",
272
+ "<|object_ref_start|>",
273
+ "<|object_ref_end|>",
274
+ "<|box_start|>",
275
+ "<|box_end|>",
276
+ "<|quad_start|>",
277
+ "<|quad_end|>",
278
+ "<|vision_start|>",
279
+ "<|vision_end|>",
280
+ "<|vision_pad|>",
281
+ "<|image_pad|>",
282
+ "<|video_pad|>"
283
+ ],
284
+ "bos_token": null,
285
+ "chat_template": "{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n {%- if content is string %}\n {{- content }}\n {%- elif content is iterable and content is not mapping %}\n {%- for item in content %}\n {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain images.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set image_count.value = image_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Picture ' ~ image_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n {%- elif 'video' in item or item.type == 'video' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain videos.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set video_count.value = video_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Video ' ~ video_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n {%- elif 'text' in item %}\n {{- item.text }}\n {%- else %}\n {{- raise_exception('Unexpected item type in content.') }}\n {%- endif %}\n {%- endfor %}\n {%- elif content is none or content is undefined %}\n {{- '' }}\n {%- else %}\n {{- raise_exception('Unexpected content type.') }}\n {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- if tools and tools is iterable and tools is not mapping %}\n {{- '<|im_start|>system\\n' }}\n {{- \"# Tools\\n\\nYou have access to the following functions:\\n\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\" }}\n {{- '\\n\\nIf you choose to call a function ONLY reply in the following format with NO suffix:\\n\\n<tool_call>\\n<function=example_function_name>\\n<parameter=example_parameter_1>\\nvalue_1\\n</parameter>\\n<parameter=example_parameter_2>\\nThis is the value for the second parameter\\nthat can span\\nmultiple lines\\n</parameter>\\n</function>\\n</tool_call>\\n\\n<IMPORTANT>\\nReminder:\\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\\n- Required parameters MUST be specified\\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\\n</IMPORTANT>' }}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {%- if content %}\n {{- '\\n\\n' + content }}\n {%- endif %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {{- '<|im_start|>system\\n' + content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" %}\n {%- set content = render_content(message.content, false)|trim %}\n {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if ns.multi_step_tool %}\n {{- raise_exception('No user query found in messages.') }}\n{%- endif %}\n{%- for message in messages %}\n {%- set content = render_content(message.content, true)|trim %}\n {%- if message.role == \"system\" %}\n {%- if not loop.first %}\n {{- raise_exception('System message must be at the beginning.') }}\n {%- endif %}\n {%- elif message.role == \"user\" %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- set reasoning_content = reasoning_content|trim %}\n {%- if loop.index0 > ns.last_query_index %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content + '\\n</think>\\n\\n' + content }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {%- if loop.first %}\n {%- if content|trim %}\n {{- '\\n\\n<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n {%- else %}\n {{- '<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n {%- endif %}\n {%- else %}\n {{- '\\n<tool_call>\\n<function=' + tool_call.name + '>\\n' }}\n {%- endif %}\n {%- if tool_call.arguments is defined %}\n {%- for args_name, args_value in tool_call.arguments|items %}\n {{- '<parameter=' + args_name + '>\\n' }}\n {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}\n {{- args_value }}\n {{- '\\n</parameter>\\n' }}\n {%- endfor %}\n {%- endif %}\n {{- '</function>\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.previtem and loop.previtem.role != \"tool\" %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if not loop.last and loop.nextitem.role != \"tool\" %}\n {{- '<|im_end|>\\n' }}\n {%- elif loop.last %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- else %}\n {{- raise_exception('Unexpected message role.') }}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '<think>\\n\\n</think>\\n\\n' }}\n {%- else %}\n {{- '<think>\\n' }}\n {%- endif %}\n{%- endif %}",
286
+ "clean_up_tokenization_spaces": false,
287
+ "eos_token": "<|endoftext|>",
288
+ "errors": "replace",
289
+ "model_max_length": 262144,
290
+ "pad_token": "<|endoftext|>",
291
+ "split_special_tokens": false,
292
+ "tokenizer_class": "Qwen2Tokenizer",
293
+ "unk_token": null,
294
+ "add_bos_token": false,
295
+ "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
296
+ "extra_special_tokens": {
297
+ "audio_bos_token": "<|audio_start|>",
298
+ "audio_eos_token": "<|audio_end|>",
299
+ "audio_token": "<|audio_pad|>",
300
+ "image_token": "<|image_pad|>",
301
+ "video_token": "<|video_pad|>",
302
+ "vision_bos_token": "<|vision_start|>",
303
+ "vision_eos_token": "<|vision_end|>"
304
+ }
305
+ }
validation.json ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "description": "Packaging smoke check on M4; illustrative examples, not a quality benchmark",
3
+ "results": [
4
+ {
5
+ "mode": "text",
6
+ "seconds": 0.2356886670000904,
7
+ "result": {
8
+ "answers": {
9
+ "color": {
10
+ "type": "choice",
11
+ "choice": "red",
12
+ "confidence": 1.0,
13
+ "probabilities": {
14
+ "red": 1.0,
15
+ "blue": 0.0,
16
+ "green": 0.0
17
+ }
18
+ }
19
+ },
20
+ "probabilities": [
21
+ [
22
+ 0.9999960660934448,
23
+ 2.600952939246781e-06,
24
+ 1.2866769338870654e-06
25
+ ]
26
+ ],
27
+ "input_tokens": 29,
28
+ "vision": null
29
+ }
30
+ },
31
+ {
32
+ "mode": "vision",
33
+ "seconds": 0.7187503750000133,
34
+ "result": {
35
+ "answers": {
36
+ "color": {
37
+ "type": "choice",
38
+ "choice": "red",
39
+ "confidence": 1.0,
40
+ "probabilities": {
41
+ "red": 1.0,
42
+ "blue": 0.0,
43
+ "green": 0.0
44
+ }
45
+ }
46
+ },
47
+ "probabilities": [
48
+ [
49
+ 0.9999994039535522,
50
+ 3.513019066758716e-07,
51
+ 2.60039115573818e-07
52
+ ]
53
+ ],
54
+ "input_tokens": 176,
55
+ "vision": {
56
+ "tokens": 144,
57
+ "source_size": [
58
+ 384,
59
+ 384
60
+ ],
61
+ "processed_size": [
62
+ 384,
63
+ 384
64
+ ]
65
+ }
66
+ }
67
+ },
68
+ {
69
+ "mode": "repeat",
70
+ "seconds": 0.4666795829999728,
71
+ "result": {
72
+ "answers": {
73
+ "color": {
74
+ "type": "choice",
75
+ "choice": "red",
76
+ "confidence": 1.0,
77
+ "probabilities": {
78
+ "red": 1.0,
79
+ "blue": 0.0,
80
+ "green": 0.0
81
+ }
82
+ }
83
+ },
84
+ "probabilities": [
85
+ [
86
+ 0.9999994039535522,
87
+ 3.513019066758716e-07,
88
+ 2.60039115573818e-07
89
+ ]
90
+ ],
91
+ "input_tokens": 176,
92
+ "vision": {
93
+ "tokens": 144,
94
+ "source_size": [
95
+ 384,
96
+ 384
97
+ ],
98
+ "processed_size": [
99
+ 384,
100
+ 384
101
+ ]
102
+ }
103
+ }
104
+ }
105
+ ]
106
+ }