Lamapi commited on
Commit
8fd3398
·
verified ·
1 Parent(s): 3bae2e6

Publish Next-Shift model

Browse files
Files changed (9) hide show
  1. README.md +56 -0
  2. evaluation.json +532 -0
  3. inference.py +56 -0
  4. model.json +0 -0
  5. nlrt.py +226 -0
  6. preview.png +0 -0
  7. selftest.json +1 -0
  8. shift_core.py +61 -0
  9. source_model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ library_name: pytorch
4
+ tags:
5
+ - domain-adaptation
6
+ - few-shot
7
+ - image-classification
8
+ - next-lab
9
+ ---
10
+
11
+ # Next Shift
12
+
13
+ Next Shift adapts a digit recogniser trained on clean scans to a **new visual environment** (grey, low-contrast, textured,
14
+ rotated, blurred captures) using only a handful of labeled target images. It first re-estimates BatchNorm statistics
15
+ on unlabeled target images (AdaBN), then fine-tunes the top MLP on the labels. The target environment is a **synthetic**
16
+ transform of real UCI digits.
17
+
18
+ **Space:** https://huggingface.co/spaces/thelamapi/Next-Shift — label a few target images; the top layers fine-tune in your browser and predictions update.
19
+
20
+ ## Results: target test accuracy (297 images, 5 seeds, mean ± std)
21
+
22
+ | Method | k=0 | k=1 | k=5 | k=10 (labels per class) |
23
+ |---|---:|---:|---:|---:|
24
+ | Next Shift (AdaBN + fine-tune top) | 0.394 ± 0.000 | 0.489 ± 0.033 | 0.588 ± 0.021 | 0.661 ± 0.010 |
25
+ | Ablation: fine-tune top, no AdaBN | 0.401 ± 0.000 | 0.459 ± 0.042 | 0.589 ± 0.014 | 0.673 ± 0.009 |
26
+ | Frozen source features + linear head | 0.401 ± 0.000 | 0.358 ± 0.031 | 0.558 ± 0.019 | 0.613 ± 0.022 |
27
+ | Random init, same labels and steps | 0.099 ± 0.023 | 0.211 ± 0.022 | 0.467 ± 0.026 | 0.636 ± 0.050 |
28
+
29
+ Source-only accuracy on the source validation split: 0.973; on the target test split: 0.401.
30
+ For each k and seed, every method sees the same labeled images. Steps and learning rates were chosen per method on a
31
+ separate 100-image target development split (seeds 100–102), never on the test split.
32
+
33
+ ![Accuracy vs labels](preview.png)
34
+
35
+ ## Method
36
+ * Encoder: 2 conv blocks (16, 32 channels, BatchNorm, ReLU, max-pool) → 512 features → fc 64 → 10 classes; pre-trained on 900 source images.
37
+ * Next Shift: AdaBN on 350 unlabeled target pool images, then 50 full-batch GD steps (lr 0.05) on the top MLP.
38
+ * Frozen representation: linear head on fixed 64-d source features (200 steps, lr 0.1).
39
+ * Random init: the whole network from scratch with Adam (100 steps, lr 0.001).
40
+ * Ablation without AdaBN: same fine-tuning on source-normalised features.
41
+ * Browser: conv stacks run through `nlrt.js` and the fine-tuning is a line-by-line port; Node parity max abs difference 5.412337245047638e-16.
42
+
43
+ ## Usage
44
+ ```python
45
+ from inference import load, adapt, predict
46
+ spec = load() # NumPy only
47
+ top = adapt(spec, target_images, labels) # images: (N,16,16) in [0,1]
48
+ probs = predict(spec, new_images, top)
49
+ ```
50
+ `source_model.safetensors` holds the PyTorch source weights (architecture in the notebook).
51
+
52
+ ## Limitations
53
+ * The domain shift is synthetic and the images are tiny; real shifts (sensors, lighting, populations) can be harder.
54
+ * AdaBN needs a batch of unlabeled target images; the released `conv_target` was estimated on the target pool.
55
+ * 96 of 297 target test images are still wrong for Next Shift at k=10 (seed 0); see `evaluation.json`.
56
+ * Trained on NVIDIA L4; profile full.
evaluation.json ADDED
@@ -0,0 +1,532 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "project": "Next Shift",
3
+ "profile": "full",
4
+ "seed": 73,
5
+ "device": {
6
+ "python": "3.13.16",
7
+ "platform": "Linux-6.6.159+-x86_64-with-glibc2.39",
8
+ "cpu_count": 12,
9
+ "numpy": "2.4.6",
10
+ "torch": "2.14.1+cu130",
11
+ "cuda": true,
12
+ "gpu": "NVIDIA L4",
13
+ "colab": true
14
+ },
15
+ "data": {
16
+ "name": "Optical Recognition of Handwritten Digits",
17
+ "source": "UCI id 80 via sklearn.datasets.load_digits",
18
+ "license": "CC BY 4.0",
19
+ "sklearn_version": "1.6.1",
20
+ "target_environment": "synthetic: 16x16 upsampling, rotation ±25°, scale 0.8-1.05, grey background with weaker ink, illumination gradient, stripe texture, blur, noise",
21
+ "split": {
22
+ "source_train": 900,
23
+ "source_val": 150,
24
+ "target_pool": 350,
25
+ "target_dev": 100,
26
+ "target_test": 297,
27
+ "method": "stratified, random_state=73; source and target use disjoint digit samples"
28
+ }
29
+ },
30
+ "pretraining": {
31
+ "epochs": 40,
32
+ "best_epoch": 5,
33
+ "source_val_acc": 0.9733333333333334,
34
+ "seconds": 1.5,
35
+ "target_test_acc_source_only": 0.4006734006734007
36
+ },
37
+ "hyper": {
38
+ "ft_steps": 50,
39
+ "ft_lr": 0.05,
40
+ "lin_steps": 200,
41
+ "lin_lr": 0.1,
42
+ "rand_steps": 100,
43
+ "rand_lr": 0.001,
44
+ "noada_steps": 50,
45
+ "noada_lr": 0.05
46
+ },
47
+ "hyper_selection": {
48
+ "split": "target development (100 images)",
49
+ "dev_seeds": [
50
+ 100,
51
+ 101,
52
+ 102
53
+ ],
54
+ "next_shift": {
55
+ "50,0.05": 0.7033333333333335,
56
+ "50,0.1": 0.6933333333333334,
57
+ "50,0.3": 0.10000000000000002,
58
+ "100,0.05": 0.6933333333333334,
59
+ "100,0.1": 0.68,
60
+ "100,0.3": 0.10333333333333333,
61
+ "200,0.05": 0.68,
62
+ "200,0.1": 0.6733333333333333,
63
+ "200,0.3": 0.10666666666666667
64
+ },
65
+ "no_adabn": {
66
+ "50,0.05": 0.68,
67
+ "50,0.1": 0.6733333333333333,
68
+ "50,0.3": 0.12666666666666668,
69
+ "100,0.05": 0.6766666666666666,
70
+ "100,0.1": 0.6633333333333332,
71
+ "100,0.3": 0.10666666666666667,
72
+ "200,0.05": 0.6666666666666666,
73
+ "200,0.1": 0.6699999999999999,
74
+ "200,0.3": 0.11333333333333334
75
+ },
76
+ "frozen": {
77
+ "200,0.01": 0.6033333333333334,
78
+ "200,0.03": 0.6233333333333334,
79
+ "200,0.1": 0.63,
80
+ "500,0.01": 0.63,
81
+ "500,0.03": 0.6166666666666667,
82
+ "500,0.1": 0.6166666666666667
83
+ },
84
+ "random_init": {
85
+ "100,0.001": 0.5333333333333333,
86
+ "100,0.003": 0.5233333333333333,
87
+ "300,0.001": 0.52,
88
+ "300,0.003": 0.5166666666666667
89
+ }
90
+ },
91
+ "budget": "for each k all methods see the same labeled images; steps chosen per method on the development split",
92
+ "results": {
93
+ "next_shift": {
94
+ "k_per_class": [
95
+ 0,
96
+ 1,
97
+ 5,
98
+ 10
99
+ ],
100
+ "mean": [
101
+ 0.3939393939393939,
102
+ 0.48888888888888893,
103
+ 0.5878787878787878,
104
+ 0.6612794612794612
105
+ ],
106
+ "std": [
107
+ 0.0,
108
+ 0.03267207938237902,
109
+ 0.020929656926491656,
110
+ 0.009896928253669397
111
+ ],
112
+ "per_seed": [
113
+ [
114
+ 0.3939393939393939,
115
+ 0.46464646464646464,
116
+ 0.5656565656565656,
117
+ 0.6767676767676768
118
+ ],
119
+ [
120
+ 0.3939393939393939,
121
+ 0.468013468013468,
122
+ 0.569023569023569,
123
+ 0.6464646464646465
124
+ ],
125
+ [
126
+ 0.3939393939393939,
127
+ 0.4882154882154882,
128
+ 0.5791245791245792,
129
+ 0.6565656565656566
130
+ ],
131
+ [
132
+ 0.3939393939393939,
133
+ 0.5521885521885522,
134
+ 0.6161616161616161,
135
+ 0.6632996632996633
136
+ ],
137
+ [
138
+ 0.3939393939393939,
139
+ 0.4713804713804714,
140
+ 0.6094276094276094,
141
+ 0.6632996632996633
142
+ ]
143
+ ]
144
+ },
145
+ "no_adabn": {
146
+ "k_per_class": [
147
+ 0,
148
+ 1,
149
+ 5,
150
+ 10
151
+ ],
152
+ "mean": [
153
+ 0.4006734006734007,
154
+ 0.45925925925925926,
155
+ 0.5892255892255892,
156
+ 0.6734006734006733
157
+ ],
158
+ "std": [
159
+ 0.0,
160
+ 0.04219380524665465,
161
+ 0.01412537169252731,
162
+ 0.00903461809090825
163
+ ],
164
+ "per_seed": [
165
+ [
166
+ 0.4006734006734007,
167
+ 0.4377104377104377,
168
+ 0.5656565656565656,
169
+ 0.6632996632996633
170
+ ],
171
+ [
172
+ 0.4006734006734007,
173
+ 0.41414141414141414,
174
+ 0.5824915824915825,
175
+ 0.6632996632996633
176
+ ],
177
+ [
178
+ 0.4006734006734007,
179
+ 0.45454545454545453,
180
+ 0.5993265993265994,
181
+ 0.6835016835016835
182
+ ],
183
+ [
184
+ 0.4006734006734007,
185
+ 0.5387205387205387,
186
+ 0.5925925925925926,
187
+ 0.6734006734006734
188
+ ],
189
+ [
190
+ 0.4006734006734007,
191
+ 0.4511784511784512,
192
+ 0.6060606060606061,
193
+ 0.6835016835016835
194
+ ]
195
+ ]
196
+ },
197
+ "frozen": {
198
+ "k_per_class": [
199
+ 0,
200
+ 1,
201
+ 5,
202
+ 10
203
+ ],
204
+ "mean": [
205
+ 0.4006734006734007,
206
+ 0.35757575757575755,
207
+ 0.5575757575757576,
208
+ 0.6127946127946128
209
+ ],
210
+ "std": [
211
+ 0.0,
212
+ 0.031180705462061222,
213
+ 0.019470481205119114,
214
+ 0.02202751142598803
215
+ ],
216
+ "per_seed": [
217
+ [
218
+ 0.4006734006734007,
219
+ 0.30976430976430974,
220
+ 0.5521885521885522,
221
+ 0.6060606060606061
222
+ ],
223
+ [
224
+ 0.4006734006734007,
225
+ 0.3400673400673401,
226
+ 0.5420875420875421,
227
+ 0.6127946127946128
228
+ ],
229
+ [
230
+ 0.4006734006734007,
231
+ 0.3838383838383838,
232
+ 0.5488215488215489,
233
+ 0.5757575757575758
234
+ ],
235
+ [
236
+ 0.4006734006734007,
237
+ 0.39730639730639733,
238
+ 0.5488215488215489,
239
+ 0.6397306397306397
240
+ ],
241
+ [
242
+ 0.4006734006734007,
243
+ 0.3569023569023569,
244
+ 0.5959595959595959,
245
+ 0.6296296296296297
246
+ ]
247
+ ]
248
+ },
249
+ "random_init": {
250
+ "k_per_class": [
251
+ 0,
252
+ 1,
253
+ 5,
254
+ 10
255
+ ],
256
+ "mean": [
257
+ 0.09898989898989899,
258
+ 0.21077441077441078,
259
+ 0.4673400673400673,
260
+ 0.6363636363636365
261
+ ],
262
+ "std": [
263
+ 0.02259658146203706,
264
+ 0.021986299955979137,
265
+ 0.02558922558922559,
266
+ 0.04989530923324634
267
+ ],
268
+ "per_seed": [
269
+ [
270
+ 0.05723905723905724,
271
+ 0.1919191919191919,
272
+ 0.4478114478114478,
273
+ 0.5791245791245792
274
+ ],
275
+ [
276
+ 0.10101010101010101,
277
+ 0.24242424242424243,
278
+ 0.494949494949495,
279
+ 0.696969696969697
280
+ ],
281
+ [
282
+ 0.10101010101010101,
283
+ 0.21212121212121213,
284
+ 0.4276094276094276,
285
+ 0.5858585858585859
286
+ ],
287
+ [
288
+ 0.1111111111111111,
289
+ 0.2255892255892256,
290
+ 0.4882154882154882,
291
+ 0.6902356902356902
292
+ ],
293
+ [
294
+ 0.12457912457912458,
295
+ 0.18181818181818182,
296
+ 0.4781144781144781,
297
+ 0.6296296296296297
298
+ ]
299
+ ]
300
+ }
301
+ },
302
+ "seeds": [
303
+ 0,
304
+ 1,
305
+ 2,
306
+ 3,
307
+ 4
308
+ ],
309
+ "failure_examples": {
310
+ "setting": "Next Shift, k=10, seed 0",
311
+ "n_wrong": 96,
312
+ "first": [
313
+ {
314
+ "target_test_index": 1,
315
+ "digit_row": 278,
316
+ "true": 2,
317
+ "predicted": 0
318
+ },
319
+ {
320
+ "target_test_index": 7,
321
+ "digit_row": 1789,
322
+ "true": 8,
323
+ "predicted": 4
324
+ },
325
+ {
326
+ "target_test_index": 8,
327
+ "digit_row": 1571,
328
+ "true": 8,
329
+ "predicted": 1
330
+ },
331
+ {
332
+ "target_test_index": 10,
333
+ "digit_row": 1200,
334
+ "true": 7,
335
+ "predicted": 4
336
+ },
337
+ {
338
+ "target_test_index": 13,
339
+ "digit_row": 550,
340
+ "true": 6,
341
+ "predicted": 5
342
+ },
343
+ {
344
+ "target_test_index": 17,
345
+ "digit_row": 730,
346
+ "true": 2,
347
+ "predicted": 8
348
+ },
349
+ {
350
+ "target_test_index": 18,
351
+ "digit_row": 349,
352
+ "true": 1,
353
+ "predicted": 8
354
+ },
355
+ {
356
+ "target_test_index": 23,
357
+ "digit_row": 1227,
358
+ "true": 1,
359
+ "predicted": 8
360
+ },
361
+ {
362
+ "target_test_index": 24,
363
+ "digit_row": 1790,
364
+ "true": 8,
365
+ "predicted": 2
366
+ },
367
+ {
368
+ "target_test_index": 25,
369
+ "digit_row": 103,
370
+ "true": 3,
371
+ "predicted": 2
372
+ },
373
+ {
374
+ "target_test_index": 28,
375
+ "digit_row": 1199,
376
+ "true": 1,
377
+ "predicted": 8
378
+ },
379
+ {
380
+ "target_test_index": 30,
381
+ "digit_row": 306,
382
+ "true": 2,
383
+ "predicted": 3
384
+ },
385
+ {
386
+ "target_test_index": 32,
387
+ "digit_row": 738,
388
+ "true": 9,
389
+ "predicted": 1
390
+ },
391
+ {
392
+ "target_test_index": 34,
393
+ "digit_row": 141,
394
+ "true": 1,
395
+ "predicted": 7
396
+ },
397
+ {
398
+ "target_test_index": 35,
399
+ "digit_row": 563,
400
+ "true": 1,
401
+ "predicted": 8
402
+ },
403
+ {
404
+ "target_test_index": 38,
405
+ "digit_row": 718,
406
+ "true": 0,
407
+ "predicted": 4
408
+ },
409
+ {
410
+ "target_test_index": 49,
411
+ "digit_row": 1088,
412
+ "true": 7,
413
+ "predicted": 8
414
+ },
415
+ {
416
+ "target_test_index": 50,
417
+ "digit_row": 281,
418
+ "true": 5,
419
+ "predicted": 9
420
+ },
421
+ {
422
+ "target_test_index": 51,
423
+ "digit_row": 447,
424
+ "true": 7,
425
+ "predicted": 9
426
+ },
427
+ {
428
+ "target_test_index": 54,
429
+ "digit_row": 1120,
430
+ "true": 1,
431
+ "predicted": 8
432
+ },
433
+ {
434
+ "target_test_index": 55,
435
+ "digit_row": 579,
436
+ "true": 3,
437
+ "predicted": 9
438
+ },
439
+ {
440
+ "target_test_index": 62,
441
+ "digit_row": 409,
442
+ "true": 3,
443
+ "predicted": 2
444
+ },
445
+ {
446
+ "target_test_index": 63,
447
+ "digit_row": 808,
448
+ "true": 5,
449
+ "predicted": 3
450
+ },
451
+ {
452
+ "target_test_index": 68,
453
+ "digit_row": 765,
454
+ "true": 9,
455
+ "predicted": 7
456
+ },
457
+ {
458
+ "target_test_index": 77,
459
+ "digit_row": 1097,
460
+ "true": 1,
461
+ "predicted": 8
462
+ },
463
+ {
464
+ "target_test_index": 79,
465
+ "digit_row": 225,
466
+ "true": 4,
467
+ "predicted": 6
468
+ },
469
+ {
470
+ "target_test_index": 82,
471
+ "digit_row": 930,
472
+ "true": 5,
473
+ "predicted": 9
474
+ },
475
+ {
476
+ "target_test_index": 93,
477
+ "digit_row": 121,
478
+ "true": 4,
479
+ "predicted": 1
480
+ },
481
+ {
482
+ "target_test_index": 95,
483
+ "digit_row": 1247,
484
+ "true": 1,
485
+ "predicted": 7
486
+ },
487
+ {
488
+ "target_test_index": 97,
489
+ "digit_row": 104,
490
+ "true": 6,
491
+ "predicted": 1
492
+ }
493
+ ]
494
+ },
495
+ "tests": {
496
+ "reload": {
497
+ "returncode": 0,
498
+ "seconds": 0.25,
499
+ "process": "fresh python -E subprocess, cwd = temporary copy of the model folder",
500
+ "passed": true,
501
+ "max_abs_diff": 0.0,
502
+ "n_images": 20,
503
+ "n_labels": 10
504
+ },
505
+ "node_parity": {
506
+ "cases": 3,
507
+ "failed": 0,
508
+ "maxDiff": 5.412337245047638e-16,
509
+ "results": [
510
+ {
511
+ "name": "zero",
512
+ "maxDiff": 3.3306690738754696e-16
513
+ },
514
+ {
515
+ "name": "one-each",
516
+ "maxDiff": 5.412337245047638e-16
517
+ },
518
+ {
519
+ "name": "twenty",
520
+ "maxDiff": 4.440892098500626e-16
521
+ }
522
+ ],
523
+ "ms": 2798,
524
+ "runtime": "node v22.23.4",
525
+ "status": "passed"
526
+ },
527
+ "browser": {
528
+ "status": "skipped",
529
+ "reason": "playwright not installed"
530
+ }
531
+ }
532
+ }
inference.py ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Next Shift inference (NumPy only): adapt to a few labeled target images and predict.
2
+
3
+ Usage:
4
+ python inference.py --selftest
5
+ from inference import load, predict, adapt
6
+ """
7
+ import json
8
+ import os
9
+ import sys
10
+
11
+ HERE = os.path.dirname(os.path.abspath(__file__))
12
+ sys.path.insert(0, HERE)
13
+
14
+ import numpy as np # noqa: E402
15
+
16
+ import shift_core as sc # noqa: E402
17
+ from nlrt import load_model # noqa: E402
18
+
19
+
20
+ def load(path=os.path.join(HERE, "model.json")):
21
+ spec = load_model(path)
22
+ t = spec["tensors"]
23
+ spec["top"] = {"w1": t["w1"], "b1": t["b1"], "w2": t["w2"], "b2": t["b2"]}
24
+ return spec
25
+
26
+
27
+ def adapt(spec, images, labels):
28
+ """images: (N, 16, 16) in [0, 1] from the target environment; returns adapted top-layer weights."""
29
+ hy = spec["meta"]["hyper"]
30
+ F = sc.conv_features(spec["nets"]["conv_target"], images)
31
+ return sc.finetune_top(spec["top"], F, np.asarray(labels), hy["ft_steps"], hy["ft_lr"])
32
+
33
+
34
+ def predict(spec, images, top=None):
35
+ """Class probabilities (N, 10) for target-environment images (16x16, values in [0, 1])."""
36
+ F = sc.conv_features(spec["nets"]["conv_target"], images)
37
+ return sc.softmax(sc.mlp_logits(top or spec["top"], F)[0])
38
+
39
+
40
+ def _selftest():
41
+ spec = load()
42
+ with open(os.path.join(HERE, "selftest.json"), encoding="utf-8") as f:
43
+ t = json.load(f)
44
+ imgs = np.asarray(t["images"], dtype=float) / 255.0
45
+ top = adapt(spec, imgs[:len(t["labels"])], t["labels"])
46
+ P = predict(spec, imgs, top)
47
+ diff = float(np.abs(P - np.asarray(t["expected_proba"])).max())
48
+ ok = diff < 1e-6
49
+ print(json.dumps({"passed": ok, "max_abs_diff": diff, "n_images": len(imgs), "n_labels": len(t["labels"])}))
50
+ return ok
51
+
52
+
53
+ if __name__ == "__main__":
54
+ if "--selftest" in sys.argv:
55
+ sys.exit(0 if _selftest() else 1)
56
+ print(__doc__)
model.json ADDED
The diff for this file is too large to render. See raw diff
 
nlrt.py ADDED
@@ -0,0 +1,226 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """nlrt: tiny portable runtime shared by Next Lab models.
2
+
3
+ The same layer list runs in NumPy (this file) and in the browser (nlrt.js).
4
+ Weights are stored as base64 little-endian float32 inside model.json.
5
+ Only NumPy is required to load and run a model.
6
+ """
7
+ import base64
8
+ import json
9
+ import math
10
+
11
+ import numpy as np
12
+
13
+ __all__ = ["b64e", "b64d", "forward", "load_model", "Mulberry32", "export_sequential"]
14
+
15
+
16
+ def b64e(a):
17
+ return base64.b64encode(np.ascontiguousarray(a, dtype="<f4").tobytes()).decode("ascii")
18
+
19
+
20
+ def b64d(s, shape=None):
21
+ a = np.frombuffer(base64.b64decode(s), dtype="<f4").astype(np.float64)
22
+ return a.reshape(shape) if shape is not None else a
23
+
24
+
25
+ def load_model(path):
26
+ with open(path, "r", encoding="utf-8") as f:
27
+ spec = json.load(f)
28
+ for net in spec.get("nets", {}).values():
29
+ _decode(net)
30
+ for k, v in list(spec.get("tensors", {}).items()):
31
+ spec["tensors"][k] = b64d(v["data"], v["shape"])
32
+ return spec
33
+
34
+
35
+ def _decode(layers):
36
+ for L in layers:
37
+ for key in ("w", "b"):
38
+ if key in L and isinstance(L[key], str):
39
+ L[key] = b64d(L[key], L.get(key + "_shape"))
40
+
41
+
42
+ class Mulberry32:
43
+ """Deterministic PRNG with an identical JavaScript twin in nlrt.js."""
44
+
45
+ def __init__(self, seed):
46
+ self.a = seed & 0xFFFFFFFF
47
+
48
+ def random(self):
49
+ self.a = (self.a + 0x6D2B79F5) & 0xFFFFFFFF
50
+ t = self.a
51
+ t = ((t ^ (t >> 15)) * (t | 1)) & 0xFFFFFFFF
52
+ t ^= (t + (((t ^ (t >> 7)) * (t | 61)) & 0xFFFFFFFF)) & 0xFFFFFFFF
53
+ return ((t ^ (t >> 14)) & 0xFFFFFFFF) / 4294967296.0
54
+
55
+ def uniform(self, lo, hi):
56
+ return lo + (hi - lo) * self.random()
57
+
58
+ def randint(self, n):
59
+ return int(math.floor(self.random() * n))
60
+
61
+ def normal(self):
62
+ u = max(self.random(), 1e-12)
63
+ v = self.random()
64
+ return math.sqrt(-2.0 * math.log(u)) * math.cos(2.0 * math.pi * v)
65
+
66
+
67
+ def _pair(v):
68
+ return list(v) if isinstance(v, (list, tuple)) else [v, v]
69
+
70
+
71
+ def _conv2d(x, w, b, stride, pad):
72
+ n, c, h, wd = x.shape
73
+ o, _, kh, kw = w.shape
74
+ sh, sw = stride
75
+ ph, pw = pad
76
+ xp = np.pad(x, ((0, 0), (0, 0), (ph, ph), (pw, pw)))
77
+ oh = (h + 2 * ph - kh) // sh + 1
78
+ ow = (wd + 2 * pw - kw) // sw + 1
79
+ cols = np.empty((n, c, kh, kw, oh, ow), dtype=x.dtype)
80
+ for i in range(kh):
81
+ for j in range(kw):
82
+ cols[:, :, i, j] = xp[:, :, i:i + sh * oh:sh, j:j + sw * ow:sw]
83
+ out = np.tensordot(cols, w, axes=([1, 2, 3], [1, 2, 3])) # n, oh, ow, o
84
+ out = out.transpose(0, 3, 1, 2)
85
+ if b is not None:
86
+ out = out + b[None, :, None, None]
87
+ return out
88
+
89
+
90
+ def _maxpool(x, k):
91
+ kh, kw = k
92
+ n, c, h, w = x.shape
93
+ oh, ow = h // kh, w // kw
94
+ x = x[:, :, :oh * kh, :ow * kw].reshape(n, c, oh, kh, ow, kw)
95
+ return x.max(axis=(3, 5))
96
+
97
+
98
+ def forward(layers, x):
99
+ """Run a layer list on a batch. x: (N, D) or (N, C, H, W) or (N, C, L)."""
100
+ x = np.asarray(x, dtype=np.float64)
101
+ for L in layers:
102
+ t = L["type"]
103
+ if t == "linear":
104
+ x = x @ L["w"].T + L["b"]
105
+ elif t == "conv2d":
106
+ squeeze = x.ndim == 3
107
+ if squeeze:
108
+ x = x[:, :, None, :]
109
+ x = _conv2d(x, L["w"], L["b"], _pair(L["stride"]), _pair(L["pad"]))
110
+ if squeeze:
111
+ x = x[:, :, 0, :]
112
+ elif t == "relu":
113
+ x = np.maximum(x, 0)
114
+ elif t == "tanh":
115
+ x = np.tanh(x)
116
+ elif t == "sigmoid":
117
+ x = 1 / (1 + np.exp(-x))
118
+ elif t == "maxpool":
119
+ squeeze = x.ndim == 3
120
+ if squeeze:
121
+ x = x[:, :, None, :]
122
+ x = _maxpool(x, _pair(L["k"]))
123
+ if squeeze:
124
+ x = x[:, :, 0, :]
125
+ elif t == "gap":
126
+ x = x.reshape(x.shape[0], x.shape[1], -1).mean(axis=2)
127
+ elif t == "flatten":
128
+ x = x.reshape(x.shape[0], -1)
129
+ elif t == "upsample":
130
+ x = x.repeat(L["s"], axis=-1)
131
+ if x.ndim == 4:
132
+ x = x.repeat(L["s"], axis=-2)
133
+ elif t == "l2norm":
134
+ x = x / np.maximum(np.linalg.norm(x, axis=1, keepdims=True), 1e-12)
135
+ elif t == "reshape":
136
+ x = x.reshape([x.shape[0]] + list(L["shape"]))
137
+ else:
138
+ raise ValueError("unknown layer " + t)
139
+ return x
140
+
141
+
142
+ def export_sequential(seq):
143
+ """Convert a torch nn.Sequential into a portable layer list (BatchNorm folded)."""
144
+ import torch.nn as nn
145
+
146
+ mods = list(seq.modules())[1:] if not isinstance(seq, (list, tuple)) else list(seq)
147
+ mods = [m for m in mods if not isinstance(m, nn.Sequential)]
148
+ layers = []
149
+
150
+ def tensor(t):
151
+ return t.detach().cpu().double().numpy()
152
+
153
+ for m in mods:
154
+ if isinstance(m, nn.Linear):
155
+ b = tensor(m.bias) if m.bias is not None else np.zeros(m.out_features)
156
+ layers.append({"type": "linear", "w": tensor(m.weight), "b": b})
157
+ elif isinstance(m, (nn.Conv2d, nn.Conv1d)):
158
+ w = tensor(m.weight)
159
+ b = tensor(m.bias) if m.bias is not None else np.zeros(w.shape[0])
160
+ if isinstance(m, nn.Conv1d):
161
+ w = w[:, :, None, :]
162
+ stride, pad = [1, m.stride[0]], [0, m.padding[0]]
163
+ else:
164
+ stride, pad = list(m.stride), list(m.padding)
165
+ layers.append({"type": "conv2d", "w": w, "b": b, "stride": stride, "pad": pad})
166
+ elif isinstance(m, (nn.BatchNorm1d, nn.BatchNorm2d)):
167
+ prev = layers[-1]
168
+ assert prev["type"] in ("conv2d", "linear"), "BatchNorm must follow conv/linear"
169
+ scale = tensor(m.weight) / np.sqrt(tensor(m.running_var) + m.eps)
170
+ shift = tensor(m.bias) - tensor(m.running_mean) * scale
171
+ if prev["type"] == "conv2d":
172
+ prev["w"] = prev["w"] * scale[:, None, None, None]
173
+ else:
174
+ prev["w"] = prev["w"] * scale[:, None]
175
+ prev["b"] = prev["b"] * scale + shift
176
+ elif isinstance(m, nn.ReLU):
177
+ layers.append({"type": "relu"})
178
+ elif isinstance(m, nn.Tanh):
179
+ layers.append({"type": "tanh"})
180
+ elif isinstance(m, nn.Sigmoid):
181
+ layers.append({"type": "sigmoid"})
182
+ elif isinstance(m, nn.MaxPool2d):
183
+ layers.append({"type": "maxpool", "k": _pair(m.kernel_size)})
184
+ elif isinstance(m, nn.MaxPool1d):
185
+ k = m.kernel_size if isinstance(m, int) else m.kernel_size
186
+ layers.append({"type": "maxpool", "k": [1, k if isinstance(k, int) else k[0]]})
187
+ elif isinstance(m, (nn.AdaptiveAvgPool2d, nn.AdaptiveAvgPool1d)):
188
+ layers.append({"type": "gap"})
189
+ elif isinstance(m, nn.Flatten):
190
+ layers.append({"type": "flatten"})
191
+ elif isinstance(m, nn.Upsample):
192
+ layers.append({"type": "upsample", "s": int(m.scale_factor)})
193
+ elif isinstance(m, nn.Dropout):
194
+ continue
195
+ elif m.__class__.__name__ == "L2Norm":
196
+ layers.append({"type": "l2norm"})
197
+ elif m.__class__.__name__ == "Reshape":
198
+ layers.append({"type": "reshape", "shape": list(m.shape)})
199
+ else:
200
+ raise ValueError("cannot export " + m.__class__.__name__)
201
+ return layers
202
+
203
+
204
+ def encode_layers(layers):
205
+ out = []
206
+ for L in layers:
207
+ L2 = dict(L)
208
+ for key in ("w", "b"):
209
+ if key in L2:
210
+ arr = np.asarray(L2[key])
211
+ L2[key + "_shape"] = list(arr.shape)
212
+ L2[key] = b64e(arr)
213
+ out.append(L2)
214
+ return out
215
+
216
+
217
+ def save_model(path, nets=None, tensors=None, meta=None):
218
+ spec = {"format": "nlrt-1", "meta": meta or {}, "nets": {}, "tensors": {}}
219
+ for name, layers in (nets or {}).items():
220
+ spec["nets"][name] = encode_layers(layers)
221
+ for name, arr in (tensors or {}).items():
222
+ arr = np.asarray(arr)
223
+ spec["tensors"][name] = {"shape": list(arr.shape), "data": b64e(arr)}
224
+ with open(path, "w", encoding="utf-8") as f:
225
+ json.dump(spec, f, separators=(",", ":"))
226
+ return spec
preview.png ADDED
selftest.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"images": [[[51, 46, 33, 60, 36, 64, 74, 121, 77, 94, 97, 75, 72, 55, 50, 45], [85, 70, 72, 45, 51, 85, 103, 138, 160, 151, 121, 89, 61, 61, 46, 31], [69, 66, 74, 61, 74, 102, 147, 156, 182, 187, 178, 117, 86, 59, 59, 53], [68, 95, 101, 89, 116, 127, 156, 176, 182, 190, 165, 130, 86, 65, 44, 57], [75, 97, 86, 136, 140, 142, 132, 178, 205, 202, 178, 123, 90, 66, 57, 61], [75, 97, 119, 139, 154, 158, 156, 162, 218, 219, 175, 108, 92, 72, 65, 53], [92, 93, 115, 151, 155, 156, 158, 186, 198, 205, 156, 110, 81, 75, 65, 63], [79, 102, 132, 132, 157, 168, 188, 173, 181, 203, 155, 113, 96, 48, 71, 77], [84, 93, 118, 113, 162, 160, 175, 187, 195, 195, 153, 132, 86, 76, 67, 65], [90, 95, 104, 100, 145, 160, 168, 212, 226, 184, 162, 120, 94, 86, 66, 70], [81, 89, 86, 107, 111, 147, 174, 204, 214, 194, 170, 123, 112, 90, 62, 94], [86, 104, 92, 97, 103, 124, 162, 188, 207, 183, 186, 170, 138, 111, 89, 90], [80, 105, 109, 97, 106, 138, 169, 171, 176, 164, 171, 180, 149, 117, 110, 67], [102, 98, 86, 101, 86, 124, 159, 156, 150, 173, 185, 194, 187, 164, 114, 100], [102, 119, 101, 107, 77, 115, 140, 143, 165, 171, 196, 198, 159, 143, 86, 83], [107, 89, 102, 82, 74, 81, 85, 109, 188, 188, 179, 160, 138, 126, 82, 94]], [[45, 34, 59, 73, 119, 114, 113, 112, 79, 70, 75, 51, 42, 28, 16, 27], [29, 30, 45, 108, 121, 150, 117, 101, 103, 129, 126, 87, 43, 16, 0, 17], [34, 37, 67, 94, 130, 137, 124, 100, 85, 132, 166, 106, 64, 17, 4, 30], [34, 29, 68, 95, 145, 139, 118, 65, 110, 125, 153, 136, 82, 24, 4, 3], [35, 26, 64, 107, 126, 147, 96, 78, 78, 122, 131, 134, 102, 22, 0, 0], [4, 9, 49, 105, 123, 136, 105, 83, 87, 116, 145, 117, 85, 27, 0, 15], [20, 22, 48, 79, 105, 108, 97, 103, 85, 133, 143, 129, 68, 0, 2, 8], [24, 19, 49, 62, 104, 93, 102, 72, 83, 140, 155, 115, 38, 3, 0, 0], [21, 19, 26, 51, 54, 82, 49, 62, 76, 134, 120, 114, 47, 27, 0, 0], [23, 31, 28, 51, 36, 40, 40, 51, 73, 141, 147, 93, 17, 0, 0, 9], [21, 0, 29, 65, 50, 35, 48, 34, 69, 121, 119, 77, 47, 8, 0, 0], [28, 23, 46, 69, 69, 68, 41, 28, 70, 126, 102, 73, 17, 1, 0, 7], [10, 40, 60, 72, 103, 68, 51, 71, 84, 98, 85, 45, 27, 0, 0, 0], [16, 33, 42, 64, 117, 97, 76, 95, 82, 82, 48, 27, 12, 0, 0, 0], [0, 14, 36, 49, 68, 64, 57, 47, 56, 62, 43, 34, 0, 0, 0, 0], [0, 5, 0, 36, 31, 24, 7, 4, 0, 20, 21, 25, 15, 0, 0, 0]], [[37, 84, 97, 113, 118, 127, 142, 132, 127, 110, 104, 117, 105, 143, 132, 127], [51, 66, 81, 94, 121, 144, 134, 146, 164, 135, 147, 133, 156, 129, 109, 115], [42, 75, 70, 108, 146, 160, 178, 163, 185, 148, 146, 136, 144, 147, 136, 143], [63, 68, 98, 118, 159, 189, 180, 157, 154, 121, 136, 138, 154, 153, 145, 129], [52, 75, 93, 140, 176, 205, 170, 150, 116, 123, 115, 167, 153, 170, 175, 144], [64, 81, 134, 169, 204, 173, 147, 109, 101, 97, 129, 157, 204, 186, 156, 149], [45, 102, 122, 162, 182, 190, 136, 121, 99, 151, 152, 167, 191, 193, 161, 139], [66, 100, 147, 177, 210, 181, 149, 119, 132, 131, 168, 185, 179, 147, 138, 106], [34, 70, 130, 160, 196, 204, 163, 164, 153, 139, 154, 161, 121, 118, 109, 109], [30, 60, 106, 138, 177, 157, 173, 158, 170, 147, 155, 151, 123, 103, 100, 107], [46, 76, 55, 94, 140, 148, 160, 177, 152, 123, 124, 104, 108, 101, 93, 115], [37, 39, 79, 83, 146, 119, 146, 150, 130, 108, 74, 80, 97, 98, 113, 91], [37, 43, 47, 76, 113, 140, 144, 137, 126, 86, 76, 64, 103, 90, 98, 110], [12, 39, 48, 82, 116, 157, 153, 130, 79, 78, 54, 65, 76, 91, 96, 73], [22, 57, 50, 90, 115, 123, 122, 120, 76, 64, 31, 47, 72, 91, 96, 81], [3, 24, 53, 68, 84, 87, 76, 85, 74, 42, 75, 50, 74, 97, 87, 93]], [[78, 92, 96, 50, 45, 41, 32, 56, 40, 73, 77, 67, 63, 21, 33, 11], [86, 81, 89, 76, 42, 55, 62, 76, 83, 103, 86, 94, 53, 24, 21, 10], [100, 84, 90, 65, 61, 80, 98, 115, 121, 127, 130, 133, 106, 54, 21, 7], [74, 84, 75, 67, 68, 103, 104, 118, 139, 166, 186, 169, 128, 52, 35, 35], [61, 65, 70, 75, 76, 87, 103, 118, 135, 182, 185, 170, 131, 67, 20, 12], [72, 58, 64, 58, 67, 76, 98, 108, 139, 136, 183, 168, 111, 66, 29, 0], [66, 77, 70, 68, 56, 69, 86, 117, 138, 172, 155, 119, 89, 39, 0, 7], [45, 72, 54, 42, 59, 52, 95, 134, 157, 159, 133, 92, 78, 18, 22, 0], [70, 75, 54, 66, 73, 82, 97, 128, 137, 127, 127, 75, 55, 19, 0, 0], [52, 54, 73, 71, 100, 89, 73, 95, 110, 105, 117, 83, 44, 4, 6, 0], [62, 81, 73, 73, 90, 81, 67, 44, 76, 107, 109, 80, 21, 1, 2, 0], [54, 72, 72, 94, 86, 48, 22, 35, 60, 93, 107, 57, 27, 2, 0, 0], [61, 59, 37, 69, 76, 49, 38, 25, 64, 97, 95, 55, 23, 8, 0, 0], [45, 48, 55, 57, 60, 39, 12, 44, 67, 81, 76, 20, 21, 0, 0, 0], [31, 52, 48, 47, 23, 0, 0, 48, 65, 53, 57, 17, 0, 0, 0, 0], [35, 38, 46, 41, 8, 14, 0, 0, 21, 25, 44, 17, 21, 0, 3, 0]], [[89, 47, 29, 72, 64, 84, 45, 28, 35, 53, 57, 35, 28, 29, 61, 51], [69, 43, 49, 49, 76, 87, 90, 67, 84, 51, 33, 36, 23, 33, 47, 64], [85, 35, 53, 102, 145, 158, 141, 137, 112, 81, 59, 20, 15, 31, 43, 41], [76, 71, 76, 133, 140, 152, 125, 117, 104, 82, 55, 20, 10, 21, 47, 35], [58, 48, 82, 126, 151, 138, 127, 106, 122, 63, 45, 20, 7, 1, 48, 35], [64, 56, 59, 92, 126, 115, 96, 81, 84, 74, 54, 25, 0, 0, 30, 34], [68, 59, 47, 66, 97, 89, 104, 81, 92, 68, 52, 4, 1, 5, 30, 32], [48, 52, 42, 37, 92, 87, 128, 89, 58, 51, 41, 19, 3, 0, 26, 19], [65, 37, 24, 65, 82, 105, 99, 87, 32, 48, 35, 23, 0, 2, 31, 26], [54, 40, 10, 15, 60, 99, 80, 86, 55, 59, 38, 11, 0, 12, 25, 26], [55, 32, 26, 45, 80, 112, 123, 66, 93, 75, 71, 62, 14, 25, 18, 35], [44, 39, 24, 37, 62, 113, 112, 117, 95, 112, 105, 58, 15, 11, 37, 13], [45, 43, 7, 43, 77, 80, 114, 110, 122, 115, 136, 57, 5, 8, 15, 32], [35, 44, 19, 19, 55, 73, 87, 62, 68, 58, 48, 20, 0, 12, 13, 27], [66, 29, 6, 21, 60, 44, 17, 17, 17, 39, 27, 8, 0, 2, 11, 26], [67, 26, 0, 29, 26, 31, 12, 0, 20, 12, 32, 0, 0, 0, 0, 22]], [[67, 64, 78, 78, 114, 131, 133, 166, 164, 150, 163, 146, 140, 127, 124, 104], [55, 53, 70, 102, 158, 203, 151, 136, 129, 131, 147, 163, 153, 125, 88, 106], [40, 57, 90, 144, 159, 167, 140, 114, 121, 142, 135, 144, 127, 123, 119, 113], [65, 83, 134, 140, 174, 126, 127, 101, 122, 132, 138, 117, 96, 121, 102, 90], [52, 107, 123, 156, 134, 142, 105, 81, 147, 119, 142, 125, 118, 109, 107, 106], [50, 71, 113, 132, 125, 130, 113, 116, 122, 120, 132, 126, 89, 102, 112, 115], [66, 58, 88, 99, 120, 94, 103, 128, 161, 145, 133, 126, 108, 117, 98, 91], [28, 39, 38, 84, 108, 107, 121, 116, 134, 140, 129, 167, 149, 120, 98, 96], [42, 33, 45, 61, 80, 101, 101, 76, 103, 100, 115, 157, 148, 110, 125, 109], [51, 24, 52, 29, 63, 87, 84, 87, 83, 81, 120, 149, 184, 148, 131, 96], [35, 23, 32, 39, 74, 92, 87, 33, 38, 76, 76, 123, 174, 163, 126, 100], [44, 8, 45, 47, 67, 102, 76, 77, 50, 35, 65, 120, 152, 171, 139, 102], [17, 21, 8, 43, 52, 94, 104, 84, 82, 67, 74, 102, 160, 182, 129, 89], [30, 0, 0, 28, 75, 107, 135, 124, 116, 103, 109, 116, 158, 148, 96, 62], [15, 19, 24, 34, 59, 102, 130, 138, 103, 91, 106, 138, 116, 99, 76, 78], [17, 7, 31, 46, 57, 103, 115, 87, 64, 48, 55, 72, 57, 73, 79, 45]], [[114, 77, 49, 89, 102, 92, 89, 102, 113, 144, 123, 69, 65, 92, 119, 96], [86, 61, 67, 74, 121, 123, 111, 181, 219, 220, 192, 96, 62, 89, 101, 98], [86, 67, 79, 121, 180, 189, 189, 194, 225, 218, 188, 116, 66, 82, 80, 90], [96, 62, 91, 172, 204, 221, 186, 198, 220, 241, 159, 104, 64, 61, 102, 85], [70, 86, 105, 168, 243, 212, 168, 169, 191, 224, 164, 84, 62, 62, 98, 83], [94, 65, 103, 164, 211, 194, 145, 137, 178, 205, 151, 75, 56, 74, 99, 89], [89, 47, 90, 165, 190, 167, 135, 121, 183, 201, 140, 87, 42, 54, 86, 71], [72, 87, 98, 132, 183, 183, 123, 154, 178, 160, 106, 65, 42, 46, 93, 85], [71, 45, 87, 104, 147, 122, 112, 146, 178, 179, 97, 68, 40, 60, 93, 73], [45, 56, 37, 79, 99, 103, 100, 146, 162, 147, 76, 53, 52, 59, 92, 64], [61, 25, 44, 69, 77, 106, 115, 153, 177, 113, 89, 40, 43, 49, 72, 63], [80, 46, 40, 56, 94, 106, 91, 155, 175, 118, 82, 49, 42, 38, 62, 63], [50, 29, 51, 47, 81, 75, 93, 123, 146, 112, 75, 28, 34, 43, 59, 78], [58, 44, 27, 24, 54, 85, 75, 106, 91, 94, 64, 35, 24, 44, 63, 64], [54, 31, 13, 38, 60, 67, 50, 37, 64, 69, 35, 18, 29, 37, 75, 62], [49, 23, 10, 51, 61, 68, 20, 23, 35, 53, 57, 20, 24, 19, 39, 69]], [[65, 73, 60, 40, 20, 52, 112, 150, 149, 48, 37, 47, 37, 46, 33, 15], [74, 89, 67, 48, 36, 48, 110, 134, 127, 100, 104, 92, 82, 67, 30, 19], [85, 75, 82, 51, 32, 37, 86, 124, 112, 109, 111, 124, 93, 55, 35, 8], [59, 94, 65, 40, 44, 53, 94, 99, 104, 112, 130, 120, 100, 61, 42, 10], [76, 89, 48, 52, 43, 65, 113, 128, 130, 83, 88, 86, 73, 68, 52, 8], [70, 71, 84, 56, 71, 97, 119, 145, 126, 96, 62, 57, 55, 74, 53, 21], [89, 80, 98, 68, 86, 117, 167, 153, 127, 70, 31, 50, 46, 45, 40, 30], [103, 75, 60, 90, 100, 127, 170, 173, 145, 71, 53, 34, 47, 64, 35, 12], [97, 72, 86, 74, 87, 115, 148, 177, 142, 97, 64, 45, 48, 66, 41, 18], [87, 97, 105, 85, 58, 88, 148, 183, 167, 155, 109, 92, 66, 58, 41, 29], [80, 93, 85, 78, 54, 62, 109, 145, 159, 131, 122, 106, 84, 75, 50, 27], [78, 119, 102, 85, 75, 70, 112, 125, 154, 138, 123, 96, 101, 56, 52, 32], [93, 123, 118, 119, 82, 104, 116, 130, 134, 136, 124, 102, 102, 75, 77, 35], [93, 135, 171, 143, 130, 142, 135, 172, 155, 152, 117, 112, 77, 86, 61, 40], [92, 95, 102, 97, 153, 145, 153, 178, 154, 149, 110, 93, 83, 67, 53, 45], [109, 120, 110, 83, 75, 86, 107, 168, 159, 118, 67, 82, 63, 92, 75, 28]], [[72, 90, 122, 116, 122, 120, 127, 156, 149, 144, 153, 123, 112, 93, 89, 102], [54, 106, 125, 137, 149, 156, 180, 182, 223, 226, 206, 158, 107, 97, 107, 112], [80, 99, 132, 159, 167, 157, 156, 188, 195, 222, 229, 224, 190, 162, 148, 130], [75, 118, 162, 177, 188, 178, 164, 143, 184, 223, 222, 229, 189, 125, 132, 103], [73, 122, 164, 189, 184, 163, 138, 153, 184, 211, 215, 204, 157, 110, 107, 129], [98, 119, 155, 189, 188, 168, 147, 140, 197, 230, 207, 174, 147, 113, 96, 128], [81, 120, 164, 192, 159, 178, 157, 173, 218, 222, 186, 166, 118, 96, 101, 112], [109, 117, 168, 198, 203, 209, 190, 206, 232, 237, 190, 167, 119, 109, 117, 129], [114, 140, 180, 200, 217, 232, 211, 243, 247, 240, 204, 168, 140, 116, 114, 137], [99, 142, 185, 203, 225, 225, 226, 229, 252, 214, 228, 189, 152, 116, 125, 151], [115, 133, 192, 228, 228, 245, 223, 209, 226, 231, 225, 185, 151, 108, 129, 150], [123, 145, 163, 220, 244, 207, 186, 193, 194, 221, 205, 177, 123, 131, 127, 168], [126, 135, 176, 220, 233, 200, 162, 163, 166, 198, 169, 150, 136, 115, 130, 136], [123, 142, 157, 188, 202, 183, 145, 139, 153, 145, 171, 157, 126, 141, 122, 156], [122, 123, 137, 159, 174, 149, 121, 134, 150, 162, 157, 163, 134, 117, 129, 166], [131, 137, 154, 165, 141, 126, 117, 121, 159, 174, 171, 134, 137, 144, 146, 140]], [[55, 85, 102, 122, 142, 132, 139, 158, 113, 87, 92, 67, 65, 33, 15, 16], [43, 104, 120, 139, 161, 138, 136, 137, 127, 167, 174, 162, 112, 49, 27, 20], [69, 78, 131, 150, 137, 131, 98, 111, 119, 155, 188, 168, 114, 59, 1, 1], [62, 94, 125, 129, 117, 119, 86, 80, 110, 188, 186, 146, 102, 48, 10, 20], [80, 82, 111, 116, 111, 99, 88, 83, 136, 151, 180, 144, 98, 33, 10, 12], [59, 91, 81, 105, 83, 93, 73, 129, 144, 150, 158, 116, 73, 41, 0, 0], [59, 77, 78, 80, 91, 93, 110, 119, 124, 165, 151, 91, 62, 28, 11, 0], [57, 76, 65, 95, 74, 79, 68, 77, 137, 163, 123, 94, 62, 15, 15, 0], [50, 70, 59, 81, 94, 63, 38, 69, 113, 146, 142, 112, 67, 19, 7, 0], [45, 70, 75, 82, 93, 54, 47, 44, 80, 110, 143, 126, 77, 32, 8, 0], [60, 64, 94, 87, 67, 38, 26, 34, 81, 102, 152, 130, 78, 22, 10, 0], [47, 64, 91, 107, 78, 60, 29, 46, 83, 113, 142, 131, 83, 31, 1, 0], [36, 81, 99, 150, 107, 89, 72, 62, 107, 113, 148, 125, 76, 35, 6, 0], [64, 88, 139, 167, 166, 140, 113, 94, 116, 136, 119, 94, 64, 20, 2, 8], [25, 65, 85, 95, 85, 78, 89, 110, 108, 105, 95, 92, 28, 0, 13, 0], [52, 71, 52, 89, 50, 33, 45, 25, 48, 67, 52, 47, 41, 20, 0, 0]], [[67, 73, 79, 36, 60, 65, 91, 101, 118, 140, 108, 111, 96, 96, 72, 72], [63, 82, 50, 27, 59, 89, 113, 146, 150, 132, 103, 123, 94, 61, 51, 80], [64, 51, 56, 51, 45, 88, 123, 153, 156, 127, 125, 107, 113, 64, 75, 91], [56, 83, 41, 56, 46, 125, 155, 168, 158, 126, 147, 138, 131, 67, 60, 74], [50, 68, 54, 53, 98, 143, 188, 176, 145, 134, 156, 149, 128, 91, 68, 84], [70, 43, 48, 55, 108, 160, 178, 165, 126, 113, 153, 191, 127, 90, 74, 84], [57, 52, 64, 87, 102, 161, 179, 143, 89, 102, 165, 205, 158, 108, 85, 64], [56, 73, 74, 80, 144, 164, 177, 146, 114, 105, 172, 201, 173, 96, 59, 67], [65, 67, 58, 100, 150, 179, 175, 149, 128, 142, 177, 198, 166, 59, 76, 70], [49, 86, 79, 100, 141, 151, 187, 158, 143, 149, 188, 205, 147, 66, 52, 36], [72, 77, 76, 55, 100, 135, 156, 160, 140, 153, 196, 205, 145, 67, 56, 54], [53, 68, 34, 58, 68, 103, 149, 122, 131, 147, 185, 210, 153, 71, 42, 56], [53, 67, 30, 27, 52, 67, 99, 99, 121, 127, 170, 218, 142, 75, 51, 67], [65, 40, 46, 15, 28, 44, 64, 64, 75, 117, 162, 181, 154, 64, 46, 21], [26, 38, 31, 20, 24, 48, 65, 65, 72, 109, 125, 152, 109, 68, 32, 56], [43, 36, 47, 10, 21, 37, 80, 70, 62, 60, 66, 97, 94, 52, 38, 44]], [[25, 32, 23, 56, 71, 89, 47, 52, 31, 26, 68, 60, 78, 48, 35, 29], [38, 2, 30, 54, 101, 120, 131, 100, 86, 84, 72, 75, 68, 58, 51, 18], [26, 14, 23, 53, 119, 133, 161, 133, 101, 119, 126, 116, 67, 35, 34, 38], [28, 44, 41, 77, 106, 140, 130, 81, 109, 120, 152, 115, 114, 59, 47, 47], [25, 8, 37, 78, 104, 130, 118, 71, 102, 156, 146, 115, 95, 54, 45, 31], [43, 12, 51, 82, 105, 88, 102, 61, 115, 141, 140, 137, 97, 71, 46, 46], [25, 27, 42, 77, 101, 132, 111, 117, 133, 139, 145, 118, 106, 60, 59, 47], [64, 31, 65, 92, 135, 164, 140, 127, 123, 129, 102, 112, 76, 54, 47, 54], [47, 50, 80, 128, 144, 175, 164, 145, 132, 107, 102, 92, 90, 77, 67, 45], [61, 77, 124, 165, 191, 181, 146, 100, 91, 90, 99, 112, 86, 76, 54, 58], [75, 75, 101, 173, 196, 173, 133, 105, 91, 71, 103, 114, 82, 75, 61, 62], [47, 66, 124, 179, 219, 191, 175, 121, 80, 100, 99, 113, 97, 91, 67, 61], [75, 64, 79, 135, 169, 181, 162, 131, 122, 140, 111, 116, 90, 76, 83, 76], [62, 61, 61, 85, 109, 116, 135, 156, 148, 135, 117, 120, 96, 94, 77, 78], [68, 46, 81, 103, 92, 110, 111, 80, 104, 92, 99, 130, 111, 82, 83, 54], [93, 59, 76, 89, 114, 119, 86, 85, 69, 79, 104, 107, 102, 101, 76, 88]], [[42, 49, 66, 54, 34, 40, 74, 100, 83, 46, 19, 35, 40, 47, 12, 0], [21, 73, 61, 90, 44, 44, 82, 130, 122, 117, 73, 53, 69, 43, 27, 0], [26, 62, 57, 67, 63, 61, 103, 129, 132, 100, 75, 90, 132, 144, 109, 0], [31, 38, 76, 90, 72, 76, 76, 118, 110, 90, 62, 89, 138, 151, 84, 13], [27, 57, 58, 89, 58, 48, 77, 85, 67, 64, 36, 89, 138, 157, 73, 6], [39, 68, 57, 70, 49, 47, 60, 54, 85, 58, 68, 81, 129, 127, 75, 12], [24, 53, 68, 77, 57, 54, 58, 73, 85, 58, 62, 78, 123, 88, 24, 0], [36, 51, 68, 68, 35, 70, 82, 108, 110, 71, 99, 98, 83, 87, 25, 1], [49, 55, 78, 58, 75, 69, 126, 143, 153, 126, 114, 93, 122, 74, 21, 0], [35, 65, 76, 73, 67, 98, 139, 165, 194, 149, 104, 96, 83, 68, 0, 0], [33, 42, 68, 80, 78, 107, 154, 170, 146, 128, 105, 65, 58, 41, 15, 0], [28, 42, 69, 83, 80, 85, 138, 168, 137, 86, 34, 41, 36, 43, 8, 0], [34, 48, 89, 100, 79, 97, 131, 149, 91, 42, 16, 16, 22, 40, 13, 0], [45, 44, 74, 93, 96, 90, 91, 108, 99, 33, 0, 0, 25, 24, 7, 0], [36, 39, 80, 72, 37, 29, 68, 76, 63, 26, 0, 0, 41, 17, 11, 0], [34, 61, 59, 54, 53, 25, 39, 42, 50, 24, 0, 4, 28, 36, 6, 0]], [[95, 115, 113, 112, 85, 91, 105, 103, 128, 140, 135, 122, 94, 118, 105, 117], [104, 91, 125, 116, 117, 119, 131, 147, 165, 150, 149, 154, 111, 105, 112, 128], [86, 105, 135, 137, 114, 141, 179, 193, 203, 232, 201, 199, 141, 120, 118, 138], [94, 106, 135, 146, 159, 203, 201, 206, 206, 233, 240, 205, 158, 125, 110, 148], [80, 103, 123, 162, 176, 189, 172, 166, 171, 227, 253, 238, 167, 159, 109, 108], [93, 112, 134, 131, 171, 196, 166, 168, 171, 223, 255, 243, 192, 155, 121, 139], [87, 101, 121, 115, 141, 168, 182, 194, 213, 255, 255, 239, 183, 142, 131, 123], [82, 123, 115, 117, 110, 136, 181, 225, 241, 255, 255, 236, 182, 127, 94, 133], [104, 115, 128, 113, 93, 119, 147, 211, 248, 255, 255, 226, 146, 125, 93, 133], [106, 129, 115, 96, 79, 128, 134, 205, 214, 255, 255, 228, 160, 115, 106, 138], [100, 106, 113, 95, 112, 135, 161, 208, 236, 242, 255, 245, 170, 142, 134, 115], [102, 106, 109, 114, 103, 120, 169, 211, 220, 232, 252, 231, 193, 142, 110, 124], [107, 101, 107, 107, 100, 146, 162, 224, 203, 212, 249, 207, 181, 143, 111, 123], [105, 100, 111, 108, 118, 115, 160, 204, 192, 232, 218, 186, 176, 126, 130, 125], [100, 102, 105, 100, 90, 101, 96, 136, 206, 232, 201, 185, 125, 110, 110, 137], [100, 107, 112, 104, 94, 83, 96, 120, 145, 142, 169, 155, 109, 100, 91, 135]], [[65, 57, 22, 0, 16, 19, 50, 29, 36, 0, 0, 0, 15, 0, 4, 3], [77, 51, 30, 11, 0, 25, 56, 48, 43, 0, 0, 0, 3, 2, 21, 0], [51, 42, 26, 28, 61, 112, 129, 84, 65, 13, 0, 0, 0, 0, 32, 0], [46, 33, 30, 37, 78, 128, 135, 98, 28, 16, 0, 0, 0, 19, 17, 1], [71, 44, 38, 52, 103, 131, 125, 86, 52, 1, 0, 0, 0, 11, 0, 13], [57, 53, 49, 60, 104, 117, 101, 78, 18, 0, 0, 0, 0, 0, 16, 5], [59, 60, 44, 58, 104, 135, 95, 58, 33, 19, 16, 0, 0, 3, 2, 0], [63, 53, 55, 81, 120, 127, 107, 69, 68, 63, 42, 28, 11, 17, 0, 18], [60, 54, 62, 83, 93, 146, 157, 132, 129, 115, 98, 71, 33, 26, 18, 17], [55, 37, 45, 101, 103, 137, 151, 165, 149, 126, 87, 69, 84, 55, 44, 15], [76, 79, 49, 68, 98, 152, 158, 155, 135, 106, 53, 91, 87, 81, 58, 22], [76, 41, 33, 42, 81, 148, 166, 157, 115, 62, 54, 38, 81, 110, 53, 0], [57, 61, 20, 42, 78, 128, 172, 133, 93, 54, 43, 60, 87, 75, 38, 30], [61, 48, 41, 24, 60, 105, 131, 166, 114, 74, 64, 65, 70, 36, 33, 0], [59, 45, 0, 0, 25, 29, 99, 98, 49, 35, 0, 0, 0, 11, 11, 0], [78, 49, 10, 9, 3, 7, 21, 64, 36, 2, 0, 0, 0, 0, 0, 5]], [[46, 51, 67, 64, 86, 99, 136, 156, 126, 37, 27, 51, 25, 18, 0, 0], [46, 66, 60, 64, 106, 141, 182, 179, 166, 132, 123, 124, 56, 33, 0, 0], [51, 76, 93, 90, 139, 151, 185, 168, 135, 142, 165, 150, 137, 82, 18, 0], [49, 76, 93, 87, 111, 123, 142, 127, 110, 109, 143, 178, 160, 113, 30, 6], [25, 55, 66, 44, 74, 90, 96, 100, 72, 95, 134, 145, 167, 103, 51, 0], [64, 68, 73, 57, 62, 91, 94, 103, 106, 129, 142, 149, 135, 92, 31, 0], [63, 97, 97, 87, 101, 116, 138, 148, 137, 149, 157, 153, 111, 74, 22, 3], [70, 99, 96, 109, 110, 161, 198, 204, 168, 177, 178, 154, 110, 89, 18, 0], [94, 105, 127, 150, 158, 184, 225, 218, 209, 186, 164, 165, 134, 81, 31, 11], [80, 87, 124, 116, 180, 189, 225, 239, 199, 164, 155, 170, 148, 101, 40, 4], [83, 119, 116, 171, 154, 208, 236, 216, 177, 154, 160, 128, 147, 90, 30, 11], [83, 120, 131, 160, 149, 183, 183, 174, 125, 101, 88, 133, 131, 77, 13, 0], [74, 139, 164, 153, 200, 179, 142, 145, 103, 52, 44, 92, 84, 63, 30, 15], [78, 98, 161, 138, 171, 159, 136, 97, 68, 44, 27, 59, 73, 39, 15, 30], [92, 86, 103, 94, 139, 131, 79, 83, 71, 51, 31, 44, 69, 44, 26, 4], [92, 99, 87, 50, 44, 58, 88, 61, 63, 54, 33, 57, 78, 62, 22, 17]], [[67, 38, 63, 81, 79, 88, 109, 117, 143, 107, 87, 74, 24, 20, 38, 39], [76, 43, 72, 99, 126, 127, 157, 158, 181, 169, 167, 124, 44, 36, 32, 44], [61, 67, 91, 134, 128, 145, 181, 186, 190, 222, 211, 159, 108, 64, 26, 53], [50, 78, 109, 169, 187, 169, 160, 143, 165, 185, 225, 190, 135, 58, 47, 60], [66, 71, 130, 153, 164, 171, 131, 113, 108, 173, 172, 169, 137, 67, 60, 69], [67, 79, 148, 192, 192, 145, 114, 92, 90, 127, 136, 143, 143, 108, 63, 68], [82, 108, 148, 173, 182, 143, 82, 71, 83, 85, 143, 172, 149, 110, 44, 54], [82, 107, 176, 170, 166, 139, 91, 67, 55, 94, 139, 172, 125, 96, 61, 72], [96, 106, 146, 191, 195, 126, 83, 64, 71, 108, 138, 164, 114, 87, 54, 87], [79, 96, 156, 218, 208, 141, 70, 63, 69, 102, 134, 135, 137, 83, 69, 77], [98, 132, 174, 216, 199, 137, 65, 68, 71, 115, 137, 152, 144, 102, 67, 50], [92, 121, 163, 207, 190, 149, 93, 83, 94, 144, 175, 155, 135, 80, 60, 78], [100, 128, 174, 202, 215, 186, 127, 107, 139, 129, 159, 134, 115, 78, 63, 71], [77, 82, 150, 206, 237, 206, 162, 145, 178, 179, 190, 136, 114, 75, 62, 84], [80, 94, 121, 130, 232, 208, 180, 174, 196, 194, 181, 137, 95, 53, 51, 74], [84, 90, 104, 127, 144, 134, 111, 169, 200, 200, 157, 113, 94, 52, 48, 92]], [[87, 98, 68, 66, 51, 73, 113, 111, 57, 58, 75, 82, 68, 69, 54, 31], [110, 95, 80, 71, 68, 132, 174, 182, 111, 68, 73, 90, 83, 76, 45, 57], [102, 100, 78, 66, 96, 154, 224, 218, 165, 107, 80, 111, 104, 76, 58, 38], [117, 89, 91, 58, 87, 170, 204, 229, 186, 144, 145, 167, 152, 106, 67, 58], [101, 94, 100, 71, 109, 147, 204, 247, 197, 155, 186, 175, 190, 116, 69, 57], [87, 110, 91, 82, 103, 170, 217, 243, 241, 192, 180, 156, 140, 106, 63, 62], [137, 101, 97, 101, 126, 190, 232, 239, 236, 171, 129, 132, 120, 76, 72, 62], [108, 111, 127, 119, 153, 192, 243, 255, 221, 130, 91, 114, 108, 95, 58, 65], [97, 111, 118, 145, 198, 244, 245, 255, 191, 125, 113, 85, 93, 80, 69, 78], [131, 124, 152, 175, 230, 235, 255, 239, 198, 109, 103, 94, 110, 83, 62, 73], [132, 145, 169, 207, 241, 255, 255, 255, 179, 129, 93, 138, 112, 96, 89, 72], [118, 123, 163, 215, 231, 255, 255, 238, 162, 118, 112, 114, 119, 104, 75, 62], [125, 142, 128, 131, 180, 230, 244, 220, 148, 115, 99, 108, 101, 89, 75, 81], [135, 136, 125, 94, 122, 148, 176, 170, 132, 98, 89, 117, 108, 86, 90, 89], [143, 134, 124, 107, 95, 97, 160, 148, 122, 94, 95, 125, 130, 94, 83, 92], [153, 148, 122, 91, 122, 121, 144, 130, 93, 116, 113, 139, 122, 110, 105, 86]], [[59, 69, 85, 43, 13, 33, 36, 41, 33, 53, 34, 12, 15, 0, 14, 15], [42, 73, 44, 38, 40, 43, 54, 88, 99, 75, 55, 41, 5, 0, 7, 0], [43, 71, 66, 50, 38, 88, 87, 99, 140, 123, 136, 99, 30, 15, 0, 0], [54, 54, 65, 35, 61, 69, 113, 101, 117, 141, 122, 119, 61, 0, 0, 0], [70, 51, 76, 84, 75, 93, 113, 115, 131, 137, 127, 103, 63, 7, 3, 3], [52, 52, 77, 47, 103, 95, 119, 127, 148, 131, 110, 68, 47, 0, 0, 0], [57, 62, 76, 80, 81, 117, 95, 110, 106, 103, 89, 44, 1, 9, 0, 0], [50, 49, 64, 84, 113, 99, 100, 60, 61, 78, 52, 18, 1, 0, 0, 0], [18, 59, 87, 94, 73, 62, 69, 64, 49, 42, 18, 14, 0, 0, 0, 0], [22, 53, 57, 66, 68, 63, 69, 67, 38, 29, 21, 6, 0, 0, 0, 0], [42, 62, 56, 60, 65, 69, 64, 51, 45, 24, 11, 15, 0, 0, 0, 0], [34, 39, 72, 86, 67, 72, 66, 40, 27, 11, 7, 0, 0, 0, 0, 0], [27, 48, 51, 76, 85, 83, 54, 40, 29, 27, 27, 0, 0, 0, 0, 0], [19, 39, 43, 48, 53, 46, 55, 24, 10, 10, 13, 0, 0, 0, 0, 0], [8, 32, 40, 0, 2, 0, 0, 0, 7, 14, 9, 0, 0, 0, 0, 0], [30, 30, 22, 9, 0, 0, 0, 0, 0, 36, 17, 0, 0, 0, 0, 0]], [[24, 31, 74, 80, 69, 94, 152, 157, 125, 99, 37, 18, 15, 4, 30, 43], [15, 18, 66, 111, 151, 151, 149, 146, 147, 127, 65, 19, 0, 18, 37, 40], [25, 53, 115, 174, 145, 134, 101, 128, 167, 118, 58, 8, 0, 8, 35, 31], [39, 68, 167, 164, 136, 99, 81, 109, 133, 149, 88, 18, 0, 3, 31, 17], [22, 89, 143, 153, 113, 75, 64, 100, 144, 134, 66, 29, 0, 2, 10, 50], [9, 77, 125, 135, 99, 48, 44, 76, 132, 136, 97, 18, 0, 0, 18, 12], [6, 42, 85, 94, 69, 24, 33, 85, 116, 146, 78, 27, 0, 0, 5, 19], [9, 17, 56, 40, 33, 24, 17, 68, 105, 108, 91, 32, 0, 0, 22, 18], [5, 21, 26, 54, 25, 10, 0, 41, 108, 134, 98, 31, 0, 9, 16, 23], [0, 1, 31, 17, 9, 6, 0, 59, 119, 142, 126, 61, 22, 26, 41, 9], [0, 3, 5, 18, 28, 3, 36, 66, 133, 180, 162, 87, 77, 60, 71, 11], [0, 0, 11, 44, 23, 6, 14, 92, 123, 164, 164, 125, 90, 92, 92, 21], [0, 14, 15, 25, 22, 10, 50, 98, 160, 163, 129, 84, 69, 31, 29, 13], [0, 0, 16, 15, 4, 19, 67, 119, 117, 112, 96, 32, 0, 0, 0, 6], [6, 0, 12, 10, 6, 6, 22, 32, 50, 55, 18, 0, 0, 0, 0, 0], [0, 0, 5, 38, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 2, 4]]], "labels": [8, 9, 4, 8, 2, 3, 7, 3, 7, 3], "expected_proba": [[0.000199858343590407, 0.0025937324712489063, 0.00249053666243136, 0.003133736133941547, 0.004658180720760722, 0.0001118649390156524, 0.0017742176194766764, 0.02625837004498942, 0.9583306861402062, 0.00044881692433896485], [0.00019596039333031956, 0.00010315466001393098, 0.007205955813889309, 0.0031801360954188777, 0.0014262119956497917, 0.0002624753572712951, 4.468666557997629e-05, 0.0011462878898001896, 0.015880299674182457, 0.9705548314548639], [0.0004076585880064497, 1.5109550448663621e-05, 8.617752534370285e-05, 8.678566380254737e-05, 0.975282399733907, 3.6386526633054616e-05, 0.00011590036365275577, 0.022580261134327467, 0.0009299036143904917, 0.00045941729948787937], [1.6234746877351084e-05, 0.0001434113296802956, 0.0016005176063313643, 0.00337629420330876, 0.0016466607709959959, 3.347143196403277e-05, 1.2652843113735616e-05, 0.009080548862002172, 0.9656117372538289, 0.018478470951897513], [0.00011787794145684881, 9.538458347155787e-05, 0.9820373603095558, 0.008005952627212376, 0.0015846687289891778, 0.0015342955520940707, 0.0011592899385115029, 0.0022681368986250086, 0.0013520932681333748, 0.0018449401519501838], [0.00017815653501663132, 8.53779728681778e-07, 0.0005201691629638638, 0.9946841717239695, 2.120405310679398e-05, 0.00018816360183790618, 5.432448356434124e-05, 0.0006838877881598372, 0.0001762193705127923, 0.0034928495011396507], [0.00010249506588797916, 0.0002825992767091235, 0.0024473443911096, 0.0007666169557995282, 0.008265613016106739, 0.00025987085950772307, 5.945773559950363e-05, 0.9684906128100735, 0.016782269507667515, 0.0025431203815385875], [7.838690855372125e-05, 9.935324799812672e-06, 0.0030154323491031615, 0.9823514453788592, 0.00022502559730488687, 0.0003868080403557804, 0.000265958004941723, 0.00056721536987715, 0.01303469753982198, 6.509548638239406e-05], [4.3838630688415294e-05, 4.301652803908759e-05, 0.0006450307868121256, 0.004518269078800073, 0.004388401318904602, 3.0013968448354617e-05, 0.000168625661346417, 0.9696197462348285, 0.020473743256857996, 6.931453527431074e-05], [8.846229245155009e-05, 8.170647367655181e-07, 0.0011611315579790455, 0.9962973499527018, 2.8367691703951215e-05, 0.0001281568624040129, 8.176373425618584e-06, 0.00020688911485474906, 0.0007651164879319017, 0.001315532601810685], [0.0009804386977537295, 0.0004925700909161146, 0.0007451553828177887, 0.008770836643042826, 0.7227699766073593, 0.00015453390613706056, 0.0016465737980411183, 0.10169814954293584, 0.15632409091614105, 0.0064176744148549585], [0.017998264625872243, 0.0005067063782794142, 0.04800013967336957, 0.20364295357199638, 0.20116495473867327, 0.0012639148760420897, 0.016229014913397892, 0.02033419654877137, 0.48550763356403936, 0.005352221109558357], [0.0006150832781938534, 2.4625661783556417e-05, 0.05081564931252808, 0.014554099596802601, 0.37301871155802124, 8.261699038463103e-05, 0.0003322688553528101, 0.5409943174553238, 0.015829131513834315, 0.0037334957777751883], [6.492644074263181e-05, 0.0002728729162842765, 0.0014175935952209656, 0.005629199220332081, 0.005038612148671329, 4.286989621629805e-05, 9.293345479586771e-05, 0.7731132191170166, 0.21380859819959733, 0.0005191750111224317], [0.0003942840987418695, 0.00021440301395948195, 0.3906102066927149, 0.04198030484898089, 0.13402791132983313, 0.00869694921710242, 0.4151563344302613, 0.001883146287404625, 0.00693273226103402, 0.00010372781996718573], [7.35842667141773e-05, 3.245298846203555e-06, 0.0017649406178985778, 0.009880805430588802, 0.2588285412256803, 1.2246865925137378e-05, 0.0001409524292774593, 0.7038021656372087, 0.025220393294741093, 0.00027312493311947705], [0.5177130254827351, 1.7228294574401535e-05, 0.010540556693360624, 0.2722624718506651, 0.17619545565784966, 0.0006595820337187163, 0.003508514042375985, 0.0023281383815671537, 0.009445840320311276, 0.0073291872428418664], [0.0001363154095622963, 0.00047405324252936936, 0.003112504085741223, 0.0005944490109853677, 0.141888025644956, 4.2445512889117954e-05, 0.0010522014913534664, 0.05590122990649151, 0.7966393227774924, 0.0001594529179992241], [0.00022850387797873566, 0.005353325031545942, 0.3778867512088609, 0.004506740541829967, 0.013129088077799687, 0.001288762215436765, 0.0023750986634342086, 0.009274587624082545, 0.515053948567251, 0.07090319419178032], [2.0414258406548077e-05, 1.7821796898847852e-06, 0.32939907665452145, 0.6575464827828146, 4.529901268463037e-05, 0.0003750468248329661, 1.6166998964534297e-05, 0.0006420916074419071, 0.0001101960843590536, 0.01184344359628439]]}
shift_core.py ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Next Shift: few-shot adaptation of a digit recogniser to a new visual environment.
2
+
3
+ Pieces shared by the notebook, inference.py and (ported line by line) web/core.js:
4
+ * conv features: nlrt layer lists exported from PyTorch (BatchNorm folded); `conv_target` uses BatchNorm
5
+ statistics re-estimated on *unlabeled* target images (AdaBN), `conv_source` the source statistics.
6
+ * finetune_top: full-batch gradient descent on the top MLP (512 -> 64 -> 10) from the pretrained weights.
7
+ * train_linear: logistic-regression probe from zero weights on frozen 64-d source features.
8
+ All arithmetic is float64 and deterministic.
9
+ """
10
+ import numpy as np
11
+
12
+ from nlrt import forward
13
+
14
+
15
+ def conv_features(layers, images):
16
+ """images: (N, 16, 16) in [0, 1] -> (N, 512)."""
17
+ return forward(layers, np.asarray(images, dtype=np.float64)[:, None, :, :])
18
+
19
+
20
+ def mlp_logits(p, F):
21
+ h = np.maximum(F @ p["w1"].T + p["b1"], 0)
22
+ return h @ p["w2"].T + p["b2"], h
23
+
24
+
25
+ def softmax(z):
26
+ z = z - z.max(axis=1, keepdims=True)
27
+ e = np.exp(z)
28
+ return e / e.sum(axis=1, keepdims=True)
29
+
30
+
31
+ def finetune_top(p0, F, y, steps, lr):
32
+ p = {k: np.array(v, dtype=np.float64) for k, v in p0.items()}
33
+ n = len(y)
34
+ for _ in range(steps):
35
+ z, h = mlp_logits(p, F)
36
+ P = softmax(z)
37
+ P[np.arange(n), y] -= 1.0
38
+ P /= n
39
+ gw2 = P.T @ h
40
+ gb2 = P.sum(axis=0)
41
+ dh = (P @ p["w2"]) * (h > 0)
42
+ gw1 = dh.T @ F
43
+ gb1 = dh.sum(axis=0)
44
+ p["w1"] -= lr * gw1
45
+ p["b1"] -= lr * gb1
46
+ p["w2"] -= lr * gw2
47
+ p["b2"] -= lr * gb2
48
+ return p
49
+
50
+
51
+ def train_linear(Z, y, steps, lr, n_classes=10):
52
+ n, d = Z.shape
53
+ W = np.zeros((n_classes, d))
54
+ b = np.zeros(n_classes)
55
+ for _ in range(steps):
56
+ P = softmax(Z @ W.T + b)
57
+ P[np.arange(n), y] -= 1.0
58
+ P /= n
59
+ W -= lr * (P.T @ Z)
60
+ b -= lr * P.sum(axis=0)
61
+ return W, b
source_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c04d374b4d38c89f7b0b570ca5172dcdc0cd651a9be9fb3374236e2aaa5392d
3
+ size 155256