yasserrmd commited on
Commit
1a2bb00
·
verified ·
1 Parent(s): 574edb1

WSP 2.0 sparse weight edit: strict-JSON fence repair (32,768 BF16 values, 16 w2 columns)

Browse files
LICENSE ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ LFM Open License v1.0
2
+
3
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
4
+
5
+ 1. Definitions.
6
+
7
+ "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by this document.
8
+
9
+ "Licensor" shall mean Liquid AI, Inc.
10
+
11
+ "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
12
+
13
+ "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
14
+
15
+ "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
16
+
17
+ "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
18
+
19
+ "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work.
20
+
21
+ "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
22
+
23
+ "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
24
+
25
+ "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
26
+
27
+ "Commercial Use" shall mean any use of the Work for direct or indirect commercial advantage or monetary compensation.
28
+
29
+ "Qualified Non-Profit Organization" shall mean a Legal Entity that is organized and operated exclusively for religious, charitable, scientific, testing for public safety, literary, or educational purposes, and which is exempt from federal income tax under Section 501(c)(3) of the United States Internal Revenue Code of 1986, as amended, or any equivalent non-profit or charitable organization in a foreign jurisdiction.
30
+
31
+ "Non-Commercial or Research Purposes" shall mean purposes that do not involve any use of the Work or a Derivative Work for Commercial Use.
32
+
33
+ "Threshold" shall mean annual revenue of 10 million United States dollars ($10,000,000) or more.
34
+
35
+ 2. Grant of Copyright License. Subject to the terms and conditions of this License, including the Commercial Use limitation set forth in Section 5, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
36
+
37
+ 3. Grant of Patent License. Subject to the terms and conditions of this License, including the Commercial Use limitation set forth in Section 5, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
38
+
39
+ 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
40
+
41
+ (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
42
+
43
+ (b) You must cause any modified files to carry prominent notices stating that You changed the files; and
44
+
45
+ (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
46
+
47
+ (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
48
+
49
+ You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
50
+
51
+ 5. Commercial Use Limitation.
52
+
53
+ (a) The rights granted under this License for Commercial Use are conditioned upon You or Your Legal Entity not exceeding the Threshold.
54
+
55
+ (b) Any Commercial Use of the Work or a Derivative Work by a Legal Entity that exceeds the Threshold is not licensed under this Agreement.
56
+
57
+ (c) The Threshold shall not apply to a Qualified Non-Profit Organization's use of the Work or a Derivative Work for Non-Commercial or Research Purposes.
58
+
59
+ 6. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
60
+
61
+ 7. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except for the reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
62
+
63
+ 8. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
64
+
65
+ 9. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
66
+
67
+ 10. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
68
+
69
+ 11. Termination. This License will terminate automatically and immediately if You fail to comply with any of its terms and conditions. Upon termination, You must cease all use of the Work and any Derivative Works and delete all copies in Your possession.
70
+
71
+ END OF TERMS AND CONDITIONS
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: lfm1.0
4
+ license_link: LICENSE
5
+ base_model: LiquidAI/LFM2.5-1.2B-Instruct
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ language: [en, ar, zh, fr, de, ja, ko, es]
9
+ tags: [liquid, lfm2.5, weight-editing, model-editing, json, training-free]
10
+ ---
11
+
12
+ # LFM2.5-1.2B-Instruct-WSP-JSONFix
13
+
14
+ `LiquidAI/LFM2.5-1.2B-Instruct` (revision `b9d816ab510f9a28dca37cb182e9da8b0226e13c`) with a **training-free, sparse, reversible weight edit** that stops the model from wrapping strict-JSON answers in markdown code fences. No gradient descent, fine-tuning, LoRA or dataset updates were used: 16 columns of the FFN down-projection (`w2`) in layers 12 to 15 were scaled by exactly 0.5, changing 32,768 stored BF16 values (0.0028% of the model).
15
+
16
+ This is a behavior repair, not a new capability.
17
+
18
+ ## Held-out test results (evaluated once, after selection)
19
+ | | Base | This model |
20
+ |---|---|---|
21
+ | Strict-JSON exact match (greedy + 3 sampled seeds, 256 units) | 167/256 | 196/256 (29 fixed, 0 broken) |
22
+ | Strict-JSON exact match, greedy (64 prompts) | 42/64 | 49/64 |
23
+ | Replies wrapped in code fences | 41/256 | 0/256 |
24
+ | Top-level JSON arrays exact | 76/128 | 97/128 (0 broken) |
25
+ | JSON objects exact | 91/128 | 99/128 |
26
+ | Prompts that request a code block (fence kept) | 48/48 | 48/48 |
27
+ | Short exact answers / general sanity | 57/64, 64/64 | 57/64, 64/64 |
28
+ | Termination tasks | 36/64 | 35/64 |
29
+ | WikiText-2 perplexity | 130.547 | -0.073% relative |
30
+
31
+ Matched controls (same size, random columns or random directions) stayed at base level (166 to 167/256). Benchmark: regenerated, template-based, 64 strict-JSON test prompts (32 objects, 32 arrays), small sample; see `wsp/WSP_1_2B_Experiment_Report.md` for methods, all 12 candidates, gates and limitations. Most fences in the base model came from weakly worded instructions such as "Answer in JSON."
32
+
33
+ ## Edit
34
+ | tensor | columns (scaled x0.5) |
35
+ |---|---|
36
+ | model.layers.12.feed_forward.w2.weight | 4731, 7264, 7919 |
37
+ | model.layers.13.feed_forward.w2.weight | 488, 5186, 6715 |
38
+ | model.layers.14.feed_forward.w2.weight | 3659 |
39
+ | model.layers.15.feed_forward.w2.weight | 99, 532, 660, 926, 2017, 2439, 3263, 3682, 5191 |
40
+
41
+ `wsp/` contains the sparse patch (`chosen_patch.safetensors`, coordinates plus original and edited values), its manifest, and `apply_patch.py` / `rebuild_patch.py` to apply, revert or regenerate it bitwise from the base model.
42
+
43
+ ## Usage
44
+ Same as the base model (chat template, tokenizer and generation settings unchanged):
45
+ ```python
46
+ from transformers import AutoModelForCausalLM, AutoTokenizer
47
+ tok = AutoTokenizer.from_pretrained("yasserrmd/LFM2.5-1.2B-Instruct-WSP-JSONFix")
48
+ model = AutoModelForCausalLM.from_pretrained("yasserrmd/LFM2.5-1.2B-Instruct-WSP-JSONFix", dtype="bfloat16")
49
+ ```
50
+
51
+ ## License and modification notice
52
+ Derivative of LiquidAI/LFM2.5-1.2B-Instruct by Liquid AI, distributed under the LFM Open License v1.0 (`LICENSE`, unchanged). Modified files: `model.safetensors` (the 32,768 values listed above). All other model files are copied from the base revision.
chat_template.jinja ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{- bos_token -}}
2
+ {%- set preserve_thinking = preserve_thinking | default(keep_past_thinking | default(false)) -%}
3
+
4
+ {%- macro format_arg_value(arg_value) -%}
5
+ {%- if arg_value is string -%}
6
+ {{- "'" + (arg_value | replace("\\", "\\\\") | replace("'", "\\'") | replace("\n", "\\n") | replace("\r", "\\r")) + "'" -}}
7
+ {%- elif arg_value is mapping or arg_value is iterable -%}
8
+ {{- arg_value | tojson -}}
9
+ {%- else -%}
10
+ {{- arg_value | string -}}
11
+ {%- endif -%}
12
+ {%- endmacro -%}
13
+
14
+ {%- macro parse_content(content) -%}
15
+ {%- if content is string -%}
16
+ {{- content -}}
17
+ {%- elif content is mapping -%}
18
+ {{- content | tojson -}}
19
+ {%- elif content is iterable -%}
20
+ {%- set _ns = namespace(result="") -%}
21
+ {%- for item in content -%}
22
+ {%- if item is string -%}
23
+ {%- set _ns.result = _ns.result + item -%}
24
+ {%- elif item is mapping and item.get("type") == "image" -%}
25
+ {%- set _ns.result = _ns.result + "<image>" -%}
26
+ {%- elif item is mapping and item.get("type") == "text" -%}
27
+ {%- set _ns.result = _ns.result + ((item.get("text") or "") | string) -%}
28
+ {%- else -%}
29
+ {%- set _ns.result = _ns.result + (item | tojson) -%}
30
+ {%- endif -%}
31
+ {%- endfor -%}
32
+ {{- _ns.result -}}
33
+ {%- endif -%}
34
+ {%- endmacro -%}
35
+
36
+ {%- macro render_tool_calls(tool_calls) -%}
37
+ {%- set tool_calls_ns = namespace(tool_calls=[]) -%}
38
+ {%- for tool_call in tool_calls -%}
39
+ {%- set func = tool_call["function"] if "function" in tool_call else tool_call -%}
40
+ {%- set func_name = func["name"] -%}
41
+ {%- set func_args = func.get("arguments") -%}
42
+ {%- set args_ns = namespace(arg_strings=[]) -%}
43
+ {%- if func_args is mapping -%}
44
+ {%- for arg_name, arg_value in func_args.items() -%}
45
+ {%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
46
+ {%- endfor -%}
47
+ {%- elif func_args is string and (func_args | trim) not in ["", "{}", "null"] -%}
48
+ {{- raise_exception("Tool call arguments must be a mapping, got a JSON-encoded string: parse arguments with json.loads() before applying the chat template") -}}
49
+ {%- endif -%}
50
+ {%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
51
+ {%- endfor -%}
52
+ {{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
53
+ {%- endmacro -%}
54
+
55
+ {%- set ns = namespace(system_prompt="", last_assistant_index=-1) -%}
56
+ {%- if messages and messages[0]["role"] == "system" -%}
57
+ {%- if messages[0].get("content") -%}
58
+ {%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
59
+ {%- endif -%}
60
+ {%- set messages = messages[1:] -%}
61
+ {%- endif -%}
62
+ {%- if tools -%}
63
+ {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
64
+ {%- for tool in tools -%}
65
+ {%- if tool is not string -%}
66
+ {%- set tool = tool | tojson -%}
67
+ {%- endif -%}
68
+ {%- set ns.system_prompt = ns.system_prompt + tool -%}
69
+ {%- if not loop.last -%}
70
+ {%- set ns.system_prompt = ns.system_prompt + ", " -%}
71
+ {%- endif -%}
72
+ {%- endfor -%}
73
+ {%- set ns.system_prompt = ns.system_prompt + "]" -%}
74
+ {%- endif -%}
75
+ {%- if ns.system_prompt -%}
76
+ {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
77
+ {%- endif -%}
78
+ {%- for message in messages -%}
79
+ {%- if message["role"] == "assistant" -%}
80
+ {%- set ns.last_assistant_index = loop.index0 -%}
81
+ {%- endif -%}
82
+ {%- endfor -%}
83
+ {%- for message in messages -%}
84
+ {{- "<|im_start|>" + message.role + "\n" -}}
85
+ {%- if message.role == "assistant" -%}
86
+ {%- generation -%}
87
+ {%- set keep_thinking = preserve_thinking or loop.index0 == ns.last_assistant_index -%}
88
+ {%- set thinking = message.thinking or message.reasoning or message.reasoning_content -%}
89
+ {%- set thinking = thinking if thinking is string else "" -%}
90
+ {%- if thinking and keep_thinking -%}
91
+ {{- "<think>" + thinking + "</think>" -}}
92
+ {%- endif -%}
93
+ {%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
94
+ {%- set _has_cfm = false -%}
95
+ {%- set content = "" -%}
96
+ {%- if message.get("content") -%}
97
+ {%- set content = parse_content(message.content) -%}
98
+ {%- endif -%}
99
+ {%- if not keep_thinking and "</think>" in content -%}
100
+ {%- set content = content.split("</think>")[-1] | trim -%}
101
+ {%- endif -%}
102
+ {%- if content.endswith(_cfm_tag) -%}
103
+ {%- set _has_cfm = true -%}
104
+ {%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
105
+ {%- set content = content[:_trunc_len] -%}
106
+ {%- endif -%}
107
+ {{- content -}}
108
+ {%- if message.tool_calls -%}
109
+ {{- render_tool_calls(message.tool_calls) -}}
110
+ {%- endif -%}
111
+ {%- if _has_cfm -%}
112
+ {{- _cfm_tag -}}
113
+ {%- endif -%}
114
+ {{- "<|im_end|>\n" -}}
115
+ {%- endgeneration -%}
116
+ {%- else %}
117
+ {%- if message.get("content") -%}
118
+ {{- parse_content(message["content"]) -}}
119
+ {%- endif -%}
120
+ {{- "<|im_end|>\n" -}}
121
+ {%- endif %}
122
+ {%- endfor -%}
123
+ {%- if add_generation_prompt -%}
124
+ {{- "<|im_start|>assistant\n" -}}
125
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Lfm2ForCausalLM"
4
+ ],
5
+ "block_auto_adjust_ff_dim": true,
6
+ "block_dim": 2048,
7
+ "block_ffn_dim_multiplier": 1.0,
8
+ "block_mlp_init_scale": 1.0,
9
+ "block_multiple_of": 256,
10
+ "block_norm_eps": 1e-05,
11
+ "block_out_init_scale": 1.0,
12
+ "block_use_swiglu": true,
13
+ "block_use_xavier_init": true,
14
+ "bos_token_id": 1,
15
+ "conv_L_cache": 3,
16
+ "conv_bias": false,
17
+ "conv_dim": 2048,
18
+ "conv_use_xavier_init": true,
19
+ "dtype": "bfloat16",
20
+ "eos_token_id": 7,
21
+ "full_attn_idxs": null,
22
+ "hidden_size": 2048,
23
+ "initializer_range": 0.02,
24
+ "intermediate_size": 12288,
25
+ "layer_types": [
26
+ "conv",
27
+ "conv",
28
+ "full_attention",
29
+ "conv",
30
+ "conv",
31
+ "full_attention",
32
+ "conv",
33
+ "conv",
34
+ "full_attention",
35
+ "conv",
36
+ "full_attention",
37
+ "conv",
38
+ "full_attention",
39
+ "conv",
40
+ "full_attention",
41
+ "conv"
42
+ ],
43
+ "max_position_embeddings": 128000,
44
+ "model_type": "lfm2",
45
+ "norm_eps": 1e-05,
46
+ "num_attention_heads": 32,
47
+ "num_heads": 32,
48
+ "num_hidden_layers": 16,
49
+ "num_key_value_heads": 8,
50
+ "pad_token_id": 0,
51
+ "rope_parameters": {
52
+ "rope_theta": 1000000.0,
53
+ "rope_type": "default"
54
+ },
55
+ "tie_word_embeddings": true,
56
+ "transformers_version": "5.22.1",
57
+ "use_cache": true,
58
+ "use_pos_enc": true,
59
+ "vocab_size": 65536
60
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 7,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.57.2"
7
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b44199d65bbca3838ca962c92af6d4899bb2ff5e62e89c270f3acc983941b0ec
3
+ size 2340697936
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<|startoftext|>",
4
+ "clean_up_tokenization_spaces": false,
5
+ "eos_token": "<|im_end|>",
6
+ "is_local": true,
7
+ "legacy": false,
8
+ "local_files_only": false,
9
+ "model_input_names": [
10
+ "input_ids",
11
+ "attention_mask"
12
+ ],
13
+ "model_max_length": 1000000000000000019884624838656,
14
+ "pad_token": "<|pad|>",
15
+ "sp_model_kwargs": {},
16
+ "spaces_between_special_tokens": false,
17
+ "tokenizer_class": "TokenizersBackend",
18
+ "use_default_system_prompt": false,
19
+ "use_fast": true
20
+ }
wsp/WSP_1_2B_Experiment_Report.md ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # WSP 2.0: Weight-Space Programming on LiquidAI/LFM2.5-1.2B-Instruct
2
+
3
+ Run date: 2026-11-18. All inference, inspection, edits and tests ran in Google Colab through the official hosted Colab MCP (Jupyter kernel `runtimes/wsp-lfm25-12b-l4/sessions/wsp-main`). No training, gradients, optimizers, LoRA/PEFT or external model APIs were used (`torch.set_grad_enabled(False)`, all parameters `requires_grad=False`).
4
+
5
+ ## Verdict
6
+
7
+ A sparse, reversible, training-free weight edit (**c11**) repaired the strict-JSON markdown-fence defect of LFM2.5-1.2B-Instruct on an untouched held-out test set, without breaking any previously correct top-level array answer. This is a **behavior repair** (suppressing an unwanted output format under weak "JSON only" instructions), not a new capability.
8
+
9
+ | Held-out test (run once) | Base | Edited (c11) |
10
+ |---|---|---|
11
+ | Strict-JSON exact, pooled greedy+s0+s1+s2 | 167/256 (65.2%) | 196/256 (76.6%) |
12
+ | Improved / broken (paired) | | 29 / 0 (exact McNemar p = 3.7e-09) |
13
+ | Strict-JSON exact, greedy | 42/64 (65.6%) | 49/64 (76.6%) (p = 0.016) |
14
+ | Fenced strict-JSON replies, pooled | 41/256 | 0/256 |
15
+ | Top-level arrays exact, pooled | 76/128 | 97/128 (0 broken) |
16
+ | Objects exact, pooled | 91/128 | 99/128 |
17
+ | Matched controls (random cols+dir / same cols random dir) | | 167/256 / 166/256 |
18
+ | Passage perplexity (WikiText-2 test, 24x512 tokens) | 130.547 | -0.073% relative |
19
+ | Changed stored scalars | | 32,768 (0.0028% of 1,170,340,608 unique) |
20
+
21
+ ## 1. Objective and constraints
22
+ Continue the WSP methodology from the November 4, 2026 LFM2.5-350M experiment on LFM2.5-1.2B-Instruct: diagnose a measurable behavior defect with forward passes only, build 12 evidence-based sparse weight patches, select one with pre-registered gates on a validation split, and evaluate it once on an untouched test split with matched controls and integrity checks. No 350M layer indices, columns, directions or patch bytes were reused.
23
+
24
+ ## 2. Model and environment
25
+ - Model: `LiquidAI/LFM2.5-1.2B-Instruct` at revision `b9d816ab510f9a28dca37cb182e9da8b0226e13c` (native Transformers/safetensors, BF16). Class `Lfm2ForCausalLM`, 16 layers (10 short-conv + 6 GQA attention), hidden 2048, FFN 8192 (SwiGLU w1/w3/w2), vocab 65,536.
26
+ - Tied weights: `model.embed_tokens.weight` = `lm_head.weight` (counted once). Unique parameters: 1,170,340,608. Budget 0.05% = 585,170 scalars; hard max 0.1% = 1,170,340.
27
+ - Colab: Pro subscription; eligible GPUs listed: T4, L4, G4, A100 (H100 listed as not eligible). No runtime existed; created `runtimes/wsp-lfm25-12b-l4` (NVIDIA L4, 22.03 GiB, compute capability 8.9, native BF16). Python 3.13.16, torch 2.14.1+cu130, transformers 5.22.1, safetensors 0.8.0. The optional `causal_conv1d` kernel was not installed; Transformers used its reference PyTorch conv path (correct, slower).
28
+ - Original snapshot SHA-256 hashes were recorded at start and re-verified at the end: all unchanged (`results/inspection/base_hashes_*.json`).
29
+
30
+ ## 3. Benchmark (regenerated)
31
+ The original 350M `prompts.jsonl` was not found in accessible storage (Google Drive searched), so a comparable benchmark was regenerated deterministically (`src/wsp_data.py`, seed 20261118) and frozen with SHA-256 hashes before any model output was examined (`data/split_manifest.json`).
32
+ - Categories: **B** strict JSON only (objects, top-level arrays, nested objects/arrays, numerals and negatives, non-ASCII and Arabic), **A** termination/repetition, **C** concise exact answers, **S** general sanity, **F** prompts that require a markdown code fence (Python, shell, and JSON-inside-```json).
33
+ - Splits: explore 92 (48 B: 24 object/24 array), validation 124 (64 B: 32/32), test 124 (64 B: 32/32). Prompts are disjoint across splits (asserted).
34
+ - Four instruction phrasings for B are rotated: "Respond with JSON only.", "Output only valid JSON with no markdown, no code fences and no explanation.", "Return strictly the JSON and nothing else.", "Answer in JSON."
35
+ - Passages: 72 random 512-token chunks of `Salesforce/wikitext` wikitext-2-raw-v1 test (revision `b08601e04326c79dfdd32d625aee71d232d685c3`), 24 per split.
36
+ - Decoding: greedy and sampled T=0.7/top-p 0.95/top-k 50 with seeds 0,1,2 (pooled = "comparable" modes), max_new_tokens 256, batch size 64, fixed composition, official chat template. A separate labeled run used the model card's settings (T=0.1, top-k 50, repetition_penalty 1.05, seed 0); it is never pooled.
37
+ - Rubric: B exact = whole reply parses with `json.loads` and deep-equals the expected value (bool distinct from int, list order exact); strict parse, schema validity, fence presence, first character and parse-after-unfencing are reported separately.
38
+
39
+ ## 4. Baseline and defect decision (explore split only)
40
+ - Base explore, greedy: B exact 33/48, 10/48 replies fenced, and 41/48 would be exact after unfencing. Validation base pooled B exact 171/256 with 50/256 fenced.
41
+ - Fences were concentrated on weak phrasings (explore, pooled over 5 modes): "Answer in JSON." fenced 30/35 array replies and 6/25 object replies; "Return strictly the JSON and nothing else." fenced 10/50; the two phrasings that explicitly say JSON-only/no fences produced 0 fences. **Arrays fence more than objects in 1.2B**, the opposite risk profile to 350M.
42
+ - Decision: a repeated, measurable fence defect exists (milder than in 350M), so it was kept as the primary target. Secondary faults (alphabet-range overrun in A, duplicated order items, numbers emitted as strings in sorts) were not targeted and were monitored as regressions.
43
+
44
+ ## 5. Mechanism (forward passes only, explore split)
45
+ - First generated token: fence `` ``` `` (id 26298) vs JSON starts `{\n` (3219), `{"` (38685), `[\n` (14701), `["` (13884). All F prompts also start with 26298, so global fence suppression would be harmful.
46
+ - Direction: u = normalized g * (E[fence] - mean(E[obj], E[arr])), with the component along g * (E[obj] - E[arr]) projected out (g = final RMSNorm weight, E = tied embedding), so that removing u cannot directly change the `{` vs `[` preference.
47
+ - Direct logit attribution at the last prompt position (fenced B vs JSON-starting B vs F): the largest fence pushes are in late FFNs (layer 15 FFN +1.00 vs +0.03; layer 13 FFN +0.39 vs -0.46; layer 15 conv operator +0.30 vs -0.24). F prompts receive a much larger push (+3.55 at layer 15) and a margin of about 14 nats over JSON starts.
48
+ - Causal tests with hooks (no weight edits): removing the u-write of the top-32 ranked w2 neurons dropped fenced-B fence log-prob from -0.29 to -7.22 and made 0/10 fenced prompts prefer a fence, while F still preferred a fence 8/8; layer-matched random neurons had no effect (9/10). In generation on explore, pooled B exact rose 138 to 164, fences 36 to 0, A/C/S/F unchanged, random-neuron hook 137. Raising alpha to 2 to 3 began removing required F fences.
49
+
50
+ ## 6. Candidates (validation split, pooled greedy+s0+s1+s2; base B 171/256, greedy 45/64, arrays 73/128)
51
+ Every candidate was computed in FP32 on a copy and cast to BF16; counts are bitwise-changed stored values. After each candidate/control, the touched tensors were restored and checked by SHA-256, and the full-model fingerprint was re-verified.
52
+
53
+ | id | hypothesis | changed (pct) | B | ctrl rnd / scd | greedy B | fences | arrays | arrays lost | lost A/C/S/F | PPL | KL first-token | KL passage | gates |
54
+ |---|---|---|---|---|---|---|---|---|---|---|---|---|---|
55
+ | c01 | Top-32 fence-writing w2 neurons (L10-15, explore DLA), full removal of u-write | 63,108 (0.0054%) | 203 | 171 / 172 | 51 | 0 | 92 | 3 | 0/0/0/0 | -0.070% | 0.878 | 0.00070 | FAIL: G5_arrays |
56
+ | c02 | Top-64 neurons, alpha 1.0 | 125,036 (0.0107%) | 201 | 172 / 171 | 51 | 0 | 89 | 5 | 0/0/0/0 | -0.067% | 1.226 | 0.00072 | FAIL: G5_arrays |
57
+ | c03 | Top-128 neurons, alpha 1.0 (column-budget sweep) | 247,436 (0.0211%) | 200 | 172 / 172 | 50 | 0 | 87 | 6 | 2/1/0/2 | +0.007% | 1.687 | 0.00076 | FAIL: G3_ACS, G5_arrays, G6_fences |
58
+ | c04 | Top-64 neurons, partial removal alpha 0.75 (dose) | 123,188 (0.0105%) | 203 | 172 / 172 | 51 | 0 | 92 | 3 | 2/1/0/0 | -0.044% | 0.864 | 0.00068 | FAIL: G3_ACS, G5_arrays |
59
+ | c05 | Top-32 neurons, overshoot alpha 1.25 (dose) | 63,558 (0.0054%) | 204 | 171 / 172 | 51 | 0 | 92 | 4 | 0/0/0/0 | -0.090% | 1.146 | 0.00073 | FAIL: G5_arrays |
60
+ | c06 | Layer-15 FFN only, top-64 neurons, alpha 1.0 | 123,162 (0.0105%) | 203 | 172 / 172 | 52 | 0 | 93 | 1 | 3/1/0/0 | +0.011% | 0.620 | 0.00018 | FAIL: G3_ACS, G5_arrays |
61
+ | c07 | Layers 12-14 FFN only, top-64 neurons, alpha 1.0 (excludes last layer) | 123,548 (0.0106%) | 203 | 171 / 172 | 51 | 0 | 92 | 1 | 0/1/0/0 | -0.048% | 0.883 | 0.00049 | FAIL: G5_arrays |
62
+ | c08 | Top-48 neurons + layer-15 conv.out_proj top-16 input channels, alpha 1.0 (operator+FFN) | 125,136 (0.0107%) | 201 | 172 / 173 | 50 | 0 | 89 | 4 | 2/0/0/0 | -0.142% | 1.296 | 0.00072 | FAIL: G3_ACS, G5_arrays |
63
+ | c09 | Top-64 by differential DLA (fencedB minus jsonB), alpha 1.0 | 125,586 (0.0107%) | 111 | 172 / 170 | 28 | 117 | 84 | 2 | 0/1/0/0 | -0.075% | 0.335 | 0.00069 | FAIL: G1_primary, G2_greedy, G5_arrays |
64
+ | c10 | Top-64 by fencedB DLA minus 0.5x positive F DLA (protect required fences), alpha 1.0 | 124,483 (0.0106%) | 200 | 174 / 172 | 50 | 0 | 89 | 5 | 0/0/0/0 | -0.026% | 0.940 | 0.00067 | FAIL: G5_arrays |
65
+ | c11 | Selected-column scaling: top-16 neurons scaled by 0.5 (whole write vector, not directional) | 32,768 (0.0028%) | 209 | 172 / 172 | 52 | 0 | 98 | 0 | 1/0/0/0 | -0.034% | 0.459 | 0.00154 | PASS |
66
+ | c12 | Top-64 neurons, alpha 1.0 with UNguarded direction u_raw (no {/[ orthogonalization) to test array guard | 125,087 (0.0107%) | 204 | 172 / 172 | 51 | 0 | 93 | 3 | 2/1/0/0 | -0.062% | 1.211 | 0.00072 | FAIL: G3_ACS, G5_arrays |
67
+
68
+ Sham (alpha 0, 0 changed values) reproduced base exactly (B 171, fences 50, KL 0). All 24 matched controls stayed at 170 to 174.
69
+ Note: 10 of 11 rejected candidates failed gate 5 because they broke at least one base-correct array answer, the same failure class as the 350M c12 patch, caught here on validation. c09 (ranking by fenced-minus-JSON differential) selected neurons that suppress fences on JSON-starting prompts, and removing them increased fences to 117.
70
+
71
+ ## 7. Selection
72
+ Only **c11** passed all six pre-registered gates (`results/analysis/preregistered_operational_gates.json`). Ranking was by primary success, then KL, then changed values; with one passing candidate the KL choice (passage vs first-token) does not matter. The KL tie-break definition was fixed after c01 to c09 results had been seen; this did not affect the outcome. The selection and patch hash were frozen before the test split was evaluated (`results/validation/selection.json`, chosen patch SHA-256 `38b9bb6891edfe0064c76faf057e94f33ed7f77a8e08fb7088db7a53efc41d59`).
73
+
74
+ **c11 edit**: scale 16 columns of FFN down-projections (w2, shape [2048, 8192]) by 0.5, i.e. halve these neurons' entire write vectors. Columns: `model.layers.12.feed_forward.w2.weight` [4731, 7264, 7919]; `model.layers.13.feed_forward.w2.weight` [488, 5186, 6715]; `model.layers.14.feed_forward.w2.weight` [3659]; `model.layers.15.feed_forward.w2.weight` [99, 532, 660, 926, 2017, 2439, 3263, 3682, 5191]. 16 x 2048 = 32,768 stored values change; scaling by 0.5 is exact in BF16, so the edit is exactly reversible. Unlike the projection candidates, c11 also halves each neuron's non-fence components, so its off-target effects are broader in principle; empirically it had the lowest first-token KL (0.459) and unchanged perplexity, but the highest passage KL among candidates (0.00154, still very small).
75
+
76
+ ## 8. Held-out test (evaluated once, after selection)
77
+ | model | B exact pooled | B greedy | fences | strict parse | schema valid | objects | arrays | arrays starting with { | A | C | S | F correct | F balanced | PPL change |
78
+ |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
79
+ | base | 167/256 | 42/64 | 41 | 211 | 186 | 91/128 | 76/128 | 0 | 36/64 | 57/64 | 64/64 | 48/48 | 47/48 | 0 |
80
+ | chosen c11 | 196/256 | 49/64 | 0 | 253 | 218 | 99/128 | 97/128 | 1 | 35/64 | 57/64 | 64/64 | 48/48 | 48/48 | -0.073% |
81
+ | sham | 167/256 | 42/64 | 41 | 211 | 186 | 91/128 | 76/128 | 0 | 36/64 | 57/64 | 64/64 | 48/48 | 47/48 | +0.000% |
82
+ | c11_rnd (random cols+dir) | 167/256 | 42/64 | 40 | 212 | 187 | 91/128 | 76/128 | 0 | 36/64 | 56/64 | 64/64 | 48/48 | 47/48 | +0.220% |
83
+ | c11_scd (same cols, random dir) | 166/256 | 42/64 | 40 | 211 | 185 | 91/128 | 75/128 | 0 | 36/64 | 56/64 | 64/64 | 48/48 | 47/48 | +0.076% |
84
+
85
+ Paired base vs c11 (95% Wilson intervals; exact McNemar on discordant pairs; pooled rows treat (prompt, mode) pairs as units, so they are not independent across seeds and p-values are optimistic):
86
+
87
+ | measure | n | base | edited | improved | broken | McNemar p |
88
+ |---|---|---|---|---|---|---|
89
+ | B exact, pooled 4 modes | 256 | 167 [59.2, 70.8]% | 196 [71.0, 81.3]% | 29 | 0 | 3.73e-09 |
90
+ | B exact, greedy | 64 | 42 [53.4, 76.1]% | 49 [64.9, 85.3]% | 7 | 0 | 1.56e-02 |
91
+ | B exact, card settings (separate) | 64 | 42 [53.4, 76.1]% | 50 [66.6, 86.5]% | 8 | 0 | 7.81e-03 |
92
+ | B top-level arrays, pooled | 128 | 76 [50.7, 67.5]% | 97 [67.7, 82.4]% | 21 | 0 | 9.54e-07 |
93
+ | B objects, pooled | 128 | 91 [62.7, 78.2]% | 99 [69.4, 83.7]% | 8 | 0 | 7.81e-03 |
94
+ | B nested (obj+arr), pooled | 80 | 50 [51.5, 72.3]% | 63 [68.6, 86.3]% | 13 | 0 | 2.44e-04 |
95
+ | B non-ASCII/Arabic, pooled | 72 | 65 [81.3, 95.2]% | 71 [92.5, 99.8]% | 6 | 0 | 3.12e-02 |
96
+ | B strict parse, pooled | 256 | 211 [77.3, 86.6]% | 253 [96.6, 99.6]% | 42 | 0 | 4.55e-13 |
97
+ | B schema valid, pooled | 256 | 186 [66.9, 77.8]% | 218 [80.3, 89.0]% | 32 | 0 | 4.66e-10 |
98
+ | A termination, pooled | 64 | 36 [44.1, 67.7]% | 35 [42.6, 66.3]% | 0 | 1 | 1.00e+00 |
99
+ | C concise, pooled | 64 | 57 [79.1, 94.6]% | 57 [79.1, 94.6]% | 0 | 0 | 1.00e+00 |
100
+ | S sanity, pooled | 64 | 64 [94.3, 100.0]% | 64 [94.3, 100.0]% | 0 | 0 | 1.00e+00 |
101
+ | F required fences, pooled | 48 | 48 [92.6, 100.0]% | 48 [92.6, 100.0]% | 0 | 0 | 1.00e+00 |
102
+
103
+ Model-card settings (separate, not pooled): B exact 42/64 base vs 50/64 edited.
104
+
105
+ Changed array answers on test (all modes): 28 fixed, 0 broken, 18 correct before and after (layout only), 17 wrong before and after. Each is listed in `results/test/chosen_changed_array_answers_test.json`. One array reply is now wrapped in an object (`{"sorted_numbers": [25, 27, 39, 57, 80]}`, an R3 sort item that base also got wrong). Non-JSON regressions: 1 A item (counting answer lost its spaces after commas).
106
+ Distribution shift on test: KL first-token 0.423 nats (mostly the intended fence-to-JSON shift on B prompts), KL passage 0.00150 nats per position, PPL -0.073%.
107
+
108
+ ## 9. Integrity and reversibility
109
+ - Native edited checkpoint saved to `checkpoints/edited_best/` (safetensors). Reloaded from disk: all 148 stored tensors equal the original file plus the patch bitwise; exactly 32,768 values differ; tied weights preserved; greedy and s0 generations on the test split are token-identical to the in-memory edited model under the same batching.
110
+ - `src/apply_patch.py` applied to the pristine snapshot reproduces the checkpoint tensors bitwise; reverting reproduces the original tensors bitwise (tensor-level equality; the reverted safetensors file is not byte-identical to the upstream file because serialization metadata differs).
111
+ - Automated tests (`tests/test_wsp_patch.py`): 8 passed (budget and counts, coordinates limited to selected columns, every patched value differs bitwise, apply/revert/apply-again bitwise with double-apply refused, checkpoint equals base+patch, original hashes unchanged, frozen and disjoint splits, scoring rules).
112
+
113
+ ## 10. Limitations
114
+ - The benchmark is regenerated and template-based, not the original 350M prompts; numbers are not directly comparable to 350M.
115
+ - The defect is instruction-phrasing dependent: most fences come from "Answer in JSON." which arguably permits a fenced reply. Under the pre-registered strict rubric they count as failures.
116
+ - Small samples (64 B prompts per split); pooled McNemar treats seeds as separate pairs, which overstates independence.
117
+ - Remaining B failures after the edit are content errors (sorted numbers emitted as strings or unsorted, duplicated order items, object wrappers), not fences.
118
+ - The patch also lowers the fence prior in other contexts; required fences in F were preserved on validation and test, but other code-heavy uses were not evaluated.
119
+ - KL tie-break definition was finalized after partial results (no effect on outcome).
120
+ - Persistence: `drive.mount` timed out in the headless MCP kernel and the runtime's file endpoint is blocked by the egress allowlist of Claude's workspace. The report, comparison CSV, README, chosen-patch manifest and `src/wsp_patch.py`, `src/apply_patch.py`, `src/rebuild_patch.py` were copied to Google Drive through the Drive connector. Everything else (generations, all candidate/control patches, the 23.9 MB bundle `/content/wsp_lfm25_1p2b_artifacts.tar.gz` and the 2.2 GB checkpoint) remains on the Colab runtime only. `src/rebuild_patch.py` regenerates the chosen patch byte-for-byte from the pinned base and the manifest, and `src/apply_patch.py` then rebuilds the checkpoint bitwise.
121
+
122
+ ## 11. Reproducibility
123
+ ```
124
+ pip freeze > requirements.lock.txt # recorded in requirements.lock.txt
125
+ python -c "from huggingface_hub import snapshot_download; print(snapshot_download('LiquidAI/LFM2.5-1.2B-Instruct', revision='b9d816ab510f9a28dca37cb182e9da8b0226e13c'))"
126
+ python src/apply_patch.py --base <snapshot_dir> --patch patches/chosen_patch.safetensors --manifest patches/chosen_patch.json --out checkpoints/edited_best
127
+ WSP_SNAP=<snapshot_dir> python -m pytest -q tests/
128
+ ```
129
+ The notebook `notebooks/WSP_LFM2_5_1_2B_Colab.ipynb` contains the full pipeline; the held-out test cell is guarded by a flag so it is not re-run accidentally.
130
+
131
+ ## 12. License notice
132
+ The base model is distributed under Liquid AI's LFM Open License v1.0 (see `checkpoints/edited_best/LICENSE`, copied unchanged). The edited checkpoint is a derivative of that model and is subject to the same license. It was not uploaded or published anywhere.
wsp/apply_patch.py ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ """Deterministic apply/revert of a WSP sparse patch to a native safetensors checkpoint (no training, no grad).
3
+ Usage:
4
+ python src/apply_patch.py --base <snapshot_dir> --patch patches/chosen_patch.safetensors --manifest patches/chosen_patch.json --out <out_dir>
5
+ python src/apply_patch.py --base <edited_dir> --patch ... --manifest ... --out <restored_dir> --revert
6
+ Never writes into --base."""
7
+ import argparse, os, shutil, json, hashlib, torch
8
+ from safetensors.torch import load_file, save_file
9
+ import sys; sys.path.insert(0, os.path.dirname(__file__))
10
+ import wsp_patch as WP
11
+
12
+ def apply_to_state(sd, patch, reverse=False):
13
+ n = 0
14
+ for name, t in patch["tensors"].items():
15
+ flat = sd[name].view(-1)
16
+ expect = t["new"] if reverse else t["orig"]
17
+ if not torch.equal(flat[t["idx"]].view(torch.int16), expect.view(torch.int16)):
18
+ raise ValueError(f"precondition failed for {name}: coordinates are not in the expected {'edited' if reverse else 'original'} state")
19
+ flat[t["idx"]] = t["orig"] if reverse else t["new"]; n += t["idx"].numel()
20
+ return n
21
+
22
+ def main():
23
+ ap = argparse.ArgumentParser(); ap.add_argument("--base", required=True); ap.add_argument("--patch", required=True)
24
+ ap.add_argument("--manifest", required=True); ap.add_argument("--out", required=True); ap.add_argument("--revert", action="store_true")
25
+ a = ap.parse_args()
26
+ assert os.path.realpath(a.out) != os.path.realpath(a.base), "refusing to overwrite base"
27
+ patch = WP.load(a.patch, a.manifest)
28
+ os.makedirs(a.out, exist_ok=True)
29
+ for fn in os.listdir(a.base):
30
+ p = os.path.join(a.base, fn)
31
+ if os.path.isfile(p) and not fn.endswith(".safetensors"): shutil.copy(p, os.path.join(a.out, fn))
32
+ sd = load_file(os.path.join(a.base, "model.safetensors"))
33
+ n = apply_to_state(sd, patch, reverse=a.revert)
34
+ save_file({k: v.contiguous() for k, v in sd.items()}, os.path.join(a.out, "model.safetensors"), metadata={"format": "pt"})
35
+ print(json.dumps({"written": a.out, "coordinates": n, "revert": a.revert}))
36
+
37
+ if __name__ == "__main__": main()
wsp/candidate_table.json ADDED
@@ -0,0 +1,512 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "id": "c01",
4
+ "hypothesis": "Top-32 fence-writing w2 neurons (L10-15, explore DLA), full removal of u-write",
5
+ "changed_scalars": 63108,
6
+ "pct": 0.005392276365411735,
7
+ "B": 203,
8
+ "B_base": 171,
9
+ "ctrl_rnd": 171,
10
+ "ctrl_scd": 172,
11
+ "greedy_B": 51,
12
+ "fence": 0,
13
+ "obj": 111,
14
+ "arr": 92,
15
+ "arr_wrong_start": 4,
16
+ "arr_lost": 3,
17
+ "arr_changed": 47,
18
+ "A": 35,
19
+ "A_lost": 0,
20
+ "C": 57,
21
+ "C_lost": 0,
22
+ "S": 64,
23
+ "S_lost": 0,
24
+ "F": 48,
25
+ "F_lost": 0,
26
+ "F_balanced": 48,
27
+ "ppl_rel_pct": -0.06974605608557788,
28
+ "kl_first_token": 0.8779671788215637,
29
+ "kl_passage": 0.000702628109138459,
30
+ "card_B": 49,
31
+ "gates": {
32
+ "G1_primary": true,
33
+ "G2_greedy": true,
34
+ "G3_ACS": true,
35
+ "G4_ppl": true,
36
+ "G5_arrays": false,
37
+ "G6_fences": true
38
+ },
39
+ "passed": false,
40
+ "fail_reasons": [
41
+ "G5_arrays"
42
+ ]
43
+ },
44
+ {
45
+ "id": "c02",
46
+ "hypothesis": "Top-64 neurons, alpha 1.0",
47
+ "changed_scalars": 125036,
48
+ "pct": 0.010683727382037485,
49
+ "B": 201,
50
+ "B_base": 171,
51
+ "ctrl_rnd": 172,
52
+ "ctrl_scd": 171,
53
+ "greedy_B": 51,
54
+ "fence": 0,
55
+ "obj": 112,
56
+ "arr": 89,
57
+ "arr_wrong_start": 5,
58
+ "arr_lost": 5,
59
+ "arr_changed": 53,
60
+ "A": 36,
61
+ "A_lost": 0,
62
+ "C": 57,
63
+ "C_lost": 0,
64
+ "S": 64,
65
+ "S_lost": 0,
66
+ "F": 48,
67
+ "F_lost": 0,
68
+ "F_balanced": 48,
69
+ "ppl_rel_pct": -0.06740658702218308,
70
+ "kl_first_token": 1.226070523262024,
71
+ "kl_passage": 0.0007198884850367904,
72
+ "card_B": 51,
73
+ "gates": {
74
+ "G1_primary": true,
75
+ "G2_greedy": true,
76
+ "G3_ACS": true,
77
+ "G4_ppl": true,
78
+ "G5_arrays": false,
79
+ "G6_fences": true
80
+ },
81
+ "passed": false,
82
+ "fail_reasons": [
83
+ "G5_arrays"
84
+ ]
85
+ },
86
+ {
87
+ "id": "c03",
88
+ "hypothesis": "Top-128 neurons, alpha 1.0 (column-budget sweep)",
89
+ "changed_scalars": 247436,
90
+ "pct": 0.021142221188312386,
91
+ "B": 200,
92
+ "B_base": 171,
93
+ "ctrl_rnd": 172,
94
+ "ctrl_scd": 172,
95
+ "greedy_B": 50,
96
+ "fence": 0,
97
+ "obj": 113,
98
+ "arr": 87,
99
+ "arr_wrong_start": 6,
100
+ "arr_lost": 6,
101
+ "arr_changed": 56,
102
+ "A": 33,
103
+ "A_lost": 2,
104
+ "C": 56,
105
+ "C_lost": 1,
106
+ "S": 64,
107
+ "S_lost": 0,
108
+ "F": 46,
109
+ "F_lost": 2,
110
+ "F_balanced": 45,
111
+ "ppl_rel_pct": 0.006832348731874619,
112
+ "kl_first_token": 1.68700110912323,
113
+ "kl_passage": 0.0007575758500024676,
114
+ "card_B": 50,
115
+ "gates": {
116
+ "G1_primary": true,
117
+ "G2_greedy": true,
118
+ "G3_ACS": false,
119
+ "G4_ppl": true,
120
+ "G5_arrays": false,
121
+ "G6_fences": false
122
+ },
123
+ "passed": false,
124
+ "fail_reasons": [
125
+ "G3_ACS",
126
+ "G5_arrays",
127
+ "G6_fences"
128
+ ]
129
+ },
130
+ {
131
+ "id": "c04",
132
+ "hypothesis": "Top-64 neurons, partial removal alpha 0.75 (dose)",
133
+ "changed_scalars": 123188,
134
+ "pct": 0.010525824632413337,
135
+ "B": 203,
136
+ "B_base": 171,
137
+ "ctrl_rnd": 172,
138
+ "ctrl_scd": 172,
139
+ "greedy_B": 51,
140
+ "fence": 0,
141
+ "obj": 111,
142
+ "arr": 92,
143
+ "arr_wrong_start": 4,
144
+ "arr_lost": 3,
145
+ "arr_changed": 47,
146
+ "A": 34,
147
+ "A_lost": 2,
148
+ "C": 56,
149
+ "C_lost": 1,
150
+ "S": 64,
151
+ "S_lost": 0,
152
+ "F": 48,
153
+ "F_lost": 0,
154
+ "F_balanced": 48,
155
+ "ppl_rel_pct": -0.04365867255089998,
156
+ "kl_first_token": 0.8644779324531555,
157
+ "kl_passage": 0.0006773422355763614,
158
+ "card_B": 49,
159
+ "gates": {
160
+ "G1_primary": true,
161
+ "G2_greedy": true,
162
+ "G3_ACS": false,
163
+ "G4_ppl": true,
164
+ "G5_arrays": false,
165
+ "G6_fences": true
166
+ },
167
+ "passed": false,
168
+ "fail_reasons": [
169
+ "G3_ACS",
170
+ "G5_arrays"
171
+ ]
172
+ },
173
+ {
174
+ "id": "c05",
175
+ "hypothesis": "Top-32 neurons, overshoot alpha 1.25 (dose)",
176
+ "changed_scalars": 63558,
177
+ "pct": 0.005430726710287746,
178
+ "B": 204,
179
+ "B_base": 171,
180
+ "ctrl_rnd": 171,
181
+ "ctrl_scd": 172,
182
+ "greedy_B": 51,
183
+ "fence": 0,
184
+ "obj": 112,
185
+ "arr": 92,
186
+ "arr_wrong_start": 4,
187
+ "arr_lost": 4,
188
+ "arr_changed": 53,
189
+ "A": 36,
190
+ "A_lost": 0,
191
+ "C": 57,
192
+ "C_lost": 0,
193
+ "S": 64,
194
+ "S_lost": 0,
195
+ "F": 48,
196
+ "F_lost": 0,
197
+ "F_balanced": 48,
198
+ "ppl_rel_pct": -0.0903214756131369,
199
+ "kl_first_token": 1.1455447673797607,
200
+ "kl_passage": 0.0007336658891290426,
201
+ "card_B": 50,
202
+ "gates": {
203
+ "G1_primary": true,
204
+ "G2_greedy": true,
205
+ "G3_ACS": true,
206
+ "G4_ppl": true,
207
+ "G5_arrays": false,
208
+ "G6_fences": true
209
+ },
210
+ "passed": false,
211
+ "fail_reasons": [
212
+ "G5_arrays"
213
+ ]
214
+ },
215
+ {
216
+ "id": "c06",
217
+ "hypothesis": "Layer-15 FFN only, top-64 neurons, alpha 1.0",
218
+ "changed_scalars": 123162,
219
+ "pct": 0.010523603056931612,
220
+ "B": 203,
221
+ "B_base": 171,
222
+ "ctrl_rnd": 172,
223
+ "ctrl_scd": 172,
224
+ "greedy_B": 52,
225
+ "fence": 0,
226
+ "obj": 110,
227
+ "arr": 93,
228
+ "arr_wrong_start": 2,
229
+ "arr_lost": 1,
230
+ "arr_changed": 41,
231
+ "A": 33,
232
+ "A_lost": 3,
233
+ "C": 56,
234
+ "C_lost": 1,
235
+ "S": 64,
236
+ "S_lost": 0,
237
+ "F": 48,
238
+ "F_lost": 0,
239
+ "F_balanced": 48,
240
+ "ppl_rel_pct": 0.010543353799160668,
241
+ "kl_first_token": 0.6203965544700623,
242
+ "kl_passage": 0.0001765814085956663,
243
+ "card_B": 51,
244
+ "gates": {
245
+ "G1_primary": true,
246
+ "G2_greedy": true,
247
+ "G3_ACS": false,
248
+ "G4_ppl": true,
249
+ "G5_arrays": false,
250
+ "G6_fences": true
251
+ },
252
+ "passed": false,
253
+ "fail_reasons": [
254
+ "G3_ACS",
255
+ "G5_arrays"
256
+ ]
257
+ },
258
+ {
259
+ "id": "c07",
260
+ "hypothesis": "Layers 12-14 FFN only, top-64 neurons, alpha 1.0 (excludes last layer)",
261
+ "changed_scalars": 123548,
262
+ "pct": 0.010556584908314145,
263
+ "B": 203,
264
+ "B_base": 171,
265
+ "ctrl_rnd": 171,
266
+ "ctrl_scd": 172,
267
+ "greedy_B": 51,
268
+ "fence": 0,
269
+ "obj": 111,
270
+ "arr": 92,
271
+ "arr_wrong_start": 7,
272
+ "arr_lost": 1,
273
+ "arr_changed": 46,
274
+ "A": 36,
275
+ "A_lost": 0,
276
+ "C": 56,
277
+ "C_lost": 1,
278
+ "S": 64,
279
+ "S_lost": 0,
280
+ "F": 48,
281
+ "F_lost": 0,
282
+ "F_balanced": 48,
283
+ "ppl_rel_pct": -0.04767011046267075,
284
+ "kl_first_token": 0.8833448886871338,
285
+ "kl_passage": 0.0004925022949464619,
286
+ "card_B": 49,
287
+ "gates": {
288
+ "G1_primary": true,
289
+ "G2_greedy": true,
290
+ "G3_ACS": true,
291
+ "G4_ppl": true,
292
+ "G5_arrays": false,
293
+ "G6_fences": true
294
+ },
295
+ "passed": false,
296
+ "fail_reasons": [
297
+ "G5_arrays"
298
+ ]
299
+ },
300
+ {
301
+ "id": "c08",
302
+ "hypothesis": "Top-48 neurons + layer-15 conv.out_proj top-16 input channels, alpha 1.0 (operator+FFN)",
303
+ "changed_scalars": 125136,
304
+ "pct": 0.010692271903121045,
305
+ "B": 201,
306
+ "B_base": 171,
307
+ "ctrl_rnd": 172,
308
+ "ctrl_scd": 173,
309
+ "greedy_B": 50,
310
+ "fence": 0,
311
+ "obj": 112,
312
+ "arr": 89,
313
+ "arr_wrong_start": 7,
314
+ "arr_lost": 4,
315
+ "arr_changed": 49,
316
+ "A": 34,
317
+ "A_lost": 2,
318
+ "C": 57,
319
+ "C_lost": 0,
320
+ "S": 64,
321
+ "S_lost": 0,
322
+ "F": 48,
323
+ "F_lost": 0,
324
+ "F_balanced": 48,
325
+ "ppl_rel_pct": -0.14208340808263803,
326
+ "kl_first_token": 1.2964017391204834,
327
+ "kl_passage": 0.0007245565066114068,
328
+ "card_B": 49,
329
+ "gates": {
330
+ "G1_primary": true,
331
+ "G2_greedy": true,
332
+ "G3_ACS": false,
333
+ "G4_ppl": true,
334
+ "G5_arrays": false,
335
+ "G6_fences": true
336
+ },
337
+ "passed": false,
338
+ "fail_reasons": [
339
+ "G3_ACS",
340
+ "G5_arrays"
341
+ ]
342
+ },
343
+ {
344
+ "id": "c09",
345
+ "hypothesis": "Top-64 by differential DLA (fencedB minus jsonB), alpha 1.0",
346
+ "changed_scalars": 125586,
347
+ "pct": 0.010730722247997055,
348
+ "B": 111,
349
+ "B_base": 171,
350
+ "ctrl_rnd": 172,
351
+ "ctrl_scd": 170,
352
+ "greedy_B": 28,
353
+ "fence": 117,
354
+ "obj": 27,
355
+ "arr": 84,
356
+ "arr_wrong_start": 1,
357
+ "arr_lost": 2,
358
+ "arr_changed": 32,
359
+ "A": 36,
360
+ "A_lost": 0,
361
+ "C": 56,
362
+ "C_lost": 1,
363
+ "S": 64,
364
+ "S_lost": 0,
365
+ "F": 48,
366
+ "F_lost": 0,
367
+ "F_balanced": 48,
368
+ "ppl_rel_pct": -0.07453622661626728,
369
+ "kl_first_token": 0.33516380190849304,
370
+ "kl_passage": 0.000685647944919765,
371
+ "card_B": 26,
372
+ "gates": {
373
+ "G1_primary": false,
374
+ "G2_greedy": false,
375
+ "G3_ACS": true,
376
+ "G4_ppl": true,
377
+ "G5_arrays": false,
378
+ "G6_fences": true
379
+ },
380
+ "passed": false,
381
+ "fail_reasons": [
382
+ "G1_primary",
383
+ "G2_greedy",
384
+ "G5_arrays"
385
+ ]
386
+ },
387
+ {
388
+ "id": "c10",
389
+ "hypothesis": "Top-64 by fencedB DLA minus 0.5x positive F DLA (protect required fences), alpha 1.0",
390
+ "changed_scalars": 124483,
391
+ "pct": 0.010636476180445411,
392
+ "B": 200,
393
+ "B_base": 171,
394
+ "ctrl_rnd": 174,
395
+ "ctrl_scd": 172,
396
+ "greedy_B": 50,
397
+ "fence": 0,
398
+ "obj": 111,
399
+ "arr": 89,
400
+ "arr_wrong_start": 5,
401
+ "arr_lost": 5,
402
+ "arr_changed": 51,
403
+ "A": 36,
404
+ "A_lost": 0,
405
+ "C": 57,
406
+ "C_lost": 0,
407
+ "S": 64,
408
+ "S_lost": 0,
409
+ "F": 48,
410
+ "F_lost": 0,
411
+ "F_balanced": 48,
412
+ "ppl_rel_pct": -0.02627391448543535,
413
+ "kl_first_token": 0.9400752186775208,
414
+ "kl_passage": 0.0006735011702403426,
415
+ "card_B": 49,
416
+ "gates": {
417
+ "G1_primary": true,
418
+ "G2_greedy": true,
419
+ "G3_ACS": true,
420
+ "G4_ppl": true,
421
+ "G5_arrays": false,
422
+ "G6_fences": true
423
+ },
424
+ "passed": false,
425
+ "fail_reasons": [
426
+ "G5_arrays"
427
+ ]
428
+ },
429
+ {
430
+ "id": "c11",
431
+ "hypothesis": "Selected-column scaling: top-16 neurons scaled by 0.5 (whole write vector, not directional)",
432
+ "changed_scalars": 32768,
433
+ "pct": 0.0027998686686602605,
434
+ "B": 209,
435
+ "B_base": 171,
436
+ "ctrl_rnd": 172,
437
+ "ctrl_scd": 172,
438
+ "greedy_B": 52,
439
+ "fence": 0,
440
+ "obj": 111,
441
+ "arr": 98,
442
+ "arr_wrong_start": 2,
443
+ "arr_lost": 0,
444
+ "arr_changed": 58,
445
+ "A": 34,
446
+ "A_lost": 1,
447
+ "C": 57,
448
+ "C_lost": 0,
449
+ "S": 64,
450
+ "S_lost": 0,
451
+ "F": 48,
452
+ "F_lost": 0,
453
+ "F_balanced": 48,
454
+ "ppl_rel_pct": -0.0338363373721002,
455
+ "kl_first_token": 0.4593990445137024,
456
+ "kl_passage": 0.001542521989904344,
457
+ "card_B": 52,
458
+ "gates": {
459
+ "G1_primary": true,
460
+ "G2_greedy": true,
461
+ "G3_ACS": true,
462
+ "G4_ppl": true,
463
+ "G5_arrays": true,
464
+ "G6_fences": true
465
+ },
466
+ "passed": true,
467
+ "fail_reasons": []
468
+ },
469
+ {
470
+ "id": "c12",
471
+ "hypothesis": "Top-64 neurons, alpha 1.0 with UNguarded direction u_raw (no {/[ orthogonalization) to test array guard",
472
+ "changed_scalars": 125087,
473
+ "pct": 0.010688085087790102,
474
+ "B": 204,
475
+ "B_base": 171,
476
+ "ctrl_rnd": 172,
477
+ "ctrl_scd": 172,
478
+ "greedy_B": 51,
479
+ "fence": 0,
480
+ "obj": 111,
481
+ "arr": 93,
482
+ "arr_wrong_start": 0,
483
+ "arr_lost": 3,
484
+ "arr_changed": 51,
485
+ "A": 33,
486
+ "A_lost": 2,
487
+ "C": 56,
488
+ "C_lost": 1,
489
+ "S": 64,
490
+ "S_lost": 0,
491
+ "F": 48,
492
+ "F_lost": 0,
493
+ "F_balanced": 48,
494
+ "ppl_rel_pct": -0.062281844356339544,
495
+ "kl_first_token": 1.2112196683883667,
496
+ "kl_passage": 0.0007187300361692905,
497
+ "card_B": 50,
498
+ "gates": {
499
+ "G1_primary": true,
500
+ "G2_greedy": true,
501
+ "G3_ACS": false,
502
+ "G4_ppl": true,
503
+ "G5_arrays": false,
504
+ "G6_fences": true
505
+ },
506
+ "passed": false,
507
+ "fail_reasons": [
508
+ "G3_ACS",
509
+ "G5_arrays"
510
+ ]
511
+ }
512
+ ]
wsp/chosen_patch.json ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "meta": [
3
+ {
4
+ "param": "model.layers.12.feed_forward.w2.weight",
5
+ "kind": "scale",
6
+ "n_cols": 3,
7
+ "alpha": null,
8
+ "scale": 0.5,
9
+ "dir": "u",
10
+ "fp32_delta_fro": 0.4245277941226959
11
+ },
12
+ {
13
+ "param": "model.layers.13.feed_forward.w2.weight",
14
+ "kind": "scale",
15
+ "n_cols": 3,
16
+ "alpha": null,
17
+ "scale": 0.5,
18
+ "dir": "u",
19
+ "fp32_delta_fro": 0.42069733142852783
20
+ },
21
+ {
22
+ "param": "model.layers.14.feed_forward.w2.weight",
23
+ "kind": "scale",
24
+ "n_cols": 1,
25
+ "alpha": null,
26
+ "scale": 0.5,
27
+ "dir": "u",
28
+ "fp32_delta_fro": 0.21136318147182465
29
+ },
30
+ {
31
+ "param": "model.layers.15.feed_forward.w2.weight",
32
+ "kind": "scale",
33
+ "n_cols": 9,
34
+ "alpha": null,
35
+ "scale": 0.5,
36
+ "dir": "u",
37
+ "fp32_delta_fro": 0.6414475440979004
38
+ }
39
+ ],
40
+ "changed_scalars": 32768,
41
+ "params": {
42
+ "model.layers.12.feed_forward.w2.weight": {
43
+ "shape": [
44
+ 2048,
45
+ 8192
46
+ ],
47
+ "dtype": "torch.bfloat16",
48
+ "stored_delta_fro": 0.4245277941226959,
49
+ "orig_param_sha256": "067ff8e12180fa846bdcb760afd760fd05faab5b1b93758570d4f2bb0c21bfff",
50
+ "n_changed": 6144
51
+ },
52
+ "model.layers.13.feed_forward.w2.weight": {
53
+ "shape": [
54
+ 2048,
55
+ 8192
56
+ ],
57
+ "dtype": "torch.bfloat16",
58
+ "stored_delta_fro": 0.42069730162620544,
59
+ "orig_param_sha256": "65e32f6f6547e8aa25c5c7af8daa4a0e3a7ea698b6b9954c97ba8bfa453e9b28",
60
+ "n_changed": 6144
61
+ },
62
+ "model.layers.14.feed_forward.w2.weight": {
63
+ "shape": [
64
+ 2048,
65
+ 8192
66
+ ],
67
+ "dtype": "torch.bfloat16",
68
+ "stored_delta_fro": 0.21136318147182465,
69
+ "orig_param_sha256": "4c062b2b883563da02f39495b5198b7d82aa18ba49020b68781327cf9ef3fb93",
70
+ "n_changed": 2048
71
+ },
72
+ "model.layers.15.feed_forward.w2.weight": {
73
+ "shape": [
74
+ 2048,
75
+ 8192
76
+ ],
77
+ "dtype": "torch.bfloat16",
78
+ "stored_delta_fro": 0.6414475440979004,
79
+ "orig_param_sha256": "fd2d9d919ab0ad882e397178b562c1019b286ff4d6b214db27a506522d00496e",
80
+ "n_changed": 18432
81
+ }
82
+ },
83
+ "id": "c11",
84
+ "hypothesis": "Selected-column scaling: top-16 neurons scaled by 0.5 (whole write vector, not directional)",
85
+ "edits_spec": [
86
+ {
87
+ "param": "model.layers.12.feed_forward.w2.weight",
88
+ "cols": [
89
+ 4731,
90
+ 7264,
91
+ 7919
92
+ ],
93
+ "kind": "scale",
94
+ "alpha": null,
95
+ "scale": 0.5,
96
+ "dir": "u"
97
+ },
98
+ {
99
+ "param": "model.layers.13.feed_forward.w2.weight",
100
+ "cols": [
101
+ 488,
102
+ 5186,
103
+ 6715
104
+ ],
105
+ "kind": "scale",
106
+ "alpha": null,
107
+ "scale": 0.5,
108
+ "dir": "u"
109
+ },
110
+ {
111
+ "param": "model.layers.14.feed_forward.w2.weight",
112
+ "cols": [
113
+ 3659
114
+ ],
115
+ "kind": "scale",
116
+ "alpha": null,
117
+ "scale": 0.5,
118
+ "dir": "u"
119
+ },
120
+ {
121
+ "param": "model.layers.15.feed_forward.w2.weight",
122
+ "cols": [
123
+ 99,
124
+ 532,
125
+ 660,
126
+ 926,
127
+ 2017,
128
+ 2439,
129
+ 3263,
130
+ 3682,
131
+ 5191
132
+ ],
133
+ "kind": "scale",
134
+ "alpha": null,
135
+ "scale": 0.5,
136
+ "dir": "u"
137
+ }
138
+ ],
139
+ "pct_unique": 0.0027998686686602605,
140
+ "within_0.05pct": true,
141
+ "seed": 20261118,
142
+ "safetensors_sha256": "38b9bb6891edfe0064c76faf057e94f33ed7f77a8e08fb7088db7a53efc41d59",
143
+ "selected": true,
144
+ "selection_utc": "2026-11-18T17:24:09Z"
145
+ }
wsp/chosen_patch.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38b9bb6891edfe0064c76faf057e94f33ed7f77a8e08fb7088db7a53efc41d59
3
+ size 394608
wsp/comparison_350M_vs_1_2B.csv ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ metric,350M_historical (Nov 4 2026),1.2B_measured_test (Nov 18 2026),comparability_note
2
+ model / revision,LiquidAI/LFM2.5-350M @ 9921cd4d82c0e3986887aa4b376f9446729ef8a3,LiquidAI/LFM2.5-1.2B-Instruct @ b9d816ab510f9a28dca37cb182e9da8b0226e13c,different model size and architecture depth
3
+ benchmark,original prompts.jsonl (48 held-out strict-JSON),"regenerated benchmark, 64 held-out strict-JSON (32 obj/32 arr)",different prompts; 350M file unavailable
4
+ decoding units,per original protocol (48 held-out),"greedy + sampled seeds 0,1,2 pooled (256 units); greedy alone 64",pooled counts not comparable to 48-prompt counts
5
+ strict-JSON exact base,1/48 (2.1%),167/256 (65.2%); greedy 42/64,1.2B base is far stronger
6
+ strict-JSON exact edited,28/48 (58.3%),196/256 (76.6%); greedy 49/64,
7
+ fence starts / fenced replies base,41/48,41/256 start with fence; 41/256 contain fence,
8
+ fence starts edited,0/48,0/256,
9
+ random controls,3 matched random: 1/48 each,random cols+dir 167/256; same cols random dir 166/256; sham 167/256,
10
+ top-level arrays edited,0/16 correct (major regression; first token over-biased to {),97/128 vs base 76/128; 0 base-correct arrays broken,1.2B arrays fenced more than objects at baseline
11
+ perplexity base -> edited,44.542 -> 44.575 (+0.074%),130.547 -> -0.073% relative,different passage corpus
12
+ sanity,unchanged,S 64->64/64; C 57->57/64; A 36->35/64; F 48->48/48,
13
+ edited tensors,model.layers.15.conv.out_proj.weight + model.layers.14.feed_forward.w2.weight (64 cols each),"w2 of layers 12,13,14,15 (16 columns total)",independently selected for 1.2B
14
+ edit operator,"rank-one projection, FP32 computed, BF16 stored",column scaling x0.5 (exact in BF16); projection variants evaluated but failed array gate,
15
+ changed stored scalars,"125,566 (0.0354%)","32,768 (0.0028%)",
16
+ candidates,12 (c01-c12); selected c12,12 (c01-c12); selected c11 (only one passing all gates),candidate ids are not the same designs
17
+ GPU,not recorded here,"NVIDIA L4 22 GiB (Colab Pro, hosted MCP)",
wsp/rebuild_patch.py ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ """Rebuild a WSP patch from its JSON manifest and the pinned base model (no binary patch needed). CPU is sufficient for scale edits.
3
+ Usage: python src/rebuild_patch.py --base <snapshot_dir> --manifest patches/chosen_patch.json --out-st rebuilt.safetensors --out-json rebuilt.json"""
4
+ import argparse, os, sys, json, torch
5
+ sys.path.insert(0, os.path.dirname(__file__))
6
+ import wsp_patch as WP
7
+ from transformers import AutoModelForCausalLM
8
+ torch.set_grad_enabled(False)
9
+ ap = argparse.ArgumentParser(); ap.add_argument("--base", required=True); ap.add_argument("--manifest", required=True)
10
+ ap.add_argument("--out-st", required=True); ap.add_argument("--out-json", required=True); a = ap.parse_args()
11
+ spec = json.load(open(a.manifest))
12
+ assert all(e["kind"] == "scale" for e in spec["edits_spec"]), "direction-based edits also need results/analysis/directions.pt"
13
+ model = AutoModelForCausalLM.from_pretrained(a.base, dtype=torch.bfloat16).eval()
14
+ p = WP.build(model, spec["edits_spec"], {})
15
+ j = WP.save(p, a.out_st, a.out_json, extra={k: spec[k] for k in ("id", "hypothesis", "edits_spec", "pct_unique", "within_0.05pct", "seed") if k in spec})
16
+ print(json.dumps({"changed_scalars": p["changed_scalars"], "safetensors_sha256": j["safetensors_sha256"], "matches_manifest": j["safetensors_sha256"] == spec["safetensors_sha256"]}))
wsp/selection.json ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "selected": "c11",
3
+ "frozen_before_test": true,
4
+ "utc": "2026-11-18T17:24:09Z",
5
+ "chosen_patch_sha256": "38b9bb6891edfe0064c76faf057e94f33ed7f77a8e08fb7088db7a53efc41d59",
6
+ "reasoning": [
7
+ "Only c11 passed all six pre-registered gates on validation.",
8
+ "c11: pooled B exact 209/256 vs base 171 (+38); matched random-position control 172 and same-column random-direction control 172.",
9
+ "Greedy B 52 vs base 45; array exact 98 vs 73 with 0 lost base-correct array answers; A lost 1 (s2 count formatting), C/S/F lost 0; F_balanced 48 vs 45.",
10
+ "Passage PPL -0.034% relative; KL first-token 0.459 (lowest of all candidates), KL passage 0.00154 (highest among candidates but tiny).",
11
+ "Ranking is identical whether KL tie-break uses passage KL or first-token KL (only one passing candidate). Note: the KL variant for tie-breaking was fixed after c01-c09 results were seen; it did not affect the outcome.",
12
+ "10 of 11 rejected candidates failed G5 (lost >=1 base-correct array answer), several also G3 (A/C losses); c09 failed G1/G2 (fences increased to 117)."
13
+ ],
14
+ "edits": [
15
+ {
16
+ "param": "model.layers.12.feed_forward.w2.weight",
17
+ "cols": [
18
+ 4731,
19
+ 7264,
20
+ 7919
21
+ ],
22
+ "kind": "scale",
23
+ "alpha": null,
24
+ "scale": 0.5,
25
+ "dir": "u"
26
+ },
27
+ {
28
+ "param": "model.layers.13.feed_forward.w2.weight",
29
+ "cols": [
30
+ 488,
31
+ 5186,
32
+ 6715
33
+ ],
34
+ "kind": "scale",
35
+ "alpha": null,
36
+ "scale": 0.5,
37
+ "dir": "u"
38
+ },
39
+ {
40
+ "param": "model.layers.14.feed_forward.w2.weight",
41
+ "cols": [
42
+ 3659
43
+ ],
44
+ "kind": "scale",
45
+ "alpha": null,
46
+ "scale": 0.5,
47
+ "dir": "u"
48
+ },
49
+ {
50
+ "param": "model.layers.15.feed_forward.w2.weight",
51
+ "cols": [
52
+ 99,
53
+ 532,
54
+ 660,
55
+ 926,
56
+ 2017,
57
+ 2439,
58
+ 3263,
59
+ 3682,
60
+ 5191
61
+ ],
62
+ "kind": "scale",
63
+ "alpha": null,
64
+ "scale": 0.5,
65
+ "dir": "u"
66
+ }
67
+ ],
68
+ "changed_scalars": 32768,
69
+ "pct_unique": 0.0027998686686602605
70
+ }
wsp/test_summary.json ADDED
@@ -0,0 +1,962 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "chosen_c11": {
3
+ "pooled": {
4
+ "B_correct": 196,
5
+ "B_strict_parse": 253,
6
+ "B_schema": 218,
7
+ "B_fence": 0,
8
+ "B_start_brace": 129,
9
+ "B_start_bracket": 127,
10
+ "B_start_fence": 0,
11
+ "B_object_exact": 99,
12
+ "B_array_exact": 97,
13
+ "B_array_wrong_container_start": 1,
14
+ "B_object_wrong_container_start": 0,
15
+ "A_correct": 35,
16
+ "C_correct": 57,
17
+ "S_correct": 64,
18
+ "F_correct": 48,
19
+ "F_balanced": 48,
20
+ "length_failures": 0
21
+ },
22
+ "greedy_B": 49,
23
+ "ppl_rel": -0.0007307205221399877,
24
+ "kl_first_token": 0.42265161871910095,
25
+ "kl_passage": 0.0014987997710704803,
26
+ "card_B": 50,
27
+ "B_pooled": {
28
+ "n": 256,
29
+ "base": 167,
30
+ "edit": 196,
31
+ "improved": 29,
32
+ "broken": 0,
33
+ "mcnemar_exact_p": 3.725290298461914e-09,
34
+ "base_ci": [
35
+ 0.5921427947221091,
36
+ 0.7080400859251775
37
+ ],
38
+ "edit_ci": [
39
+ 0.7100415017359011,
40
+ 0.8133542901619322
41
+ ]
42
+ },
43
+ "B_greedy": {
44
+ "n": 64,
45
+ "base": 42,
46
+ "edit": 49,
47
+ "improved": 7,
48
+ "broken": 0,
49
+ "mcnemar_exact_p": 0.015625,
50
+ "base_ci": [
51
+ 0.5340341181762039,
52
+ 0.7607702496453097
53
+ ],
54
+ "edit_ci": [
55
+ 0.6486650974929234,
56
+ 0.8525023278036498
57
+ ]
58
+ },
59
+ "B_card": {
60
+ "n": 64,
61
+ "base": 42,
62
+ "edit": 50,
63
+ "improved": 8,
64
+ "broken": 0,
65
+ "mcnemar_exact_p": 0.0078125,
66
+ "base_ci": [
67
+ 0.5340341181762039,
68
+ 0.7607702496453097
69
+ ],
70
+ "edit_ci": [
71
+ 0.6656698366922221,
72
+ 0.8649780253865025
73
+ ]
74
+ },
75
+ "B_array_pooled": {
76
+ "n": 128,
77
+ "base": 76,
78
+ "edit": 97,
79
+ "improved": 21,
80
+ "broken": 0,
81
+ "mcnemar_exact_p": 9.5367431640625e-07,
82
+ "base_ci": [
83
+ 0.5071382471838717,
84
+ 0.674898378585233
85
+ ],
86
+ "edit_ci": [
87
+ 0.6767870437261758,
88
+ 0.8238136771388622
89
+ ]
90
+ },
91
+ "B_object_pooled": {
92
+ "n": 128,
93
+ "base": 91,
94
+ "edit": 99,
95
+ "improved": 8,
96
+ "broken": 0,
97
+ "mcnemar_exact_p": 0.0078125,
98
+ "base_ci": [
99
+ 0.6271652314384711,
100
+ 0.7824171765420145
101
+ ],
102
+ "edit_ci": [
103
+ 0.6935716289452746,
104
+ 0.8373685295479476
105
+ ]
106
+ },
107
+ "B_nested_pooled": {
108
+ "n": 80,
109
+ "base": 50,
110
+ "edit": 63,
111
+ "improved": 13,
112
+ "broken": 0,
113
+ "mcnemar_exact_p": 0.000244140625,
114
+ "base_ci": [
115
+ 0.515485228513904,
116
+ 0.7230598374199523
117
+ ],
118
+ "edit_ci": [
119
+ 0.6857763938182242,
120
+ 0.8628772578296451
121
+ ]
122
+ },
123
+ "B_nonascii_pooled": {
124
+ "n": 72,
125
+ "base": 65,
126
+ "edit": 71,
127
+ "improved": 6,
128
+ "broken": 0,
129
+ "mcnemar_exact_p": 0.03125,
130
+ "base_ci": [
131
+ 0.812647509746432,
132
+ 0.952104288738832
133
+ ],
134
+ "edit_ci": [
135
+ 0.9254321971571302,
136
+ 0.9975441113595678
137
+ ]
138
+ },
139
+ "B_strict_parse_pooled": {
140
+ "n": 256,
141
+ "base": 211,
142
+ "edit": 253,
143
+ "improved": 42,
144
+ "broken": 0,
145
+ "mcnemar_exact_p": 4.547473508864641e-13,
146
+ "base_ci": [
147
+ 0.772896116682119,
148
+ 0.8659546293108245
149
+ ],
150
+ "edit_ci": [
151
+ 0.9661178322810585,
152
+ 0.9960067851781936
153
+ ]
154
+ },
155
+ "B_schema_pooled": {
156
+ "n": 256,
157
+ "base": 186,
158
+ "edit": 218,
159
+ "improved": 32,
160
+ "broken": 0,
161
+ "mcnemar_exact_p": 4.656612873077393e-10,
162
+ "base_ci": [
163
+ 0.6689134962060558,
164
+ 0.7775123262950373
165
+ ],
166
+ "edit_ci": [
167
+ 0.8028238602392018,
168
+ 0.8899058643314598
169
+ ]
170
+ },
171
+ "A": {
172
+ "n": 64,
173
+ "base": 36,
174
+ "edit": 35,
175
+ "improved": 0,
176
+ "broken": 1,
177
+ "mcnemar_exact_p": 1.0,
178
+ "base_ci": [
179
+ 0.44085991778287875,
180
+ 0.6770618293457267
181
+ ],
182
+ "edit_ci": [
183
+ 0.4257338252067978,
184
+ 0.6627074851396563
185
+ ]
186
+ },
187
+ "C": {
188
+ "n": 64,
189
+ "base": 57,
190
+ "edit": 57,
191
+ "improved": 0,
192
+ "broken": 0,
193
+ "mcnemar_exact_p": 1.0,
194
+ "base_ci": [
195
+ 0.7910113670806147,
196
+ 0.9459995524731696
197
+ ],
198
+ "edit_ci": [
199
+ 0.7910113670806147,
200
+ 0.9459995524731696
201
+ ]
202
+ },
203
+ "S": {
204
+ "n": 64,
205
+ "base": 64,
206
+ "edit": 64,
207
+ "improved": 0,
208
+ "broken": 0,
209
+ "mcnemar_exact_p": 1.0,
210
+ "base_ci": [
211
+ 0.9433739770288436,
212
+ 1.0
213
+ ],
214
+ "edit_ci": [
215
+ 0.9433739770288436,
216
+ 1.0
217
+ ]
218
+ },
219
+ "F": {
220
+ "n": 48,
221
+ "base": 48,
222
+ "edit": 48,
223
+ "improved": 0,
224
+ "broken": 0,
225
+ "mcnemar_exact_p": 1.0,
226
+ "base_ci": [
227
+ 0.9258973488472576,
228
+ 0.9999999999999998
229
+ ],
230
+ "edit_ci": [
231
+ 0.9258973488472576,
232
+ 0.9999999999999998
233
+ ]
234
+ }
235
+ },
236
+ "sham": {
237
+ "pooled": {
238
+ "B_correct": 167,
239
+ "B_strict_parse": 211,
240
+ "B_schema": 186,
241
+ "B_fence": 41,
242
+ "B_start_brace": 119,
243
+ "B_start_bracket": 96,
244
+ "B_start_fence": 41,
245
+ "B_object_exact": 91,
246
+ "B_array_exact": 76,
247
+ "B_array_wrong_container_start": 0,
248
+ "B_object_wrong_container_start": 0,
249
+ "A_correct": 36,
250
+ "C_correct": 57,
251
+ "S_correct": 64,
252
+ "F_correct": 48,
253
+ "F_balanced": 47,
254
+ "length_failures": 0
255
+ },
256
+ "greedy_B": 42,
257
+ "ppl_rel": 0.0,
258
+ "kl_first_token": 0.0,
259
+ "kl_passage": 0.0,
260
+ "card_B": 42,
261
+ "B_pooled": {
262
+ "n": 256,
263
+ "base": 167,
264
+ "edit": 167,
265
+ "improved": 0,
266
+ "broken": 0,
267
+ "mcnemar_exact_p": 1.0,
268
+ "base_ci": [
269
+ 0.5921427947221091,
270
+ 0.7080400859251775
271
+ ],
272
+ "edit_ci": [
273
+ 0.5921427947221091,
274
+ 0.7080400859251775
275
+ ]
276
+ },
277
+ "B_greedy": {
278
+ "n": 64,
279
+ "base": 42,
280
+ "edit": 42,
281
+ "improved": 0,
282
+ "broken": 0,
283
+ "mcnemar_exact_p": 1.0,
284
+ "base_ci": [
285
+ 0.5340341181762039,
286
+ 0.7607702496453097
287
+ ],
288
+ "edit_ci": [
289
+ 0.5340341181762039,
290
+ 0.7607702496453097
291
+ ]
292
+ },
293
+ "B_card": {
294
+ "n": 64,
295
+ "base": 42,
296
+ "edit": 42,
297
+ "improved": 0,
298
+ "broken": 0,
299
+ "mcnemar_exact_p": 1.0,
300
+ "base_ci": [
301
+ 0.5340341181762039,
302
+ 0.7607702496453097
303
+ ],
304
+ "edit_ci": [
305
+ 0.5340341181762039,
306
+ 0.7607702496453097
307
+ ]
308
+ },
309
+ "B_array_pooled": {
310
+ "n": 128,
311
+ "base": 76,
312
+ "edit": 76,
313
+ "improved": 0,
314
+ "broken": 0,
315
+ "mcnemar_exact_p": 1.0,
316
+ "base_ci": [
317
+ 0.5071382471838717,
318
+ 0.674898378585233
319
+ ],
320
+ "edit_ci": [
321
+ 0.5071382471838717,
322
+ 0.674898378585233
323
+ ]
324
+ },
325
+ "B_object_pooled": {
326
+ "n": 128,
327
+ "base": 91,
328
+ "edit": 91,
329
+ "improved": 0,
330
+ "broken": 0,
331
+ "mcnemar_exact_p": 1.0,
332
+ "base_ci": [
333
+ 0.6271652314384711,
334
+ 0.7824171765420145
335
+ ],
336
+ "edit_ci": [
337
+ 0.6271652314384711,
338
+ 0.7824171765420145
339
+ ]
340
+ },
341
+ "B_nested_pooled": {
342
+ "n": 80,
343
+ "base": 50,
344
+ "edit": 50,
345
+ "improved": 0,
346
+ "broken": 0,
347
+ "mcnemar_exact_p": 1.0,
348
+ "base_ci": [
349
+ 0.515485228513904,
350
+ 0.7230598374199523
351
+ ],
352
+ "edit_ci": [
353
+ 0.515485228513904,
354
+ 0.7230598374199523
355
+ ]
356
+ },
357
+ "B_nonascii_pooled": {
358
+ "n": 72,
359
+ "base": 65,
360
+ "edit": 65,
361
+ "improved": 0,
362
+ "broken": 0,
363
+ "mcnemar_exact_p": 1.0,
364
+ "base_ci": [
365
+ 0.812647509746432,
366
+ 0.952104288738832
367
+ ],
368
+ "edit_ci": [
369
+ 0.812647509746432,
370
+ 0.952104288738832
371
+ ]
372
+ },
373
+ "B_strict_parse_pooled": {
374
+ "n": 256,
375
+ "base": 211,
376
+ "edit": 211,
377
+ "improved": 0,
378
+ "broken": 0,
379
+ "mcnemar_exact_p": 1.0,
380
+ "base_ci": [
381
+ 0.772896116682119,
382
+ 0.8659546293108245
383
+ ],
384
+ "edit_ci": [
385
+ 0.772896116682119,
386
+ 0.8659546293108245
387
+ ]
388
+ },
389
+ "B_schema_pooled": {
390
+ "n": 256,
391
+ "base": 186,
392
+ "edit": 186,
393
+ "improved": 0,
394
+ "broken": 0,
395
+ "mcnemar_exact_p": 1.0,
396
+ "base_ci": [
397
+ 0.6689134962060558,
398
+ 0.7775123262950373
399
+ ],
400
+ "edit_ci": [
401
+ 0.6689134962060558,
402
+ 0.7775123262950373
403
+ ]
404
+ },
405
+ "A": {
406
+ "n": 64,
407
+ "base": 36,
408
+ "edit": 36,
409
+ "improved": 0,
410
+ "broken": 0,
411
+ "mcnemar_exact_p": 1.0,
412
+ "base_ci": [
413
+ 0.44085991778287875,
414
+ 0.6770618293457267
415
+ ],
416
+ "edit_ci": [
417
+ 0.44085991778287875,
418
+ 0.6770618293457267
419
+ ]
420
+ },
421
+ "C": {
422
+ "n": 64,
423
+ "base": 57,
424
+ "edit": 57,
425
+ "improved": 0,
426
+ "broken": 0,
427
+ "mcnemar_exact_p": 1.0,
428
+ "base_ci": [
429
+ 0.7910113670806147,
430
+ 0.9459995524731696
431
+ ],
432
+ "edit_ci": [
433
+ 0.7910113670806147,
434
+ 0.9459995524731696
435
+ ]
436
+ },
437
+ "S": {
438
+ "n": 64,
439
+ "base": 64,
440
+ "edit": 64,
441
+ "improved": 0,
442
+ "broken": 0,
443
+ "mcnemar_exact_p": 1.0,
444
+ "base_ci": [
445
+ 0.9433739770288436,
446
+ 1.0
447
+ ],
448
+ "edit_ci": [
449
+ 0.9433739770288436,
450
+ 1.0
451
+ ]
452
+ },
453
+ "F": {
454
+ "n": 48,
455
+ "base": 48,
456
+ "edit": 48,
457
+ "improved": 0,
458
+ "broken": 0,
459
+ "mcnemar_exact_p": 1.0,
460
+ "base_ci": [
461
+ 0.9258973488472576,
462
+ 0.9999999999999998
463
+ ],
464
+ "edit_ci": [
465
+ 0.9258973488472576,
466
+ 0.9999999999999998
467
+ ]
468
+ }
469
+ },
470
+ "c11_rnd": {
471
+ "pooled": {
472
+ "B_correct": 167,
473
+ "B_strict_parse": 212,
474
+ "B_schema": 187,
475
+ "B_fence": 40,
476
+ "B_start_brace": 119,
477
+ "B_start_bracket": 97,
478
+ "B_start_fence": 40,
479
+ "B_object_exact": 91,
480
+ "B_array_exact": 76,
481
+ "B_array_wrong_container_start": 0,
482
+ "B_object_wrong_container_start": 0,
483
+ "A_correct": 36,
484
+ "C_correct": 56,
485
+ "S_correct": 64,
486
+ "F_correct": 48,
487
+ "F_balanced": 47,
488
+ "length_failures": 0
489
+ },
490
+ "greedy_B": 42,
491
+ "ppl_rel": 0.002196327451208909,
492
+ "kl_first_token": 0.0005274508730508387,
493
+ "kl_passage": 0.0006625415990129113,
494
+ "card_B": 42,
495
+ "B_pooled": {
496
+ "n": 256,
497
+ "base": 167,
498
+ "edit": 167,
499
+ "improved": 1,
500
+ "broken": 1,
501
+ "mcnemar_exact_p": 1.0,
502
+ "base_ci": [
503
+ 0.5921427947221091,
504
+ 0.7080400859251775
505
+ ],
506
+ "edit_ci": [
507
+ 0.5921427947221091,
508
+ 0.7080400859251775
509
+ ]
510
+ },
511
+ "B_greedy": {
512
+ "n": 64,
513
+ "base": 42,
514
+ "edit": 42,
515
+ "improved": 0,
516
+ "broken": 0,
517
+ "mcnemar_exact_p": 1.0,
518
+ "base_ci": [
519
+ 0.5340341181762039,
520
+ 0.7607702496453097
521
+ ],
522
+ "edit_ci": [
523
+ 0.5340341181762039,
524
+ 0.7607702496453097
525
+ ]
526
+ },
527
+ "B_card": {
528
+ "n": 64,
529
+ "base": 42,
530
+ "edit": 42,
531
+ "improved": 0,
532
+ "broken": 0,
533
+ "mcnemar_exact_p": 1.0,
534
+ "base_ci": [
535
+ 0.5340341181762039,
536
+ 0.7607702496453097
537
+ ],
538
+ "edit_ci": [
539
+ 0.5340341181762039,
540
+ 0.7607702496453097
541
+ ]
542
+ },
543
+ "B_array_pooled": {
544
+ "n": 128,
545
+ "base": 76,
546
+ "edit": 76,
547
+ "improved": 1,
548
+ "broken": 1,
549
+ "mcnemar_exact_p": 1.0,
550
+ "base_ci": [
551
+ 0.5071382471838717,
552
+ 0.674898378585233
553
+ ],
554
+ "edit_ci": [
555
+ 0.5071382471838717,
556
+ 0.674898378585233
557
+ ]
558
+ },
559
+ "B_object_pooled": {
560
+ "n": 128,
561
+ "base": 91,
562
+ "edit": 91,
563
+ "improved": 0,
564
+ "broken": 0,
565
+ "mcnemar_exact_p": 1.0,
566
+ "base_ci": [
567
+ 0.6271652314384711,
568
+ 0.7824171765420145
569
+ ],
570
+ "edit_ci": [
571
+ 0.6271652314384711,
572
+ 0.7824171765420145
573
+ ]
574
+ },
575
+ "B_nested_pooled": {
576
+ "n": 80,
577
+ "base": 50,
578
+ "edit": 49,
579
+ "improved": 0,
580
+ "broken": 1,
581
+ "mcnemar_exact_p": 1.0,
582
+ "base_ci": [
583
+ 0.515485228513904,
584
+ 0.7230598374199523
585
+ ],
586
+ "edit_ci": [
587
+ 0.5029345562232477,
588
+ 0.711756003117223
589
+ ]
590
+ },
591
+ "B_nonascii_pooled": {
592
+ "n": 72,
593
+ "base": 65,
594
+ "edit": 65,
595
+ "improved": 0,
596
+ "broken": 0,
597
+ "mcnemar_exact_p": 1.0,
598
+ "base_ci": [
599
+ 0.812647509746432,
600
+ 0.952104288738832
601
+ ],
602
+ "edit_ci": [
603
+ 0.812647509746432,
604
+ 0.952104288738832
605
+ ]
606
+ },
607
+ "B_strict_parse_pooled": {
608
+ "n": 256,
609
+ "base": 211,
610
+ "edit": 212,
611
+ "improved": 1,
612
+ "broken": 0,
613
+ "mcnemar_exact_p": 1.0,
614
+ "base_ci": [
615
+ 0.772896116682119,
616
+ 0.8659546293108245
617
+ ],
618
+ "edit_ci": [
619
+ 0.7771451946116188,
620
+ 0.8694025483209986
621
+ ]
622
+ },
623
+ "B_schema_pooled": {
624
+ "n": 256,
625
+ "base": 186,
626
+ "edit": 187,
627
+ "improved": 2,
628
+ "broken": 1,
629
+ "mcnemar_exact_p": 1.0,
630
+ "base_ci": [
631
+ 0.6689134962060558,
632
+ 0.7775123262950373
633
+ ],
634
+ "edit_ci": [
635
+ 0.6730019753620341,
636
+ 0.781120844078733
637
+ ]
638
+ },
639
+ "A": {
640
+ "n": 64,
641
+ "base": 36,
642
+ "edit": 36,
643
+ "improved": 0,
644
+ "broken": 0,
645
+ "mcnemar_exact_p": 1.0,
646
+ "base_ci": [
647
+ 0.44085991778287875,
648
+ 0.6770618293457267
649
+ ],
650
+ "edit_ci": [
651
+ 0.44085991778287875,
652
+ 0.6770618293457267
653
+ ]
654
+ },
655
+ "C": {
656
+ "n": 64,
657
+ "base": 57,
658
+ "edit": 56,
659
+ "improved": 0,
660
+ "broken": 1,
661
+ "mcnemar_exact_p": 1.0,
662
+ "base_ci": [
663
+ 0.7910113670806147,
664
+ 0.9459995524731696
665
+ ],
666
+ "edit_ci": [
667
+ 0.7722521366978408,
668
+ 0.935278346073792
669
+ ]
670
+ },
671
+ "S": {
672
+ "n": 64,
673
+ "base": 64,
674
+ "edit": 64,
675
+ "improved": 0,
676
+ "broken": 0,
677
+ "mcnemar_exact_p": 1.0,
678
+ "base_ci": [
679
+ 0.9433739770288436,
680
+ 1.0
681
+ ],
682
+ "edit_ci": [
683
+ 0.9433739770288436,
684
+ 1.0
685
+ ]
686
+ },
687
+ "F": {
688
+ "n": 48,
689
+ "base": 48,
690
+ "edit": 48,
691
+ "improved": 0,
692
+ "broken": 0,
693
+ "mcnemar_exact_p": 1.0,
694
+ "base_ci": [
695
+ 0.9258973488472576,
696
+ 0.9999999999999998
697
+ ],
698
+ "edit_ci": [
699
+ 0.9258973488472576,
700
+ 0.9999999999999998
701
+ ]
702
+ }
703
+ },
704
+ "c11_scd": {
705
+ "pooled": {
706
+ "B_correct": 166,
707
+ "B_strict_parse": 211,
708
+ "B_schema": 185,
709
+ "B_fence": 40,
710
+ "B_start_brace": 120,
711
+ "B_start_bracket": 96,
712
+ "B_start_fence": 40,
713
+ "B_object_exact": 91,
714
+ "B_array_exact": 75,
715
+ "B_array_wrong_container_start": 0,
716
+ "B_object_wrong_container_start": 0,
717
+ "A_correct": 36,
718
+ "C_correct": 56,
719
+ "S_correct": 64,
720
+ "F_correct": 48,
721
+ "F_balanced": 47,
722
+ "length_failures": 0
723
+ },
724
+ "greedy_B": 42,
725
+ "ppl_rel": 0.0007641077187694739,
726
+ "kl_first_token": 0.0037935180589556694,
727
+ "kl_passage": 0.0009359674295410514,
728
+ "card_B": 42,
729
+ "B_pooled": {
730
+ "n": 256,
731
+ "base": 167,
732
+ "edit": 166,
733
+ "improved": 0,
734
+ "broken": 1,
735
+ "mcnemar_exact_p": 1.0,
736
+ "base_ci": [
737
+ 0.5921427947221091,
738
+ 0.7080400859251775
739
+ ],
740
+ "edit_ci": [
741
+ 0.5881468207176852,
742
+ 0.7043390629899275
743
+ ]
744
+ },
745
+ "B_greedy": {
746
+ "n": 64,
747
+ "base": 42,
748
+ "edit": 42,
749
+ "improved": 0,
750
+ "broken": 0,
751
+ "mcnemar_exact_p": 1.0,
752
+ "base_ci": [
753
+ 0.5340341181762039,
754
+ 0.7607702496453097
755
+ ],
756
+ "edit_ci": [
757
+ 0.5340341181762039,
758
+ 0.7607702496453097
759
+ ]
760
+ },
761
+ "B_card": {
762
+ "n": 64,
763
+ "base": 42,
764
+ "edit": 42,
765
+ "improved": 0,
766
+ "broken": 0,
767
+ "mcnemar_exact_p": 1.0,
768
+ "base_ci": [
769
+ 0.5340341181762039,
770
+ 0.7607702496453097
771
+ ],
772
+ "edit_ci": [
773
+ 0.5340341181762039,
774
+ 0.7607702496453097
775
+ ]
776
+ },
777
+ "B_array_pooled": {
778
+ "n": 128,
779
+ "base": 76,
780
+ "edit": 75,
781
+ "improved": 0,
782
+ "broken": 1,
783
+ "mcnemar_exact_p": 1.0,
784
+ "base_ci": [
785
+ 0.5071382471838717,
786
+ 0.674898378585233
787
+ ],
788
+ "edit_ci": [
789
+ 0.4993170001197534,
790
+ 0.6675499068352593
791
+ ]
792
+ },
793
+ "B_object_pooled": {
794
+ "n": 128,
795
+ "base": 91,
796
+ "edit": 91,
797
+ "improved": 0,
798
+ "broken": 0,
799
+ "mcnemar_exact_p": 1.0,
800
+ "base_ci": [
801
+ 0.6271652314384711,
802
+ 0.7824171765420145
803
+ ],
804
+ "edit_ci": [
805
+ 0.6271652314384711,
806
+ 0.7824171765420145
807
+ ]
808
+ },
809
+ "B_nested_pooled": {
810
+ "n": 80,
811
+ "base": 50,
812
+ "edit": 50,
813
+ "improved": 0,
814
+ "broken": 0,
815
+ "mcnemar_exact_p": 1.0,
816
+ "base_ci": [
817
+ 0.515485228513904,
818
+ 0.7230598374199523
819
+ ],
820
+ "edit_ci": [
821
+ 0.515485228513904,
822
+ 0.7230598374199523
823
+ ]
824
+ },
825
+ "B_nonascii_pooled": {
826
+ "n": 72,
827
+ "base": 65,
828
+ "edit": 64,
829
+ "improved": 0,
830
+ "broken": 1,
831
+ "mcnemar_exact_p": 1.0,
832
+ "base_ci": [
833
+ 0.812647509746432,
834
+ 0.952104288738832
835
+ ],
836
+ "edit_ci": [
837
+ 0.7957685574140959,
838
+ 0.9426124893992622
839
+ ]
840
+ },
841
+ "B_strict_parse_pooled": {
842
+ "n": 256,
843
+ "base": 211,
844
+ "edit": 211,
845
+ "improved": 1,
846
+ "broken": 1,
847
+ "mcnemar_exact_p": 1.0,
848
+ "base_ci": [
849
+ 0.772896116682119,
850
+ 0.8659546293108245
851
+ ],
852
+ "edit_ci": [
853
+ 0.772896116682119,
854
+ 0.8659546293108245
855
+ ]
856
+ },
857
+ "B_schema_pooled": {
858
+ "n": 256,
859
+ "base": 186,
860
+ "edit": 185,
861
+ "improved": 0,
862
+ "broken": 1,
863
+ "mcnemar_exact_p": 1.0,
864
+ "base_ci": [
865
+ 0.6689134962060558,
866
+ 0.7775123262950373
867
+ ],
868
+ "edit_ci": [
869
+ 0.6648301483971852,
870
+ 0.7738986771642338
871
+ ]
872
+ },
873
+ "A": {
874
+ "n": 64,
875
+ "base": 36,
876
+ "edit": 36,
877
+ "improved": 0,
878
+ "broken": 0,
879
+ "mcnemar_exact_p": 1.0,
880
+ "base_ci": [
881
+ 0.44085991778287875,
882
+ 0.6770618293457267
883
+ ],
884
+ "edit_ci": [
885
+ 0.44085991778287875,
886
+ 0.6770618293457267
887
+ ]
888
+ },
889
+ "C": {
890
+ "n": 64,
891
+ "base": 57,
892
+ "edit": 56,
893
+ "improved": 0,
894
+ "broken": 1,
895
+ "mcnemar_exact_p": 1.0,
896
+ "base_ci": [
897
+ 0.7910113670806147,
898
+ 0.9459995524731696
899
+ ],
900
+ "edit_ci": [
901
+ 0.7722521366978408,
902
+ 0.935278346073792
903
+ ]
904
+ },
905
+ "S": {
906
+ "n": 64,
907
+ "base": 64,
908
+ "edit": 64,
909
+ "improved": 0,
910
+ "broken": 0,
911
+ "mcnemar_exact_p": 1.0,
912
+ "base_ci": [
913
+ 0.9433739770288436,
914
+ 1.0
915
+ ],
916
+ "edit_ci": [
917
+ 0.9433739770288436,
918
+ 1.0
919
+ ]
920
+ },
921
+ "F": {
922
+ "n": 48,
923
+ "base": 48,
924
+ "edit": 48,
925
+ "improved": 0,
926
+ "broken": 0,
927
+ "mcnemar_exact_p": 1.0,
928
+ "base_ci": [
929
+ 0.9258973488472576,
930
+ 0.9999999999999998
931
+ ],
932
+ "edit_ci": [
933
+ 0.9258973488472576,
934
+ 0.9999999999999998
935
+ ]
936
+ }
937
+ },
938
+ "base": {
939
+ "pooled": {
940
+ "B_correct": 167,
941
+ "B_strict_parse": 211,
942
+ "B_schema": 186,
943
+ "B_fence": 41,
944
+ "B_start_brace": 119,
945
+ "B_start_bracket": 96,
946
+ "B_start_fence": 41,
947
+ "B_object_exact": 91,
948
+ "B_array_exact": 76,
949
+ "B_array_wrong_container_start": 0,
950
+ "B_object_wrong_container_start": 0,
951
+ "A_correct": 36,
952
+ "C_correct": 57,
953
+ "S_correct": 64,
954
+ "F_correct": 48,
955
+ "F_balanced": 47,
956
+ "length_failures": 0
957
+ },
958
+ "greedy_B": 42,
959
+ "ppl": 130.5471182180624,
960
+ "card_B": 42
961
+ }
962
+ }
wsp/wsp_patch.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ """WSP 2.0 sparse reversible weight patches. No gradients. FP32 compute on copies, cast to stored dtype, count bitwise changes."""
3
+ import json, hashlib, torch
4
+ from safetensors.torch import save_file, load_file
5
+ torch.set_grad_enabled(False)
6
+
7
+ def _bits(t):
8
+ return t.contiguous().view(torch.int16) if t.dtype in (torch.bfloat16, torch.float16) else t.contiguous().view(torch.int32)
9
+
10
+ def tensor_sha(t):
11
+ return hashlib.sha256(t.detach().contiguous().view(torch.uint8).cpu().numpy().tobytes()).hexdigest()
12
+
13
+ def build(model, edits, dirs):
14
+ """edits: list of {param, cols, kind: proj|scale, alpha|scale, dir: key in dirs}. Returns patch dict."""
15
+ P = dict(model.named_parameters()); out = {}; meta = []
16
+ for e in edits:
17
+ W = P[e["param"]].data; W32 = out[e["param"]]["W32"] if e["param"] in out else W.float().clone()
18
+ J = torch.as_tensor(e["cols"], device=W.device, dtype=torch.long)
19
+ old = W32[:, J].clone()
20
+ if e["kind"] == "proj":
21
+ d = dirs[e["dir"]].to(W.device).float(); d = d / d.norm()
22
+ W32[:, J] = old - e["alpha"] * torch.outer(d, d @ old)
23
+ elif e["kind"] == "scale":
24
+ W32[:, J] = old * e["scale"]
25
+ else: raise ValueError(e["kind"])
26
+ meta.append(dict(param=e["param"], kind=e["kind"], n_cols=len(e["cols"]), alpha=e.get("alpha"), scale=e.get("scale"), dir=e.get("dir"),
27
+ fp32_delta_fro=float((W32[:, J] - old).norm())))
28
+ out.setdefault(e["param"], {})["W32"] = W32
29
+ patch = {"tensors": {}, "meta": meta}
30
+ total = 0
31
+ for name, d in out.items():
32
+ W = P[name].data; new = d["W32"].to(W.dtype)
33
+ ch = (_bits(new) != _bits(W)).reshape(-1).nonzero().squeeze(-1)
34
+ patch["tensors"][name] = {"idx": ch.cpu(), "orig": W.reshape(-1)[ch].cpu().clone(), "new": new.reshape(-1)[ch].cpu().clone(),
35
+ "shape": list(W.shape), "dtype": str(W.dtype),
36
+ "stored_delta_fro": float((new.float() - W.float()).norm()), "orig_param_sha256": tensor_sha(W)}
37
+ total += ch.numel()
38
+ patch["changed_scalars"] = total
39
+ return patch
40
+
41
+ def apply(model, patch, reverse=False):
42
+ P = dict(model.named_parameters())
43
+ for name, t in patch["tensors"].items():
44
+ flat = P[name].data.view(-1); idx = t["idx"].to(flat.device)
45
+ flat[idx] = (t["orig"] if reverse else t["new"]).to(flat.device)
46
+
47
+ def verify(model, patch, state):
48
+ """state: 'orig' or 'new'. Bitwise check of patched coordinates."""
49
+ P = dict(model.named_parameters())
50
+ for name, t in patch["tensors"].items():
51
+ flat = P[name].data.view(-1); idx = t["idx"].to(flat.device)
52
+ ref = (t["new"] if state == "new" else t["orig"]).to(flat.device)
53
+ if not torch.equal(_bits(flat[idx]), _bits(ref)): return False
54
+ return True
55
+
56
+ def save(patch, path_st, path_json, extra=None):
57
+ tens = {}
58
+ for name, t in patch["tensors"].items():
59
+ tens[f"{name}::idx"] = t["idx"].contiguous(); tens[f"{name}::orig"] = t["orig"].contiguous(); tens[f"{name}::new"] = t["new"].contiguous()
60
+ md = {"format": "wsp2-sparse-patch-v1", "changed_scalars": str(patch["changed_scalars"])}
61
+ save_file(tens, path_st, metadata=md)
62
+ j = {"meta": patch["meta"], "changed_scalars": patch["changed_scalars"],
63
+ "params": {n: {k: v for k, v in t.items() if k not in ("idx", "orig", "new")} | {"n_changed": int(t["idx"].numel())} for n, t in patch["tensors"].items()}}
64
+ if extra: j.update(extra)
65
+ j["safetensors_sha256"] = hashlib.sha256(open(path_st, "rb").read()).hexdigest()
66
+ json.dump(j, open(path_json, "w"), indent=1)
67
+ return j
68
+
69
+ def load(path_st, path_json):
70
+ tens = load_file(path_st); j = json.load(open(path_json)); patch = {"tensors": {}, "meta": j["meta"], "changed_scalars": j["changed_scalars"]}
71
+ for n, info in j["params"].items():
72
+ patch["tensors"][n] = {"idx": tens[f"{n}::idx"], "orig": tens[f"{n}::orig"], "new": tens[f"{n}::new"], **info}
73
+ return patch