moindy commited on
Commit
529e9dd
·
0 Parent(s):

Publish MADL model weights

Browse files
.gitattributes ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ *.pt filter=lfs diff=lfs merge=lfs -text
3
+ *.pth filter=lfs diff=lfs merge=lfs -text
4
+ agent_a_qwen_lora/tokenizer.json filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
MANIFEST.json ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "format_version": "madl.weights.v1",
3
+ "base_model": "Qwen/Qwen2.5-VL-7B-Instruct",
4
+ "external_weights_not_included": [
5
+ "SAM ViT-H (sam_vit_h_4b8939.pth)"
6
+ ],
7
+ "files": [
8
+ {
9
+ "path": "agent_a_qwen_lora/adapter_config.json",
10
+ "bytes": 3553,
11
+ "sha256": "64389a52b2f8ab288e51c135253a2e86144638bc88dda9f9bf9e00e951967f3d"
12
+ },
13
+ {
14
+ "path": "agent_a_qwen_lora/adapter_model.safetensors",
15
+ "bytes": 259096432,
16
+ "sha256": "fdf98c24a7ef5389b5eee3bdf94a05413d219bd7021ac6d291e1f60ade890e37"
17
+ },
18
+ {
19
+ "path": "agent_a_qwen_lora/chat_template.jinja",
20
+ "bytes": 1017,
21
+ "sha256": "a0bc6f6fc7a29a80017a433e8f03a1cc1236e838a944a2d034295a60c4f2fddb"
22
+ },
23
+ {
24
+ "path": "agent_a_qwen_lora/processor_config.json",
25
+ "bytes": 1424,
26
+ "sha256": "d2e174783f6eb77885c1c9b59a5739e540f64668cff3abe97df446fac96999c7"
27
+ },
28
+ {
29
+ "path": "agent_a_qwen_lora/tokenizer.json",
30
+ "bytes": 11421892,
31
+ "sha256": "3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8"
32
+ },
33
+ {
34
+ "path": "agent_a_qwen_lora/tokenizer_config.json",
35
+ "bytes": 735,
36
+ "sha256": "d64774ede2507fce562bb5a5287f5fb7ba0b71f83236fc877970fe90348bf9b3"
37
+ },
38
+ {
39
+ "path": "agent_b_dualstream/madl_agent_b_dualstream_v1.pt",
40
+ "bytes": 230858058,
41
+ "sha256": "65a96f53964a8370af0984ef915a6aac513742762b5f805e406bb6261838f10d"
42
+ },
43
+ {
44
+ "path": "agent_b_visual_ranker/madl_agent_b_visual_ranker_v1.pt",
45
+ "bytes": 111951107,
46
+ "sha256": "f7f210ca8cf0ad0686f2331b049f6662b919ea5f0912808279898d4cee23fc8c"
47
+ }
48
+ ]
49
+ }
README.md ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-VL-7B-Instruct
4
+ datasets:
5
+ - Huang-yating/SID_Set
6
+ language:
7
+ - en
8
+ library_name: transformers
9
+ pipeline_tag: image-classification
10
+ tags:
11
+ - image-forensics
12
+ - image-manipulation-localization
13
+ - multi-agent
14
+ - qwen2.5-vl
15
+ ---
16
+
17
+ # MADL model weights
18
+
19
+ This repository is the weight companion for **MADL: Towards Dependable Image
20
+ Forgery Detection via Multi-Agent Forensic Reasoning**. The executable code,
21
+ configuration files, and evaluation scripts are distributed in the companion
22
+ [GitHub code repository](https://github.com/shannima/MADL).
23
+
24
+ Model repository: [moindy/MADL](https://huggingface.co/moindy/MADL).
25
+
26
+ MADL produces one of three image-level labels—`real`, `synthetic`, or
27
+ `tampered`—and returns a binary localization mask only for locally tampered
28
+ images. The release contains the trainable components developed for MADL; it
29
+ does not redistribute the Qwen base model or SAM ViT-H.
30
+
31
+ ## Files
32
+
33
+ | Directory | Component | Purpose |
34
+ |---|---|---|
35
+ | `agent_a_qwen_lora/` | Agent A LoRA adapter | Adapts Qwen2.5-VL-7B-Instruct to three-class semantic verification and region proposals. |
36
+ | `agent_b_dualstream/` | Agent B dual-stream checkpoint | RGB/SRM ConvNeXt-Tiny classifier and continuous manipulation heatmap. |
37
+ | `agent_b_visual_ranker/` | Agent B visual-ranker checkpoint | Selects a final SAM candidate using RGB, mask, heatmap, and geometry evidence. |
38
+ | `MANIFEST.json` | Integrity manifest | Records the byte size and SHA-256 of every released model artifact. |
39
+
40
+ The dual-stream release checkpoint contains only the model state, architecture,
41
+ class names, and a compact training summary. Optimizer state and local training
42
+ paths were intentionally removed. The visual-ranker checkpoint uses the
43
+ strict four-field runtime schema required by the companion loader.
44
+
45
+ ## External dependencies
46
+
47
+ 1. Download `Qwen/Qwen2.5-VL-7B-Instruct` through Transformers or the
48
+ Hugging Face Hub. PEFT reads this identifier from `adapter_config.json`.
49
+ 2. Download the official SAM ViT-H checkpoint
50
+ `sam_vit_h_4b8939.pth` from Meta's Segment Anything release.
51
+ 3. Place this repository under the runtime weight root and place SAM at
52
+ `external/sam_vit_h_4b8939.pth`, or override the relative paths in the MADL
53
+ YAML configuration.
54
+
55
+ Expected layout:
56
+
57
+ ```text
58
+ models/
59
+ ├── agent_a_qwen_lora/
60
+ ├── agent_b_dualstream/
61
+ │ └── madl_agent_b_dualstream_v1.pt
62
+ ├── agent_b_visual_ranker/
63
+ │ └── madl_agent_b_visual_ranker_v1.pt
64
+ └── external/
65
+ └── sam_vit_h_4b8939.pth
66
+ ```
67
+
68
+ Verify the files before inference:
69
+
70
+ ```bash
71
+ python scripts/verify_weights.py /path/to/models
72
+ ```
73
+
74
+ ## Training data and evaluation scope
75
+
76
+ The released components were trained for the SID-Set task, which distinguishes
77
+ authentic, fully synthetic, and locally tampered images and provides masks for
78
+ the locally tampered class. The release does not include SID-Set images. Users
79
+ must obtain the dataset under its own terms.
80
+
81
+ ## Intended use
82
+
83
+ The weights are intended for academic research on image forgery detection,
84
+ localization, evidence interaction, and multi-agent forensic reasoning. They
85
+ are not a substitute for human forensic examination and should not be used as
86
+ the sole basis for legal, disciplinary, or content-removal decisions.
87
+
88
+ ## Limitations
89
+
90
+ - Very small, fragmented, or weak-trace manipulations remain difficult.
91
+ - Cross-dataset behavior has not been established by the final manuscript.
92
+ - Multiple model calls increase latency and GPU memory requirements.
93
+ - The natural-language field is a structured evidence trace; the release does
94
+ not claim that it has been independently evaluated as explanation quality.
95
+ - Results depend on the exact Qwen, SAM, preprocessing, and threshold versions
96
+ documented by the companion code.
97
+
98
+ ## Licenses and attribution
99
+
100
+ The released MADL code and original weight packaging are provided under
101
+ Apache-2.0. Qwen2.5-VL, SAM, SID-Set, and other dependencies retain their own
102
+ licenses and terms. The [code README](https://github.com/shannima/MADL#third-party-components)
103
+ lists the relevant upstream projects. Dataset-derived use should preserve SID-Set attribution.
agent_a_qwen_lora/adapter_config.json ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen2.5-VL-7B-Instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 16,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.0,
22
+ "megatron_config": null,
23
+ "megatron_core": "megatron.core",
24
+ "modules_to_save": [
25
+ "model.visual.merger"
26
+ ],
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.18.1",
29
+ "qalora_group_size": 16,
30
+ "r": 8,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "layers.11.mlp.up_proj",
35
+ "layers.26.mlp.down_proj",
36
+ "layers.22.mlp.gate_proj",
37
+ "layers.18.mlp.gate_proj",
38
+ "layers.24.mlp.gate_proj",
39
+ "layers.17.mlp.up_proj",
40
+ "layers.13.mlp.gate_proj",
41
+ "layers.7.mlp.gate_proj",
42
+ "layers.5.mlp.gate_proj",
43
+ "layers.8.mlp.up_proj",
44
+ "layers.25.mlp.up_proj",
45
+ "layers.4.mlp.gate_proj",
46
+ "layers.7.mlp.up_proj",
47
+ "layers.0.mlp.gate_proj",
48
+ "layers.25.mlp.down_proj",
49
+ "layers.17.mlp.gate_proj",
50
+ "layers.26.mlp.up_proj",
51
+ "layers.10.mlp.gate_proj",
52
+ "layers.1.mlp.gate_proj",
53
+ "layers.17.mlp.down_proj",
54
+ "layers.19.mlp.down_proj",
55
+ "layers.22.mlp.up_proj",
56
+ "v_proj",
57
+ "layers.23.mlp.down_proj",
58
+ "layers.9.mlp.gate_proj",
59
+ "layers.20.mlp.gate_proj",
60
+ "layers.21.mlp.gate_proj",
61
+ "layers.6.mlp.up_proj",
62
+ "layers.18.mlp.down_proj",
63
+ "layers.18.mlp.up_proj",
64
+ "layers.3.mlp.down_proj",
65
+ "layers.3.mlp.up_proj",
66
+ "layers.20.mlp.down_proj",
67
+ "layers.14.mlp.up_proj",
68
+ "layers.16.mlp.down_proj",
69
+ "layers.2.mlp.up_proj",
70
+ "layers.1.mlp.down_proj",
71
+ "layers.23.mlp.gate_proj",
72
+ "layers.6.mlp.down_proj",
73
+ "layers.16.mlp.gate_proj",
74
+ "layers.24.mlp.up_proj",
75
+ "q_proj",
76
+ "layers.21.mlp.up_proj",
77
+ "layers.14.mlp.down_proj",
78
+ "layers.22.mlp.down_proj",
79
+ "layers.6.mlp.gate_proj",
80
+ "layers.19.mlp.gate_proj",
81
+ "layers.2.mlp.down_proj",
82
+ "layers.27.mlp.up_proj",
83
+ "layers.27.mlp.down_proj",
84
+ "layers.11.mlp.down_proj",
85
+ "layers.5.mlp.up_proj",
86
+ "layers.3.mlp.gate_proj",
87
+ "layers.15.mlp.up_proj",
88
+ "layers.19.mlp.up_proj",
89
+ "layers.7.mlp.down_proj",
90
+ "layers.0.mlp.down_proj",
91
+ "layers.2.mlp.gate_proj",
92
+ "layers.9.mlp.up_proj",
93
+ "layers.13.mlp.up_proj",
94
+ "layers.24.mlp.down_proj",
95
+ "layers.12.mlp.up_proj",
96
+ "o_proj",
97
+ "layers.5.mlp.down_proj",
98
+ "layers.13.mlp.down_proj",
99
+ "layers.0.mlp.up_proj",
100
+ "layers.10.mlp.up_proj",
101
+ "layers.21.mlp.down_proj",
102
+ "layers.14.mlp.gate_proj",
103
+ "layers.15.mlp.gate_proj",
104
+ "layers.1.mlp.up_proj",
105
+ "layers.27.mlp.gate_proj",
106
+ "layers.11.mlp.gate_proj",
107
+ "layers.25.mlp.gate_proj",
108
+ "layers.8.mlp.gate_proj",
109
+ "layers.12.mlp.down_proj",
110
+ "layers.9.mlp.down_proj",
111
+ "layers.12.mlp.gate_proj",
112
+ "layers.8.mlp.down_proj",
113
+ "layers.10.mlp.down_proj",
114
+ "layers.23.mlp.up_proj",
115
+ "layers.16.mlp.up_proj",
116
+ "layers.4.mlp.down_proj",
117
+ "layers.26.mlp.gate_proj",
118
+ "k_proj",
119
+ "layers.15.mlp.down_proj",
120
+ "layers.20.mlp.up_proj",
121
+ "layers.4.mlp.up_proj"
122
+ ],
123
+ "target_parameters": null,
124
+ "task_type": "CAUSAL_LM",
125
+ "trainable_token_indices": null,
126
+ "use_dora": false,
127
+ "use_qalora": false,
128
+ "use_rslora": false
129
+ }
agent_a_qwen_lora/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdf98c24a7ef5389b5eee3bdf94a05413d219bd7021ac6d291e1f60ade890e37
3
+ size 259096432
agent_a_qwen_lora/chat_template.jinja ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
2
+ You are a helpful assistant.<|im_end|>
3
+ {% endif %}<|im_start|>{{ message['role'] }}
4
+ {% if message['content'] is string %}{{ message['content'] }}<|im_end|>
5
+ {% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
6
+ {% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
7
+ {% endif %}
agent_a_qwen_lora/processor_config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "image_processor": {
3
+ "data_format": "channels_first",
4
+ "do_convert_rgb": true,
5
+ "do_normalize": true,
6
+ "do_rescale": true,
7
+ "do_resize": true,
8
+ "image_mean": [
9
+ 0.48145466,
10
+ 0.4578275,
11
+ 0.40821073
12
+ ],
13
+ "image_processor_type": "Qwen2VLImageProcessorFast",
14
+ "image_std": [
15
+ 0.26862954,
16
+ 0.26130258,
17
+ 0.27577711
18
+ ],
19
+ "merge_size": 2,
20
+ "patch_size": 14,
21
+ "resample": 3,
22
+ "rescale_factor": 0.00392156862745098,
23
+ "size": {
24
+ "longest_edge": 12845056,
25
+ "shortest_edge": 3136
26
+ },
27
+ "temporal_patch_size": 2
28
+ },
29
+ "processor_class": "Qwen2_5_VLProcessor",
30
+ "video_processor": {
31
+ "data_format": "channels_first",
32
+ "default_to_square": true,
33
+ "do_convert_rgb": true,
34
+ "do_normalize": true,
35
+ "do_rescale": true,
36
+ "do_resize": true,
37
+ "do_sample_frames": false,
38
+ "image_mean": [
39
+ 0.48145466,
40
+ 0.4578275,
41
+ 0.40821073
42
+ ],
43
+ "image_processor_type": "Qwen2VLImageProcessor",
44
+ "image_std": [
45
+ 0.26862954,
46
+ 0.26130258,
47
+ 0.27577711
48
+ ],
49
+ "max_frames": 768,
50
+ "merge_size": 2,
51
+ "min_frames": 4,
52
+ "patch_size": 14,
53
+ "resample": 3,
54
+ "rescale_factor": 0.00392156862745098,
55
+ "return_metadata": false,
56
+ "size": {
57
+ "longest_edge": 12845056,
58
+ "shortest_edge": 3136
59
+ },
60
+ "temporal_patch_size": 2,
61
+ "video_processor_type": "Qwen2VLVideoProcessor"
62
+ }
63
+ }
agent_a_qwen_lora/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
3
+ size 11421892
agent_a_qwen_lora/tokenizer_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": true,
24
+ "model_max_length": 131072,
25
+ "pad_token": "<|endoftext|>",
26
+ "padding_side": "right",
27
+ "processor_class": "Qwen2_5_VLProcessor",
28
+ "split_special_tokens": false,
29
+ "tokenizer_class": "Qwen2Tokenizer",
30
+ "unk_token": null
31
+ }
agent_b_dualstream/madl_agent_b_dualstream_v1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65a96f53964a8370af0984ef915a6aac513742762b5f805e406bb6261838f10d
3
+ size 230858058
agent_b_visual_ranker/madl_agent_b_visual_ranker_v1.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7f210ca8cf0ad0686f2331b049f6662b919ea5f0912808279898d4cee23fc8c
3
+ size 111951107