Simplify model card layout and evaluation; clarify license scope

#2
by Skywalker0410 - opened
Files changed (5) hide show
  1. LICENSE +16 -47
  2. LICENSE-Apache-2.0 +202 -0
  3. LICENSE-Kimi-K3 +52 -0
  4. README.md +27 -96
  5. checksums.sha256 +4 -2
LICENSE CHANGED
@@ -1,52 +1,21 @@
1
- Kimi K3 License
 
2
 
3
- Copyright (c) 2026 Moonshot AI
 
 
 
4
 
5
- Permission is hereby granted, free of charge, to any person (the "Licensee")
6
- obtaining a copy of this software — including the model weights, parameters,
7
- configuration files, inference and training code, and associated documentation
8
- (collectively, the "Software") — to deal in the Software without restriction.
9
- This includes, without limitation, the rights to use, copy, modify, merge,
10
- publish, distribute, sublicense, and/or sell copies of the Software; to run,
11
- deploy, fine-tune, or otherwise modify the Software and create derivative works
12
- from it; and to permit persons to whom the Software is furnished to do so, in
13
- each case subject to the following conditions:
14
 
15
- 1. The above copyright notice and this permission notice shall be included in
16
- all copies or substantial portions of the Software. Licensee's use of the
17
- Software must comply with applicable laws and regulations.
 
18
 
19
- 2. "Model as a Service" means giving a third party access to language model
20
- inference or fine-tuning (e.g., via API) in a manner that allows such third
21
- party to exercise meaningful control over the inputs, parameters, or training
22
- data. This does not include (a) end-user products with model capabilities solely
23
- embedded within specific features or harnesses, or (b) mere relaying of requests
24
- to models hosted by others.
25
 
26
- If the Licensee or any of its affiliates operates a Model as a Service business,
27
- and the aggregate revenue of the Licensee and its affiliates exceeds 20 million
28
- US dollars (or the equivalent in other currencies) in total over any consecutive
29
- 12 months, the Licensee must enter into a separate agreement with Moonshot AI
30
- before using the Software or its derivative works for any commercial purpose.
31
-
32
- 3. If the Software (or any derivative works thereof) is used for any of the
33
- Licensee's commercial products or services that have more than 100 million
34
- monthly active users, or more than 20 million US dollars (or equivalent in other
35
- currencies) in monthly revenue, "Kimi K3" must be prominently displayed on the
36
- user interface of such product or service.
37
-
38
- 4. The requirements set forth in Sections 2 and 3 do not apply to: (a) internal
39
- use of the Software, defined as any use that does not make the Software, its
40
- outputs, or its underlying capabilities available to third parties; or (b) any
41
- use of the Software accessed through Moonshot AI's official products or
42
- certified inference partners.
43
-
44
- 5. THE SOFTWARE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS”
45
- BASIS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
46
- LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
47
- AND NONINFRINGEMENT. IN NO EVENT SHALL MOONSHOT AI OR ITS AFFILIATES OR
48
- COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
49
- IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
50
- CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
51
-
52
- For any questions regarding this license, please contact <license@moonshot.ai>.
 
1
+ License scope
2
+ =============
3
 
4
+ Original contributions by the GroundingPI and GroundAnything authors are
5
+ licensed under the Apache License, Version 2.0, provided in LICENSE-Apache-2.0.
6
+ The project imposes no additional use or commercial restrictions on those
7
+ contributions beyond that license.
8
 
9
+ This grant covers only rights held by the contributing authors. It does not
10
+ relicense third-party material or waive any upstream requirements.
 
 
 
 
 
 
 
11
 
12
+ Kimi-derived material remains subject to the Kimi K3 License, reproduced
13
+ unchanged in LICENSE-Kimi-K3. Its conditions, including conditions that apply
14
+ to derivative works, continue to apply to the combined model package where
15
+ applicable. Apache 2.0 is not an alternative license for that material.
16
 
17
+ Other third-party components retain their own licenses and copyright notices.
18
+ Preserve the applicable license texts and notices when redistributing.
 
 
 
 
19
 
20
+ Upstream Kimi K3 license source:
21
+ https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/LICENSE
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
LICENSE-Apache-2.0 ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
LICENSE-Kimi-K3 ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Kimi K3 License
2
+
3
+ Copyright (c) 2026 Moonshot AI
4
+
5
+ Permission is hereby granted, free of charge, to any person (the "Licensee")
6
+ obtaining a copy of this software — including the model weights, parameters,
7
+ configuration files, inference and training code, and associated documentation
8
+ (collectively, the "Software") — to deal in the Software without restriction.
9
+ This includes, without limitation, the rights to use, copy, modify, merge,
10
+ publish, distribute, sublicense, and/or sell copies of the Software; to run,
11
+ deploy, fine-tune, or otherwise modify the Software and create derivative works
12
+ from it; and to permit persons to whom the Software is furnished to do so, in
13
+ each case subject to the following conditions:
14
+
15
+ 1. The above copyright notice and this permission notice shall be included in
16
+ all copies or substantial portions of the Software. Licensee's use of the
17
+ Software must comply with applicable laws and regulations.
18
+
19
+ 2. "Model as a Service" means giving a third party access to language model
20
+ inference or fine-tuning (e.g., via API) in a manner that allows such third
21
+ party to exercise meaningful control over the inputs, parameters, or training
22
+ data. This does not include (a) end-user products with model capabilities solely
23
+ embedded within specific features or harnesses, or (b) mere relaying of requests
24
+ to models hosted by others.
25
+
26
+ If the Licensee or any of its affiliates operates a Model as a Service business,
27
+ and the aggregate revenue of the Licensee and its affiliates exceeds 20 million
28
+ US dollars (or the equivalent in other currencies) in total over any consecutive
29
+ 12 months, the Licensee must enter into a separate agreement with Moonshot AI
30
+ before using the Software or its derivative works for any commercial purpose.
31
+
32
+ 3. If the Software (or any derivative works thereof) is used for any of the
33
+ Licensee's commercial products or services that have more than 100 million
34
+ monthly active users, or more than 20 million US dollars (or equivalent in other
35
+ currencies) in monthly revenue, "Kimi K3" must be prominently displayed on the
36
+ user interface of such product or service.
37
+
38
+ 4. The requirements set forth in Sections 2 and 3 do not apply to: (a) internal
39
+ use of the Software, defined as any use that does not make the Software, its
40
+ outputs, or its underlying capabilities available to third parties; or (b) any
41
+ use of the Software accessed through Moonshot AI's official products or
42
+ certified inference partners.
43
+
44
+ 5. THE SOFTWARE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS”
45
+ BASIS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
46
+ LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
47
+ AND NONINFRINGEMENT. IN NO EVENT SHALL MOONSHOT AI OR ITS AFFILIATES OR
48
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
49
+ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
50
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
51
+
52
+ For any questions regarding this license, please contact <license@moonshot.ai>.
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  license: other
3
- license_name: kimi-k3
4
- license_link: https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/LICENSE
5
  language:
6
  - en
7
  - zh
@@ -18,20 +18,18 @@ tags:
18
  inference: false
19
  ---
20
 
21
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/logo.png" width="150" alt="GroundAnything logo" /></p>
22
-
23
- # GroundAnything-VLM: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
24
 
25
  **Model family:** [GroundAnything — DLM / parallel decoding](https://huggingface.co/GroundingPI/GroundAnything) · [GroundAnything-VLM — autoregressive](https://huggingface.co/GroundingPI/GroundAnything-VLM).
26
 
27
  **This repository contains the autoregressive GroundAnything-VLM checkpoint.** The family overview and figures below are shared with the DLM page; use the VLM serving recipe for these weights.
28
 
29
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig1-teaser.png" width="100%" alt="GroundAnything Figure 1: broad visual grounding and parallel visual evidence extraction" /></p>
30
 
31
  ## 🔗 Quick Links
32
 
33
  - 🚀 **Online Demo:** Coming soon — XXX.
34
- - 💻 **GitHub Code:** Coming soon — XXX.
35
  - 📄 **Paper:** [arXiv:2609.39600](https://arxiv.org/abs/2609.39600).
36
  - 🧪 **Evaluation data:** Coming soon — XXX.
37
 
@@ -49,13 +47,13 @@ An optional **self-speculative mode** achieves a **4.51× speedup** over the AR
49
 
50
  <video controls playsinline preload="none" width="100%" poster="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/demo-poster.jpg" src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/demo.mp4"></video>
51
 
52
- **Parallel decoding in action**
53
 
54
  <video controls playsinline preload="none" width="100%" src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/decoding.mp4"></video>
55
 
56
  ### License/Terms of Use:
57
 
58
- This checkpoint is released under the **Kimi K3 License**; see the repository's [LICENSE](LICENSE) and the [upstream license](https://huggingface.co/moonshotai/Kimi-K3/blob/f831ab66814297da540d832a5235f8e904f29d06/LICENSE). The model incorporates Kimi K3-derived vision components and implementation code. The license includes additional conditions for certain commercial uses. Third-party components retain their respective licenses and copyright notices.
59
 
60
  ### Deployment Geography:
61
 
@@ -105,13 +103,7 @@ Global.
105
  - **Spatial vocabulary:** 1,000 coordinate tokens shared with semantic labels and protocol markers.
106
  - **DLM conversion:** the shared decoder and vocabulary head support both causal prediction and bidirectional response-block denoising. A mask token is added for diffusion generation.
107
 
108
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig2-architecture.png" width="100%" alt="GroundAnything Figure 2: vision-language architecture and autoregressive-to-diffusion conversion" /></p>
109
-
110
- *Figure 2. Shared model architecture and the conversion to parallel grounding.*
111
-
112
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig4-attention-mask.png" width="640" alt="GroundAnything Figure 4: clean-stream causal and noisy-response block attention" /></p>
113
-
114
- *Figure 4. The attention mask used during diffusion conversion. The clean stream uses causal attention. A noisy response block attends bidirectionally within its block and reads the clean conditioning context and strictly preceding clean response blocks. Conversion uses B=32; the illustration uses two-token blocks.*
115
 
116
  ## Input(s):
117
 
@@ -158,49 +150,35 @@ The default DLM service uses **BF16, Triton attention, eager execution, one GPU,
158
  | [GroundAnything](https://huggingface.co/GroundingPI/GroundAnything) | Entropy-guided blockwise diffusion; optional self-speculation | **GAM** |
159
  | [GroundAnything-VLM](https://huggingface.co/GroundingPI/GroundAnything-VLM) | Autoregressive generation | **GAM** |
160
 
161
- The main GroundAnything benchmark results use **entropy-guided diffusion without autoregressive verification**. GroundAnything-VLM uses autoregressive decoding. The self-speculative speed–quality operating point is a separate experiment and should not be substituted for the main diffusion benchmark setting.
162
-
163
- ## Testing and Evaluation Datasets:
164
-
165
- ### Data Modality:
166
-
167
- Image and text, with task-specific box, point, text-region, or interaction annotations.
168
-
169
- ## Evaluation Dataset:
170
 
171
- **Evaluation data: XXX — download URL coming soon.**
172
 
173
- The shared evaluation toolkit provides **42 task recipes across eight task families**: Grounding, Referring, Dense, OCR, Layout, GUI, Pointing, and VisualPrompt. These are executable task/split recipes, not a count of distinct datasets. Dataset paths are registered in `configs/datasets.yaml`; task IDs are listed in `configs/eval/tasks.json`.
174
 
175
- ### Evaluation Modes
176
-
177
- The evaluator accepts seven mode identifiers. A mode selects the **prompt and output parser** independently of the inference engine and decoding algorithm.
178
-
179
- | Mode | Supported evaluation interface | Output / coordinates |
180
- |:---|:---|:---|
181
- | **`GAM`** | **GroundingPI, GroundAnything, and GroundAnything-VLM** | Native spatial tokens, **0–999** |
182
- | `DLM` | Legacy compatible diffusion-checkpoint alias | Same GAM spatial-token protocol |
183
- | `RLV2` | Compatible RL-checkpoint alias | Same GAM spatial-token protocol |
184
- | `VLM` | Generic VLM baselines | Explicit pixel, 0–1000, or 0–1 coordinate mode |
185
- | `REXOMNI` | Rex-Omni adapter | Model-specific spatial-token parser |
186
- | `LOCATEANYTHING` | LocateAnything adapter | Its own output parser and generation-mode setting |
187
- | `GROUNDINGDINO` | External compatible GroundingDINO bridge | JSON coordinate responses |
188
 
189
- **Use `mode: GAM` for all three of our checkpoints.** The `-VLM` suffix identifies an autoregressive checkpoint; it does not select the evaluator's generic `VLM` mode. The DLM's entropy-guided or self-speculative algorithm is selected separately through `decoder`.
190
 
191
- External baseline weights and model services are separate from the evaluation toolkit. Metrics remain task-specific: box localization quality, point accuracy, OCR text-and-region matching, GUI grounding accuracy, and counting error. Full protocols and benchmark tables are provided in the paper and supplementary material.
192
 
193
  ## Quantitative Evaluation Benchmarks
194
 
195
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig7-grounding-performance.png" width="100%" alt="GroundAnything Figure 7: GroundAnything and GroundAnything-VLM benchmark overview" /></p>
196
-
197
- *Figure 7. Paper-reported capability overview. Detailed task-level scores, baseline settings, and evaluation protocols are provided in the paper and supplementary material.*
198
 
199
  ## Inference:
200
 
201
  ### Installation
202
 
203
- Run the commands from the **GroundAnything source repository root** after obtaining the code package, using **Linux x86_64 and Python 3.12**. The code link above is reserved for the public release. Use the serving profile in the supplied source bundle so that the custom model adapter, decoding implementation, and dependencies remain aligned.
204
 
205
  ```bash
206
  python3 -m pip install -r requirements.txt huggingface_hub
@@ -264,7 +242,7 @@ point_result = client.predict(
264
  )
265
  ```
266
 
267
- The HTTP client retains no model weights. A generic interactive request has no benchmark task identity; it does not automatically receive the benchmark's task-specific sampling and stopping policy. Use the evaluator below to reproduce the reported protocol.
268
 
269
  ### Supported Tasks & Prompt Templates
270
 
@@ -299,64 +277,17 @@ After a block is complete, a causal forward reconstructs its authoritative KV ca
299
 
300
  The model uses its own shared weights to draft tokens with bidirectional attention and verify them with causal attention. Verification accepts the **longest consecutive matching prefix**, stops at the first mismatch, applies the causal correction, and discards the rejected suffix cache states. The shipped speculative route uses greedy verification; it is not a general stochastic speculative sampler.
301
 
302
- <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig6-self-speculative-decoding.png" width="100%" alt="GroundAnything Figure 6: linear and quadratic self-speculative schedules with shared model weights" /></p>
303
-
304
- *Figure 6. The paper studies two schedules: linear drafting and verification use two model forwards and 2B query tokens per round; quadratic fusion uses one forward after initialization with B(B + 1) query tokens. These counts describe queries and model calls, not total Transformer FLOPs.*
305
 
306
  The documented `--decoder speculative` service is the linear shared-weight route. Exact greedy verification is relative to the converted model's causal branch; it does not imply identical outputs to the separately trained GroundAnything-VLM checkpoint.
307
 
308
- ### Benchmark Sampling Policy
309
-
310
- Entropy-guided evaluation applies the five task profiles from `infer/decode/configs/task_profiles.json`:
311
-
312
- | Task profile | Recipes | Temperature | Top-p | Output-token budget |
313
- |:---|---:|---:|---:|---:|
314
- | Strict single target | 14 | 0 | 1 | 512 |
315
- | Medium, non-OCR | 12 | 0.1 | 0.95 | 4,096 |
316
- | Dense, non-OCR | 8 | 0.1 | 0.95 | 8,192 |
317
- | Medium OCR | 4 | 0.3 | 0.95 | 4,096 |
318
- | Dense OCR | 4 | 0 | 1 | 4,096 |
319
-
320
- The evaluator sends outer sampling parameters and the nested `custom_params.gam_dlm_decode` contract, adds the strict-single-target policy when required, and checks the server's actual decoder before sending requests. Unknown task profiles and mismatched decoders fail early. The entropy-guided benchmark policy rejects a global `max_tokens` override because it would replace the task budgets.
321
-
322
- Self-speculative evaluation uses greedy sampling (`temperature=0`, `top_p=1`, repetition penalty 1). GroundAnything-VLM retains its own autoregressive task recipes. All of these routes use **GAM** prompts and spatial-token parsing.
323
-
324
- ### Run Evaluation
325
-
326
- ```bash
327
- python3 run.py setup eval
328
-
329
- # Match the command to the service that is already running.
330
- python3 run.py eval --decoder denoise
331
- python3 run.py eval --decoder speculative
332
- python3 run.py eval --config configs/eval/vlm.yaml
333
- ```
334
-
335
- Choose one evaluation command for the intended checkpoint/decoder:
336
-
337
- | Checkpoint / decoder | Recipe | Mode | Endpoint model ID |
338
- |:---|:---|:---|:---|
339
- | GroundAnything / entropy-guided | `configs/eval/dlm.yaml` | **GAM** | `groundinganything` |
340
- | GroundAnything / self-speculative | `configs/eval/dlm_speculative.yaml` | **GAM** | `groundinganything` |
341
- | GroundAnything-VLM / autoregressive | `configs/eval/vlm.yaml` | **GAM** | `groundinganything-vlm` |
342
-
343
- Use `service_contract: openai` and the corresponding port (8101 for DLM, 8102 for VLM). For a preflight that checks configured inputs without sending inference requests:
344
-
345
- ```bash
346
- .venv-eval/bin/python scripts/evaluate.py configs/eval/dlm.yaml --dry-run
347
- ```
348
-
349
- Configure the running service, `data_root`, dataset registry, task list, and a fresh `run_id` before launching. The default `limit: 8` is a smoke test; set **`limit: null`** for full evaluation. Evaluation connects to an existing service and does not start or change the decoder.
350
-
351
- Each run saves `run.json` (configuration and provenance), `responses.jsonl` (raw responses, finish reasons, and token usage), task logs, and `summary.json` (metrics and completion state). Record the checkpoint revision and task configuration when comparing results.
352
-
353
  ## Inference Infrastructure
354
 
355
  ### SGLang Execution
356
 
357
  The custom SGLang integration coordinates model loading, request scheduling, attention kernels, and KV-cache ownership for blockwise generation. Denoising uses bidirectional attention inside the active block; completed history is retained as causal KV. Self-speculation additionally verifies proposals and removes rejected suffix states. These cache semantics must be preserved when optimizing execution.
358
 
359
- The supplied recipe selects **BF16 + Triton attention + eager execution**. DLM recipes record the engine source, decoder, effective settings, profile digest, and package versions in `outputs/sglang/<decoder>/engine_runtime.json`; DLM evaluation checks decoder alignment before starting workers. The VLM service records its causal model/runtime configuration separately in `outputs/sglang/vlm/engine_runtime.json`. For higher service concurrency, use independent replicas with distinct devices, ports, and output directories; the default queue is not continuous multi-request batching.
360
 
361
  ### CUDA Graph and Selective FP8
362
 
 
1
  ---
2
  license: other
3
+ license_name: apache-2.0-with-upstream-kimi-k3
4
+ license_link: LICENSE
5
  language:
6
  - en
7
  - zh
 
18
  inference: false
19
  ---
20
 
21
+ # <img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/logo.png" width="40" style="display: inline-block; vertical-align: middle;" alt="GroundAnything logo" /> GroundAnything-VLM: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
 
 
22
 
23
  **Model family:** [GroundAnything — DLM / parallel decoding](https://huggingface.co/GroundingPI/GroundAnything) · [GroundAnything-VLM — autoregressive](https://huggingface.co/GroundingPI/GroundAnything-VLM).
24
 
25
  **This repository contains the autoregressive GroundAnything-VLM checkpoint.** The family overview and figures below are shared with the DLM page; use the VLM serving recipe for these weights.
26
 
27
+ <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig1-teaser.png" width="100%" alt="GroundAnything: broad visual grounding and parallel visual evidence extraction" /></p>
28
 
29
  ## 🔗 Quick Links
30
 
31
  - 🚀 **Online Demo:** Coming soon — XXX.
32
+ - 💻 **GitHub Code:** [groundingpi/GroundAnything](https://github.com/groundingpi/GroundAnything).
33
  - 📄 **Paper:** [arXiv:2609.39600](https://arxiv.org/abs/2609.39600).
34
  - 🧪 **Evaluation data:** Coming soon — XXX.
35
 
 
47
 
48
  <video controls playsinline preload="none" width="100%" poster="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/demo-poster.jpg" src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/demo.mp4"></video>
49
 
50
+ **Parallel Decoding**
51
 
52
  <video controls playsinline preload="none" width="100%" src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/decoding.mp4"></video>
53
 
54
  ### License/Terms of Use:
55
 
56
+ Our original contributions are available under **Apache 2.0**, with no additional restrictions imposed by this project. Third-party material retains its applicable licenses, including the **Kimi K3 License** for Kimi-derived material and applicable derivative works. Its conditions continue to apply when using or redistributing the combined model package. See [LICENSE](LICENSE) for the scope and full license texts.
57
 
58
  ### Deployment Geography:
59
 
 
103
  - **Spatial vocabulary:** 1,000 coordinate tokens shared with semantic labels and protocol markers.
104
  - **DLM conversion:** the shared decoder and vocabulary head support both causal prediction and bidirectional response-block denoising. A mask token is added for diffusion generation.
105
 
106
+ <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig2-architecture.png" width="100%" alt="GroundAnything: vision-language architecture and autoregressive-to-diffusion conversion" /></p>
 
 
 
 
 
 
107
 
108
  ## Input(s):
109
 
 
150
  | [GroundAnything](https://huggingface.co/GroundingPI/GroundAnything) | Entropy-guided blockwise diffusion; optional self-speculation | **GAM** |
151
  | [GroundAnything-VLM](https://huggingface.co/GroundingPI/GroundAnything-VLM) | Autoregressive generation | **GAM** |
152
 
153
+ GroundAnything's main results use **entropy-guided decoding**; GroundAnything-VLM uses **autoregressive decoding**. Self-speculative decoding is an optional acceleration mode.
 
 
 
 
 
 
 
 
154
 
155
+ ## Evaluation
156
 
157
+ The evaluation toolkit supports **7 modes**:
158
 
159
+ | Mode | Supported models |
160
+ |:---|:---|
161
+ | **`GAM`** | **GroundingPI, GroundAnything, GroundAnything-VLM** |
162
+ | `VLM` | Generic vision-language baselines |
163
+ | `REXOMNI` | Rex-Omni |
164
+ | `LOCATEANYTHING` | LocateAnything |
165
+ | `GROUNDINGDINO` | GroundingDINO through a compatible service |
166
+ | `DLM` | Legacy diffusion checkpoints using the GAM protocol |
167
+ | `RLV2` | Legacy RL checkpoints using the GAM protocol |
 
 
 
 
168
 
169
+ **All three released checkpoints use GAM mode.**
170
 
171
+ Evaluation code and instructions: [GitHub](https://github.com/groundingpi/GroundAnything).
172
 
173
  ## Quantitative Evaluation Benchmarks
174
 
175
+ <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig7-grounding-performance.png" width="100%" alt="GroundAnything: GroundAnything and GroundAnything-VLM benchmark overview" /></p>
 
 
176
 
177
  ## Inference:
178
 
179
  ### Installation
180
 
181
+ Run the commands from the **GroundAnything source repository root** after obtaining the code package, using **Linux x86_64 and Python 3.12**. Use the serving profile in the supplied source bundle so that the custom model adapter, decoding implementation, and dependencies remain aligned.
182
 
183
  ```bash
184
  python3 -m pip install -r requirements.txt huggingface_hub
 
242
  )
243
  ```
244
 
245
+ The HTTP client retains no model weights.
246
 
247
  ### Supported Tasks & Prompt Templates
248
 
 
277
 
278
  The model uses its own shared weights to draft tokens with bidirectional attention and verify them with causal attention. Verification accepts the **longest consecutive matching prefix**, stops at the first mismatch, applies the causal correction, and discards the rejected suffix cache states. The shipped speculative route uses greedy verification; it is not a general stochastic speculative sampler.
279
 
280
+ <p align="center"><img src="https://huggingface.co/GroundingPI/GroundAnything-VLM/resolve/c367acf62cc8255782e3d19bd8c312f9b7ab35db/assets/fig6-self-speculative-decoding.png" width="100%" alt="GroundAnything: linear and quadratic self-speculative schedules with shared model weights" /></p>
 
 
281
 
282
  The documented `--decoder speculative` service is the linear shared-weight route. Exact greedy verification is relative to the converted model's causal branch; it does not imply identical outputs to the separately trained GroundAnything-VLM checkpoint.
283
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
284
  ## Inference Infrastructure
285
 
286
  ### SGLang Execution
287
 
288
  The custom SGLang integration coordinates model loading, request scheduling, attention kernels, and KV-cache ownership for blockwise generation. Denoising uses bidirectional attention inside the active block; completed history is retained as causal KV. Self-speculation additionally verifies proposals and removes rejected suffix states. These cache semantics must be preserved when optimizing execution.
289
 
290
+ The supplied recipe selects **BF16 + Triton attention + eager execution**. DLM recipes record the engine source, decoder, effective settings, and package versions in `outputs/sglang/<decoder>/engine_runtime.json`. The VLM service records its causal model/runtime configuration separately in `outputs/sglang/vlm/engine_runtime.json`. For higher service concurrency, use independent replicas with distinct devices, ports, and output directories; the default queue is not continuous multi-request batching.
291
 
292
  ### CUDA Graph and Selective FP8
293
 
checksums.sha256 CHANGED
@@ -1,5 +1,5 @@
1
- 20c797ce19af0c17de52c6afb144644768a591c521655f5ebf5712c9850f2887 LICENSE
2
- 94d3ab6ada08396e953ebd1bf160575dd098bb01d924bfac0b7de5601b369cc2 README.md
3
  7316e325dde0dd407bcc80ed5d3d080223e54572a4685b2aa1ee52041e41d466 added_tokens.json
4
  a0bc6f6fc7a29a80017a433e8f03a1cc1236e838a944a2d034295a60c4f2fddb chat_template.jinja
5
  2a23a7cf76128488c8866582bab06ffb1db387f7f7de82e06fafea65514ce10d config.json
@@ -22,3 +22,5 @@ a839e10631620ae5fe70ec3aa9d009af521f7ea03faa66b32adce287105003be modeling_groun
22
  1a45e3949a652ba4fa8e65017b134404adee1146e1c61fa05e55cdcfa1279e01 tokenizer.json
23
  d9089e4621f850e727a86c64110730d10feb459a9532e8bd8b77c8303d417bd3 tokenizer_config.json
24
  ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910 vocab.json
 
 
 
1
+ c419c191a33774f4e1f834b3fb36033dc84e8ba316a3d1935eed1fa529884a74 LICENSE
2
+ 1fc64576af2dec8f1eb4fe61170843b2fec0c0713157c7493920761bc945c266 README.md
3
  7316e325dde0dd407bcc80ed5d3d080223e54572a4685b2aa1ee52041e41d466 added_tokens.json
4
  a0bc6f6fc7a29a80017a433e8f03a1cc1236e838a944a2d034295a60c4f2fddb chat_template.jinja
5
  2a23a7cf76128488c8866582bab06ffb1db387f7f7de82e06fafea65514ce10d config.json
 
22
  1a45e3949a652ba4fa8e65017b134404adee1146e1c61fa05e55cdcfa1279e01 tokenizer.json
23
  d9089e4621f850e727a86c64110730d10feb459a9532e8bd8b77c8303d417bd3 tokenizer_config.json
24
  ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910 vocab.json
25
+ cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE-Apache-2.0
26
+ 20c797ce19af0c17de52c6afb144644768a591c521655f5ebf5712c9850f2887 LICENSE-Kimi-K3