Image Classification
Transformers
Safetensors
qwen3_5
feature-extraction
clef
bitsandbytes
quantized
8-bit precision
Instructions to use Aikimi/clef-flash-int8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Aikimi/clef-flash-int8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="Aikimi/clef-flash-int8") pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("Aikimi/clef-flash-int8") model = AutoModel.from_pretrained("Aikimi/clef-flash-int8", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add files using upload-large-folder tool
Browse files- .gitattributes +1 -0
- LICENSE +202 -0
- README.md +76 -0
- chat_template.jinja +154 -0
- clef_runtime/__init__.py +1 -0
- clef_runtime/bundle.py +78 -0
- clef_runtime/cache.py +39 -0
- clef_runtime/core.py +339 -0
- clef_runtime/runtime.py +203 -0
- complete.json +132 -0
- config.json +128 -0
- inference.py +41 -0
- joint_head.safetensors +3 -0
- joint_head_config.json +8 -0
- joint_schema_model.py +576 -0
- lm-head.safetensors +3 -0
- model-00001-of-00005.safetensors +3 -0
- model-00002-of-00005.safetensors +3 -0
- model-00003-of-00005.safetensors +3 -0
- model-00004-of-00005.safetensors +3 -0
- model-00005-of-00005.safetensors +3 -0
- model.safetensors.index.json +0 -0
- processor_config.json +60 -0
- requirements.txt +8 -0
- tokenizer.json +3 -0
- tokenizer_config.json +30 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
Apache License
|
| 3 |
+
Version 2.0, January 2004
|
| 4 |
+
http://www.apache.org/licenses/
|
| 5 |
+
|
| 6 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 7 |
+
|
| 8 |
+
1. Definitions.
|
| 9 |
+
|
| 10 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 11 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 12 |
+
|
| 13 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 14 |
+
the copyright owner that is granting the License.
|
| 15 |
+
|
| 16 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 17 |
+
other entities that control, are controlled by, or are under common
|
| 18 |
+
control with that entity. For the purposes of this definition,
|
| 19 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 20 |
+
direction or management of such entity, whether by contract or
|
| 21 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 22 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 23 |
+
|
| 24 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 25 |
+
exercising permissions granted by this License.
|
| 26 |
+
|
| 27 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 28 |
+
including but not limited to software source code, documentation
|
| 29 |
+
source, and configuration files.
|
| 30 |
+
|
| 31 |
+
"Object" form shall mean any form resulting from mechanical
|
| 32 |
+
transformation or translation of a Source form, including but
|
| 33 |
+
not limited to compiled object code, generated documentation,
|
| 34 |
+
and conversions to other media types.
|
| 35 |
+
|
| 36 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 37 |
+
Object form, made available under the License, as indicated by a
|
| 38 |
+
copyright notice that is included in or attached to the work
|
| 39 |
+
(an example is provided in the Appendix below).
|
| 40 |
+
|
| 41 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 42 |
+
form, that is based on (or derived from) the Work and for which the
|
| 43 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 44 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 45 |
+
of this License, Derivative Works shall not include works that remain
|
| 46 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 47 |
+
the Work and Derivative Works thereof.
|
| 48 |
+
|
| 49 |
+
"Contribution" shall mean any work of authorship, including
|
| 50 |
+
the original version of the Work and any modifications or additions
|
| 51 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 52 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 53 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 54 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 55 |
+
means any form of electronic, verbal, or written communication sent
|
| 56 |
+
to the Licensor or its representatives, including but not limited to
|
| 57 |
+
communication on electronic mailing lists, source code control systems,
|
| 58 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 59 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 60 |
+
excluding communication that is conspicuously marked or otherwise
|
| 61 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 62 |
+
|
| 63 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 64 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 65 |
+
subsequently incorporated within the Work.
|
| 66 |
+
|
| 67 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 68 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 69 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 70 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 71 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 72 |
+
Work and such Derivative Works in Source or Object form.
|
| 73 |
+
|
| 74 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 75 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 76 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 77 |
+
(except as stated in this section) patent license to make, have made,
|
| 78 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 79 |
+
where such license applies only to those patent claims licensable
|
| 80 |
+
by such Contributor that are necessarily infringed by their
|
| 81 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 82 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 83 |
+
institute patent litigation against any entity (including a
|
| 84 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 85 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 86 |
+
or contributory patent infringement, then any patent licenses
|
| 87 |
+
granted to You under this License for that Work shall terminate
|
| 88 |
+
as of the date such litigation is filed.
|
| 89 |
+
|
| 90 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 91 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 92 |
+
modifications, and in Source or Object form, provided that You
|
| 93 |
+
meet the following conditions:
|
| 94 |
+
|
| 95 |
+
(a) You must give any other recipients of the Work or
|
| 96 |
+
Derivative Works a copy of this License; and
|
| 97 |
+
|
| 98 |
+
(b) You must cause any modified files to carry prominent notices
|
| 99 |
+
stating that You changed the files; and
|
| 100 |
+
|
| 101 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 102 |
+
that You distribute, all copyright, patent, trademark, and
|
| 103 |
+
attribution notices from the Source form of the Work,
|
| 104 |
+
excluding those notices that do not pertain to any part of
|
| 105 |
+
the Derivative Works; and
|
| 106 |
+
|
| 107 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 108 |
+
distribution, then any Derivative Works that You distribute must
|
| 109 |
+
include a readable copy of the attribution notices contained
|
| 110 |
+
within such NOTICE file, excluding those notices that do not
|
| 111 |
+
pertain to any part of the Derivative Works, in at least one
|
| 112 |
+
of the following places: within a NOTICE text file distributed
|
| 113 |
+
as part of the Derivative Works; within the Source form or
|
| 114 |
+
documentation, if provided along with the Derivative Works; or,
|
| 115 |
+
within a display generated by the Derivative Works, if and
|
| 116 |
+
wherever such third-party notices normally appear. The contents
|
| 117 |
+
of the NOTICE file are for informational purposes only and
|
| 118 |
+
do not modify the License. You may add Your own attribution
|
| 119 |
+
notices within Derivative Works that You distribute, alongside
|
| 120 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 121 |
+
that such additional attribution notices cannot be construed
|
| 122 |
+
as modifying the License.
|
| 123 |
+
|
| 124 |
+
You may add Your own copyright statement to Your modifications and
|
| 125 |
+
may provide additional or different license terms and conditions
|
| 126 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 127 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 128 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 129 |
+
the conditions stated in this License.
|
| 130 |
+
|
| 131 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 132 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 133 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 134 |
+
this License, without any additional terms or conditions.
|
| 135 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 136 |
+
the terms of any separate license agreement you may have executed
|
| 137 |
+
with Licensor regarding such Contributions.
|
| 138 |
+
|
| 139 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 140 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 141 |
+
except as required for reasonable and customary use in describing the
|
| 142 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 143 |
+
|
| 144 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 145 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 146 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 147 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 148 |
+
implied, including, without limitation, any warranties or conditions
|
| 149 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 150 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 151 |
+
appropriateness of using or redistributing the Work and assume any
|
| 152 |
+
risks associated with Your exercise of permissions under this License.
|
| 153 |
+
|
| 154 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 155 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 156 |
+
unless required by applicable law (such as deliberate and grossly
|
| 157 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 158 |
+
liable to You for damages, including any direct, indirect, special,
|
| 159 |
+
incidental, or consequential damages of any character arising as a
|
| 160 |
+
result of this License or out of the use or inability to use the
|
| 161 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 162 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 163 |
+
other commercial damages or losses), even if such Contributor
|
| 164 |
+
has been advised of the possibility of such damages.
|
| 165 |
+
|
| 166 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 167 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 168 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 169 |
+
or other liability obligations and/or rights consistent with this
|
| 170 |
+
License. However, in accepting such obligations, You may act only
|
| 171 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 172 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 173 |
+
defend, and hold each Contributor harmless for any liability
|
| 174 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 175 |
+
of your accepting any such warranty or additional liability.
|
| 176 |
+
|
| 177 |
+
END OF TERMS AND CONDITIONS
|
| 178 |
+
|
| 179 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 180 |
+
|
| 181 |
+
To apply the Apache License to your work, attach the following
|
| 182 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 183 |
+
replaced with your own identifying information. (Don't include
|
| 184 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 185 |
+
comment syntax for the file format. We also recommend that a
|
| 186 |
+
file or class name and description of purpose be included on the
|
| 187 |
+
same "printed page" as the copyright notice for easier
|
| 188 |
+
identification within third-party archives.
|
| 189 |
+
|
| 190 |
+
Copyright 2026 Alibaba Cloud
|
| 191 |
+
|
| 192 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 193 |
+
you may not use this file except in compliance with the License.
|
| 194 |
+
You may obtain a copy of the License at
|
| 195 |
+
|
| 196 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 197 |
+
|
| 198 |
+
Unless required by applicable law or agreed to in writing, software
|
| 199 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 200 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 201 |
+
See the License for the specific language governing permissions and
|
| 202 |
+
limitations under the License.
|
README.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Cloudflare/clef-flash
|
| 4 |
+
base_model_relation: quantized
|
| 5 |
+
library_name: transformers
|
| 6 |
+
tags:
|
| 7 |
+
- clef
|
| 8 |
+
- bitsandbytes
|
| 9 |
+
- image-classification
|
| 10 |
+
- quantized
|
| 11 |
+
---
|
| 12 |
+
# Clef-Flash 9B INT8
|
| 13 |
+
|
| 14 |
+
Quantized from [Cloudflare/clef-flash](https://huggingface.co/Cloudflare/clef-flash/tree/17f0b0ad64efb65d273590632833508766b2aae6)
|
| 15 |
+
at `17f0b0ad64efb65d273590632833508766b2aae6`. The original model, schema encoder and joint head are
|
| 16 |
+
by Cloudflare. This derived release packages INT8 decoder weights with the
|
| 17 |
+
original BF16 vision encoder, joint head and dense input/output vocabulary.
|
| 18 |
+
It includes every inference asset; the original BF16 checkpoint is not required.
|
| 19 |
+
The output vocabulary is in `lm-head.safetensors` and indexed separately.
|
| 20 |
+
|
| 21 |
+
Clef performs a single forward pass over a schema of choice, score and noul
|
| 22 |
+
(boolean) questions. It does not generate free-form text. Probabilities are
|
| 23 |
+
distributions over the supplied alternatives, not calibrated accuracy estimates.
|
| 24 |
+
|
| 25 |
+
## Usage
|
| 26 |
+
|
| 27 |
+
Use Forge Neo's Clef setup and GUI, or this repository's standalone CLI:
|
| 28 |
+
|
| 29 |
+
```powershell
|
| 30 |
+
python -m pip install torch==2.13.0 torchvision==0.28.0 --index-url https://download.pytorch.org/whl/cu130
|
| 31 |
+
python -m pip install -r requirements.txt
|
| 32 |
+
python inference.py --profile flash-int8 --image example.png
|
| 33 |
+
python inference.py --profile flash-int8 --image example.png --questions questions.json
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
`questions.json` is the original Clef schema: each question ID maps to a `type`,
|
| 37 |
+
`instructions`, and, for choice/score, `criteria`. Use `--state` for context,
|
| 38 |
+
`--max-pixels` for image processing (default 262144), and `--max-length` for the
|
| 39 |
+
total input token cap (default 2048). Inputs exceeding the cap are refused.
|
| 40 |
+
An NVIDIA CUDA GPU with BF16 support is required. The standalone loader reads
|
| 41 |
+
weights locally and never downloads code during inference.
|
| 42 |
+
|
| 43 |
+
The 27B release is shared by `clef-24gb` and `clef-16gb`. The latter keeps the
|
| 44 |
+
input embedding on CPU. Both keep the dense output vocabulary on CPU and move
|
| 45 |
+
only requested rows to CUDA; decoder layers remain on CUDA. The 9B INT8 release
|
| 46 |
+
uses `flash-int8`. Packed lm_head replacement and whole-layer CPU offload are
|
| 47 |
+
not part of these profiles. The supplied loader is required for this placement.
|
| 48 |
+
|
| 49 |
+
## Validation and limits
|
| 50 |
+
|
| 51 |
+
Verified on Windows, RTX 3090 24GB, RAM 64GB, with Transformers 5.10.2,
|
| 52 |
+
bitsandbytes 0.50.2, safetensors 0.8.0 and PyTorch 2.13.0+cu130. We use SDPA,
|
| 53 |
+
batch=1, no KV cache, and safetensors `pread` to avoid persistent Windows
|
| 54 |
+
copy-on-write mappings of all shards.
|
| 55 |
+
|
| 56 |
+
Four images and a text-only request were checked with the same schemas.
|
| 57 |
+
27B 16GB and 24GB profiles produced identical answers and probabilities.
|
| 58 |
+
Peak image allocations were 13.38 GiB and 15.73 GiB respectively. The 16GB
|
| 59 |
+
profile passed with a 15 GiB PyTorch allocator budget on the 3090; an actual
|
| 60 |
+
16GB card was not available. The 9B INT8 peak was 9.82 GiB versus 16.26 GiB
|
| 61 |
+
for BF16, with no choice changes in those four images and a maximum probability
|
| 62 |
+
difference of 0.0423. This small comparison is not a general accuracy evaluation.
|
| 63 |
+
INT8 was slower than BF16 on this test; its benefit here is lower memory use.
|
| 64 |
+
Longer inputs or more image pixels require more memory.
|
| 65 |
+
|
| 66 |
+
## Provenance and license
|
| 67 |
+
|
| 68 |
+
Apache-2.0; see `LICENSE` and the upstream model card. The original
|
| 69 |
+
`joint_schema_model.py`, head, vocabulary values and processor assets are
|
| 70 |
+
retained. Decoder linear layers were converted with bitsandbytes and saved
|
| 71 |
+
using Transformers. Vision and the judgment head remain BF16. The included
|
| 72 |
+
`clef_runtime` and `inference.py` add CPU row gathering, profile selection,
|
| 73 |
+
input validation and Windows `pread` loading. `complete.json` records the
|
| 74 |
+
upstream commit, quantization settings, file sizes and SHA-256 hashes.
|
| 75 |
+
No user images, evaluation histories, local configuration or credentials are
|
| 76 |
+
included in this model release.
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set image_count = namespace(value=0) %}
|
| 2 |
+
{%- set video_count = namespace(value=0) %}
|
| 3 |
+
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 4 |
+
{%- if content is string %}
|
| 5 |
+
{{- content }}
|
| 6 |
+
{%- elif content is iterable and content is not mapping %}
|
| 7 |
+
{%- for item in content %}
|
| 8 |
+
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
|
| 9 |
+
{%- if is_system_content %}
|
| 10 |
+
{{- raise_exception('System message cannot contain images.') }}
|
| 11 |
+
{%- endif %}
|
| 12 |
+
{%- if do_vision_count %}
|
| 13 |
+
{%- set image_count.value = image_count.value + 1 %}
|
| 14 |
+
{%- endif %}
|
| 15 |
+
{%- if add_vision_id %}
|
| 16 |
+
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 19 |
+
{%- elif 'video' in item or item.type == 'video' %}
|
| 20 |
+
{%- if is_system_content %}
|
| 21 |
+
{{- raise_exception('System message cannot contain videos.') }}
|
| 22 |
+
{%- endif %}
|
| 23 |
+
{%- if do_vision_count %}
|
| 24 |
+
{%- set video_count.value = video_count.value + 1 %}
|
| 25 |
+
{%- endif %}
|
| 26 |
+
{%- if add_vision_id %}
|
| 27 |
+
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 28 |
+
{%- endif %}
|
| 29 |
+
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 30 |
+
{%- elif 'text' in item %}
|
| 31 |
+
{{- item.text }}
|
| 32 |
+
{%- else %}
|
| 33 |
+
{{- raise_exception('Unexpected item type in content.') }}
|
| 34 |
+
{%- endif %}
|
| 35 |
+
{%- endfor %}
|
| 36 |
+
{%- elif content is none or content is undefined %}
|
| 37 |
+
{{- '' }}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{{- raise_exception('Unexpected content type.') }}
|
| 40 |
+
{%- endif %}
|
| 41 |
+
{%- endmacro %}
|
| 42 |
+
{%- if not messages %}
|
| 43 |
+
{{- raise_exception('No messages provided.') }}
|
| 44 |
+
{%- endif %}
|
| 45 |
+
{%- if tools and tools is iterable and tools is not mapping %}
|
| 46 |
+
{{- '<|im_start|>system\n' }}
|
| 47 |
+
{{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
|
| 48 |
+
{%- for tool in tools %}
|
| 49 |
+
{{- "\n" }}
|
| 50 |
+
{{- tool | tojson }}
|
| 51 |
+
{%- endfor %}
|
| 52 |
+
{{- "\n</tools>" }}
|
| 53 |
+
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
|
| 54 |
+
{%- if messages[0].role == 'system' %}
|
| 55 |
+
{%- set content = render_content(messages[0].content, false, true)|trim %}
|
| 56 |
+
{%- if content %}
|
| 57 |
+
{{- '\n\n' + content }}
|
| 58 |
+
{%- endif %}
|
| 59 |
+
{%- endif %}
|
| 60 |
+
{{- '<|im_end|>\n' }}
|
| 61 |
+
{%- else %}
|
| 62 |
+
{%- if messages[0].role == 'system' %}
|
| 63 |
+
{%- set content = render_content(messages[0].content, false, true)|trim %}
|
| 64 |
+
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 65 |
+
{%- endif %}
|
| 66 |
+
{%- endif %}
|
| 67 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 68 |
+
{%- for message in messages[::-1] %}
|
| 69 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 70 |
+
{%- if ns.multi_step_tool and message.role == "user" %}
|
| 71 |
+
{%- set content = render_content(message.content, false)|trim %}
|
| 72 |
+
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
|
| 73 |
+
{%- set ns.multi_step_tool = false %}
|
| 74 |
+
{%- set ns.last_query_index = index %}
|
| 75 |
+
{%- endif %}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{%- endfor %}
|
| 78 |
+
{%- if ns.multi_step_tool %}
|
| 79 |
+
{{- raise_exception('No user query found in messages.') }}
|
| 80 |
+
{%- endif %}
|
| 81 |
+
{%- for message in messages %}
|
| 82 |
+
{%- set content = render_content(message.content, true)|trim %}
|
| 83 |
+
{%- if message.role == "system" %}
|
| 84 |
+
{%- if not loop.first %}
|
| 85 |
+
{{- raise_exception('System message must be at the beginning.') }}
|
| 86 |
+
{%- endif %}
|
| 87 |
+
{%- elif message.role == "user" %}
|
| 88 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 89 |
+
{%- elif message.role == "assistant" %}
|
| 90 |
+
{%- set reasoning_content = '' %}
|
| 91 |
+
{%- if message.reasoning_content is string %}
|
| 92 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 93 |
+
{%- else %}
|
| 94 |
+
{%- if '</think>' in content %}
|
| 95 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 96 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 97 |
+
{%- endif %}
|
| 98 |
+
{%- endif %}
|
| 99 |
+
{%- set reasoning_content = reasoning_content|trim %}
|
| 100 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 101 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 102 |
+
{%- else %}
|
| 103 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 104 |
+
{%- endif %}
|
| 105 |
+
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 106 |
+
{%- for tool_call in message.tool_calls %}
|
| 107 |
+
{%- if tool_call.function is defined %}
|
| 108 |
+
{%- set tool_call = tool_call.function %}
|
| 109 |
+
{%- endif %}
|
| 110 |
+
{%- if loop.first %}
|
| 111 |
+
{%- if content|trim %}
|
| 112 |
+
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
|
| 113 |
+
{%- else %}
|
| 114 |
+
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
|
| 115 |
+
{%- endif %}
|
| 116 |
+
{%- else %}
|
| 117 |
+
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
|
| 118 |
+
{%- endif %}
|
| 119 |
+
{%- if tool_call.arguments is defined %}
|
| 120 |
+
{%- for args_name, args_value in tool_call.arguments|items %}
|
| 121 |
+
{{- '<parameter=' + args_name + '>\n' }}
|
| 122 |
+
{%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
|
| 123 |
+
{{- args_value }}
|
| 124 |
+
{{- '\n</parameter>\n' }}
|
| 125 |
+
{%- endfor %}
|
| 126 |
+
{%- endif %}
|
| 127 |
+
{{- '</function>\n</tool_call>' }}
|
| 128 |
+
{%- endfor %}
|
| 129 |
+
{%- endif %}
|
| 130 |
+
{{- '<|im_end|>\n' }}
|
| 131 |
+
{%- elif message.role == "tool" %}
|
| 132 |
+
{%- if loop.previtem and loop.previtem.role != "tool" %}
|
| 133 |
+
{{- '<|im_start|>user' }}
|
| 134 |
+
{%- endif %}
|
| 135 |
+
{{- '\n<tool_response>\n' }}
|
| 136 |
+
{{- content }}
|
| 137 |
+
{{- '\n</tool_response>' }}
|
| 138 |
+
{%- if not loop.last and loop.nextitem.role != "tool" %}
|
| 139 |
+
{{- '<|im_end|>\n' }}
|
| 140 |
+
{%- elif loop.last %}
|
| 141 |
+
{{- '<|im_end|>\n' }}
|
| 142 |
+
{%- endif %}
|
| 143 |
+
{%- else %}
|
| 144 |
+
{{- raise_exception('Unexpected message role.') }}
|
| 145 |
+
{%- endif %}
|
| 146 |
+
{%- endfor %}
|
| 147 |
+
{%- if add_generation_prompt %}
|
| 148 |
+
{{- '<|im_start|>assistant\n' }}
|
| 149 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 150 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 151 |
+
{%- else %}
|
| 152 |
+
{{- '<think>\n' }}
|
| 153 |
+
{%- endif %}
|
| 154 |
+
{%- endif %}
|
clef_runtime/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Local Clef decision models and the shared Forge Studio interface."""
|
clef_runtime/bundle.py
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Validate complete, self-contained quantized releases without importing torch."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
from .cache import cache_directory, identity
|
| 9 |
+
from .core import PROFILES, ClefError, sha256, source_manifest
|
| 10 |
+
|
| 11 |
+
SUPPORT_FILES = (
|
| 12 |
+
"joint_head.safetensors",
|
| 13 |
+
"joint_head_config.json",
|
| 14 |
+
"joint_schema_model.py",
|
| 15 |
+
"tokenizer.json",
|
| 16 |
+
"tokenizer_config.json",
|
| 17 |
+
"processor_config.json",
|
| 18 |
+
"chat_template.jinja",
|
| 19 |
+
"LICENSE",
|
| 20 |
+
)
|
| 21 |
+
PORTABLE_MODULES = ("__init__.py", "core.py", "cache.py", "bundle.py", "runtime.py")
|
| 22 |
+
REQUIRED_FILES = {
|
| 23 |
+
*SUPPORT_FILES,
|
| 24 |
+
"config.json",
|
| 25 |
+
"model.safetensors.index.json",
|
| 26 |
+
"lm-head.safetensors",
|
| 27 |
+
"README.md",
|
| 28 |
+
"inference.py",
|
| 29 |
+
"requirements.txt",
|
| 30 |
+
*("clef_runtime/" + name for name in PORTABLE_MODULES),
|
| 31 |
+
}
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def bundle_directory(root, profile):
|
| 35 |
+
converted = cache_directory(root, profile)
|
| 36 |
+
return converted.with_name(converted.name + "-bundle") if converted is not None else None
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def read_bundle(directory, profile, *, verify_hashes=False):
|
| 40 |
+
directory = Path(directory)
|
| 41 |
+
try:
|
| 42 |
+
manifest = json.loads((directory / "complete.json").read_text(encoding="utf-8"))
|
| 43 |
+
expected = {**identity(profile), "bundle_format": 1}
|
| 44 |
+
if any(manifest.get(key) != value for key, value in expected.items()):
|
| 45 |
+
raise ClefError("Clef量子化配布の版・精度が一致しません。セットアップを再実行してください。")
|
| 46 |
+
files = manifest["files"]
|
| 47 |
+
names = {item["path"] for item in files}
|
| 48 |
+
if not REQUIRED_FILES <= names or len(names) != len(files):
|
| 49 |
+
raise ClefError("Clef量子化配布の構成が不足しています。セットアップを再実行してください。")
|
| 50 |
+
for item in files:
|
| 51 |
+
path = directory / item["path"]
|
| 52 |
+
if (
|
| 53 |
+
not path.resolve().is_relative_to(directory.resolve())
|
| 54 |
+
or not path.is_file()
|
| 55 |
+
or path.stat().st_size != item["size"]
|
| 56 |
+
):
|
| 57 |
+
raise ClefError(f"Clef量子化配布が不足・変更されています: {item['path']}")
|
| 58 |
+
if (verify_hashes or path.suffix == ".py") and sha256(path) != item["sha256"]:
|
| 59 |
+
raise ClefError(f"ClefのSHA-256が一致しません: {item['path']}")
|
| 60 |
+
index = json.loads((directory / "model.safetensors.index.json").read_text(encoding="utf-8"))["weight_map"]
|
| 61 |
+
if (
|
| 62 |
+
index.get("lm_head.weight") != "lm-head.safetensors"
|
| 63 |
+
or "language_model.embed_tokens.weight" not in index
|
| 64 |
+
or not set(index.values()) <= names
|
| 65 |
+
):
|
| 66 |
+
raise ClefError("Clef量子化配布の語彙・本体の構成が一致しません。")
|
| 67 |
+
return directory, manifest
|
| 68 |
+
except ClefError:
|
| 69 |
+
raise
|
| 70 |
+
except (OSError, ValueError, KeyError, TypeError) as exc:
|
| 71 |
+
raise ClefError("Clef量子化モデル未導入です。aikimi-clef-setup.batを実行してください。") from exc
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def bundle_manifest(root, profile, *, verify_hashes=False):
|
| 75 |
+
directory = bundle_directory(root, profile)
|
| 76 |
+
if directory is None:
|
| 77 |
+
return source_manifest(root, PROFILES[profile]["model"], verify_hashes=verify_hashes)
|
| 78 |
+
return read_bundle(directory, profile, verify_hashes=verify_hashes)
|
clef_runtime/cache.py
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Storage location and fixed conversion provenance shared by release tools."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
from .core import MODELS, PROFILES
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def cache_directory(root, profile):
|
| 12 |
+
settings = PROFILES[profile]
|
| 13 |
+
if settings["precision"] == "bf16":
|
| 14 |
+
return None
|
| 15 |
+
root = Path(root)
|
| 16 |
+
try:
|
| 17 |
+
storage = json.loads((root / "storage.json").read_text(encoding="utf-8"))
|
| 18 |
+
except FileNotFoundError:
|
| 19 |
+
base = root / "quantized"
|
| 20 |
+
except (OSError, ValueError) as exc:
|
| 21 |
+
raise ValueError("Clefのstorage.jsonを読み込めません。--cache-dirで保存先を設定し直してください。") from exc
|
| 22 |
+
else:
|
| 23 |
+
destination = storage.get("quantized") if isinstance(storage, dict) else None
|
| 24 |
+
if not isinstance(destination, str) or not Path(destination).is_absolute():
|
| 25 |
+
raise ValueError("Clefの量子化キャッシュ保存先は絶対パスで指定してください。")
|
| 26 |
+
base = Path(destination)
|
| 27 |
+
return base / f"{settings['model']}-{settings['precision']}"
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def identity(profile):
|
| 31 |
+
settings = PROFILES[profile]
|
| 32 |
+
return {
|
| 33 |
+
**MODELS[settings["model"]],
|
| 34 |
+
"precision": settings["precision"],
|
| 35 |
+
"format": 1,
|
| 36 |
+
"transformers": "5.10.2",
|
| 37 |
+
"bitsandbytes": "0.50.2",
|
| 38 |
+
"vision": "bf16",
|
| 39 |
+
}
|
clef_runtime/core.py
ADDED
|
@@ -0,0 +1,339 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Small, torch-free contracts for images, schemas and immutable decision runs."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import copy
|
| 6 |
+
import csv
|
| 7 |
+
import hashlib
|
| 8 |
+
import io
|
| 9 |
+
import json
|
| 10 |
+
import math
|
| 11 |
+
import os
|
| 12 |
+
from pathlib import Path
|
| 13 |
+
|
| 14 |
+
ROOT = Path(__file__).resolve().parents[2]
|
| 15 |
+
RUNTIME = ROOT / "models" / "Clef"
|
| 16 |
+
OUTPUTS = ROOT / "outputs" / "clef"
|
| 17 |
+
MODELS = {
|
| 18 |
+
"clef-flash": {"repo": "Cloudflare/clef-flash", "revision": "17f0b0ad64efb65d273590632833508766b2aae6"},
|
| 19 |
+
"clef": {"repo": "Cloudflare/clef", "revision": "2f3de3dd85f379784083b0814d997ab627200f0c"},
|
| 20 |
+
}
|
| 21 |
+
PROFILES = {
|
| 22 |
+
"flash-int8": {
|
| 23 |
+
"label": "Flash INT8 · 軽量",
|
| 24 |
+
"model": "clef-flash",
|
| 25 |
+
"precision": "int8",
|
| 26 |
+
"cpu_embeddings": False,
|
| 27 |
+
"min_vram_gib": 12,
|
| 28 |
+
},
|
| 29 |
+
"clef-24gb": {
|
| 30 |
+
"label": "Clef NF4 · 24GB",
|
| 31 |
+
"model": "clef",
|
| 32 |
+
"precision": "nf4",
|
| 33 |
+
"cpu_embeddings": False,
|
| 34 |
+
"min_vram_gib": 20,
|
| 35 |
+
},
|
| 36 |
+
"clef-16gb": {
|
| 37 |
+
"label": "Clef NF4 · 16GB",
|
| 38 |
+
"model": "clef",
|
| 39 |
+
"precision": "nf4",
|
| 40 |
+
"cpu_embeddings": True,
|
| 41 |
+
"min_vram_gib": 14,
|
| 42 |
+
},
|
| 43 |
+
"flash-bf16": {
|
| 44 |
+
"label": "Flash BF16 · 比較用",
|
| 45 |
+
"model": "clef-flash",
|
| 46 |
+
"precision": "bf16",
|
| 47 |
+
"cpu_embeddings": False,
|
| 48 |
+
"min_vram_gib": 20,
|
| 49 |
+
},
|
| 50 |
+
}
|
| 51 |
+
TEMPLATES = {
|
| 52 |
+
"画像の評価": {
|
| 53 |
+
"style": {
|
| 54 |
+
"type": "choice",
|
| 55 |
+
"instructions": "画像の主な表現形式を分類してください。",
|
| 56 |
+
"criteria": {
|
| 57 |
+
"photo": "写真または写実的な画像",
|
| 58 |
+
"illustration": "イラスト・アニメ・絵画",
|
| 59 |
+
"design": "図・ポスター・グラフィックデザイン",
|
| 60 |
+
},
|
| 61 |
+
},
|
| 62 |
+
"quality": {
|
| 63 |
+
"type": "score",
|
| 64 |
+
"instructions": "画像の見た目の完成度を評価してください。",
|
| 65 |
+
"criteria": [
|
| 66 |
+
"大きな破綻や欠落がある",
|
| 67 |
+
"目立つ問題があり修正が必要",
|
| 68 |
+
"おおむね整っている",
|
| 69 |
+
"細部まで整い完成度が高い",
|
| 70 |
+
],
|
| 71 |
+
},
|
| 72 |
+
"text": {"type": "noul", "instructions": "画像に読める文字が含まれている。"},
|
| 73 |
+
},
|
| 74 |
+
"生成画像の確認": {
|
| 75 |
+
"artifact": {"type": "noul", "instructions": "画像に不自然な形、重複、形状の破綻が見られる。"},
|
| 76 |
+
"composition": {
|
| 77 |
+
"type": "score",
|
| 78 |
+
"instructions": "構図の明瞭さと主題の伝わりやすさを評価してください。",
|
| 79 |
+
"criteria": ["主題が不明瞭", "主題は分かるが構図に問題がある", "主題が明瞭で構図が整っている"],
|
| 80 |
+
},
|
| 81 |
+
},
|
| 82 |
+
"文章・JSONの分類": {
|
| 83 |
+
"department": {
|
| 84 |
+
"type": "choice",
|
| 85 |
+
"instructions": "問い合わせを担当するチームはどれですか。",
|
| 86 |
+
"criteria": {"billing": "請求や支払い", "technical": "技術的な問題や障害", "other": "その他"},
|
| 87 |
+
},
|
| 88 |
+
"outage": {"type": "noul", "instructions": "サービスが停止している。"},
|
| 89 |
+
},
|
| 90 |
+
}
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
class ClefError(ValueError):
|
| 94 |
+
"""An input or installation problem to show directly in the Studio."""
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def atomic_json(path, data):
|
| 98 |
+
path = Path(path)
|
| 99 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 100 |
+
temporary = path.with_name(path.name + ".tmp")
|
| 101 |
+
temporary.write_text(json.dumps(data, ensure_ascii=False, indent=2, allow_nan=False), encoding="utf-8")
|
| 102 |
+
os.replace(temporary, path)
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
def sha256(path):
|
| 106 |
+
digest = hashlib.sha256()
|
| 107 |
+
with Path(path).open("rb") as stream:
|
| 108 |
+
for chunk in iter(lambda: stream.read(8 * 1024 * 1024), b""):
|
| 109 |
+
digest.update(chunk)
|
| 110 |
+
return digest.hexdigest()
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
def canonical_hash(value):
|
| 114 |
+
return hashlib.sha256(
|
| 115 |
+
json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False).encode()
|
| 116 |
+
).hexdigest()
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
def _unique(pairs):
|
| 120 |
+
result = {}
|
| 121 |
+
for key, value in pairs:
|
| 122 |
+
if key in result:
|
| 123 |
+
raise ClefError(f"IDが重複しています: {key}")
|
| 124 |
+
result[key] = value
|
| 125 |
+
return result
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def parse_schema(value):
|
| 129 |
+
if isinstance(value, str):
|
| 130 |
+
if len(value) > 100_000:
|
| 131 |
+
raise ClefError("判断項目のJSONは100,000文字以内にしてください。")
|
| 132 |
+
try:
|
| 133 |
+
value = json.loads(value, object_pairs_hook=_unique)
|
| 134 |
+
except json.JSONDecodeError as exc:
|
| 135 |
+
raise ClefError(f"JSONの{exc.lineno}行目・{exc.colno}文字目を確認してください。") from exc
|
| 136 |
+
if not isinstance(value, dict) or not 1 <= len(value) <= 32:
|
| 137 |
+
raise ClefError("判断項目は1〜32件のオブジェクトにしてください。")
|
| 138 |
+
for qid, question in value.items():
|
| 139 |
+
if not isinstance(qid, str) or not qid.strip() or len(qid) > 80:
|
| 140 |
+
raise ClefError("項目IDは1〜80文字にしてください���")
|
| 141 |
+
if not isinstance(question, dict) or question.get("type") not in {"choice", "score", "noul"}:
|
| 142 |
+
raise ClefError(f"{qid}: 種類はchoice / score / noulを指定してください。")
|
| 143 |
+
if set(question) - {"type", "instructions", "criteria"}:
|
| 144 |
+
raise ClefError(f"{qid}: 未対応の設定があります。")
|
| 145 |
+
if not isinstance(question.get("instructions", ""), str):
|
| 146 |
+
raise ClefError(f"{qid}: 判断することは文章で指定してください。")
|
| 147 |
+
criteria, kind = question.get("criteria"), question["type"]
|
| 148 |
+
if kind == "choice":
|
| 149 |
+
if not isinstance(criteria, dict) or not 2 <= len(criteria) <= 64:
|
| 150 |
+
raise ClefError(f"{qid}: 選択肢はIDと説明の組を2〜64件指定してください。")
|
| 151 |
+
if any(
|
| 152 |
+
not isinstance(k, str) or not k.strip() or not isinstance(v, str) or not v.strip()
|
| 153 |
+
for k, v in criteria.items()
|
| 154 |
+
):
|
| 155 |
+
raise ClefError(f"{qid}: 選択肢のIDと説明を入力してください。")
|
| 156 |
+
elif kind == "score":
|
| 157 |
+
if (
|
| 158 |
+
not isinstance(criteria, list)
|
| 159 |
+
or not 2 <= len(criteria) <= 32
|
| 160 |
+
or any(not isinstance(x, str) or not x.strip() for x in criteria)
|
| 161 |
+
):
|
| 162 |
+
raise ClefError(f"{qid}: 段階の説明を低い順に2〜32件指定してください。")
|
| 163 |
+
elif criteria is not None and (
|
| 164 |
+
not isinstance(criteria, dict)
|
| 165 |
+
or set(criteria) - {"true", "false"}
|
| 166 |
+
or any(not isinstance(x, str) for x in criteria.values())
|
| 167 |
+
):
|
| 168 |
+
raise ClefError(f"{qid}: 真偽の説明はtrue / falseで指定してください。")
|
| 169 |
+
return copy.deepcopy(value)
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
def edit_question(schema, previous, qid, kind, instructions, options):
|
| 173 |
+
result = copy.deepcopy(schema)
|
| 174 |
+
qid = (qid or "").strip()
|
| 175 |
+
if qid in result and qid != previous:
|
| 176 |
+
raise ClefError(f"IDが重複しています: {qid}")
|
| 177 |
+
question = {"type": kind, "instructions": (instructions or "").strip()}
|
| 178 |
+
lines = [x.strip() for x in (options or "").splitlines() if x.strip()]
|
| 179 |
+
if kind == "choice":
|
| 180 |
+
pairs = []
|
| 181 |
+
for line in lines:
|
| 182 |
+
if ":" not in line:
|
| 183 |
+
raise ClefError("選択肢は1行に「ID: 説明」の形で指定してください。")
|
| 184 |
+
key, description = line.split(":", 1)
|
| 185 |
+
pairs.append((key.strip(), description.strip()))
|
| 186 |
+
question["criteria"] = _unique(pairs)
|
| 187 |
+
elif kind == "score":
|
| 188 |
+
question["criteria"] = lines
|
| 189 |
+
if previous and previous != qid:
|
| 190 |
+
result.pop(previous, None)
|
| 191 |
+
result[qid] = question
|
| 192 |
+
return parse_schema(result)
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
def integer(value, label, lower, upper):
|
| 196 |
+
try:
|
| 197 |
+
number = int(value)
|
| 198 |
+
if (
|
| 199 |
+
isinstance(value, bool)
|
| 200 |
+
or not math.isfinite(float(value))
|
| 201 |
+
or float(value) != number
|
| 202 |
+
or not lower <= number <= upper
|
| 203 |
+
):
|
| 204 |
+
raise ValueError
|
| 205 |
+
return number
|
| 206 |
+
except (ValueError, TypeError, OverflowError) as exc:
|
| 207 |
+
raise ClefError(f"{label}は{lower}〜{upper}の整数にしてください。") from exc
|
| 208 |
+
|
| 209 |
+
|
| 210 |
+
def validate_request(request):
|
| 211 |
+
profile = request.get("profile")
|
| 212 |
+
if profile not in PROFILES:
|
| 213 |
+
raise ClefError("実行プロファイルを選んでください。")
|
| 214 |
+
schema = parse_schema(request.get("questions"))
|
| 215 |
+
state = request.get("state", "")
|
| 216 |
+
if not isinstance(state, (str, dict, list, int, float, bool, type(None))):
|
| 217 |
+
raise ClefError("補足入力は文章またはJSONにしてください。")
|
| 218 |
+
if len(json.dumps(state, ensure_ascii=False)) > 60_000:
|
| 219 |
+
raise ClefError("補足入力は60,000文字以内にしてください。")
|
| 220 |
+
return {
|
| 221 |
+
"profile": profile,
|
| 222 |
+
"questions": schema,
|
| 223 |
+
"state": state,
|
| 224 |
+
"max_pixels": integer(request.get("max_pixels", 262144), "画像の最大画素数", 65536, 1048576),
|
| 225 |
+
"max_length": integer(request.get("max_length", 4096), "入力トークン上限", 512, 8192),
|
| 226 |
+
}
|
| 227 |
+
|
| 228 |
+
|
| 229 |
+
def fingerprint(request, items):
|
| 230 |
+
profile = PROFILES[request["profile"]]
|
| 231 |
+
return canonical_hash(
|
| 232 |
+
{
|
| 233 |
+
"request": request,
|
| 234 |
+
"model": MODELS[profile["model"]],
|
| 235 |
+
"profile": profile,
|
| 236 |
+
"images": [item["sha256"] for item in items],
|
| 237 |
+
"preprocessing": "exif-rgb-v1",
|
| 238 |
+
}
|
| 239 |
+
)
|
| 240 |
+
|
| 241 |
+
|
| 242 |
+
def summary(answer, question):
|
| 243 |
+
kind = answer["type"]
|
| 244 |
+
if kind == "noul":
|
| 245 |
+
return f"P(真) {answer['noul']:.3f}"
|
| 246 |
+
if kind == "choice":
|
| 247 |
+
choice = answer["choice"]
|
| 248 |
+
return f"{choice} · p={answer['confidence']:.3f}"
|
| 249 |
+
return f"期待値 {answer['score']:.2f} (0–{len(question['criteria']) - 1})"
|
| 250 |
+
|
| 251 |
+
|
| 252 |
+
def ambiguity(answer):
|
| 253 |
+
if answer["type"] == "noul":
|
| 254 |
+
return 1 - 2 * abs(answer["noul"] - 0.5)
|
| 255 |
+
probabilities = list(answer["probabilities"].values())
|
| 256 |
+
return -sum(p * math.log(p) for p in probabilities if p > 0) / math.log(len(probabilities))
|
| 257 |
+
|
| 258 |
+
|
| 259 |
+
def compare_runs(left, right):
|
| 260 |
+
for key in ("questions", "state", "max_pixels"):
|
| 261 |
+
if canonical_hash(left["request"].get(key)) != canonical_hash(right["request"].get(key)):
|
| 262 |
+
raise ClefError("比較には判断項目・補足入力・処理解像度の一致が必要です。")
|
| 263 |
+
other = {x["sha256"]: x for x in right["items"] if x["status"] == "done"}
|
| 264 |
+
rows = []
|
| 265 |
+
for item in left["items"]:
|
| 266 |
+
if item["status"] != "done" or item["sha256"] not in other:
|
| 267 |
+
continue
|
| 268 |
+
differences = {}
|
| 269 |
+
for qid, answer in item["answers"].items():
|
| 270 |
+
target = other[item["sha256"]]["answers"][qid]
|
| 271 |
+
if answer["type"] == "noul":
|
| 272 |
+
delta = target["noul"] - answer["noul"]
|
| 273 |
+
differences[qid] = {
|
| 274 |
+
"delta": delta,
|
| 275 |
+
"changed": (answer["noul"] >= 0.5) != (target["noul"] >= 0.5),
|
| 276 |
+
"max_probability_delta": abs(delta),
|
| 277 |
+
}
|
| 278 |
+
else:
|
| 279 |
+
delta = max(abs(p - target["probabilities"][key]) for key, p in answer["probabilities"].items())
|
| 280 |
+
changed = (
|
| 281 |
+
answer.get("choice") != target.get("choice")
|
| 282 |
+
if answer["type"] == "choice"
|
| 283 |
+
else max(answer["probabilities"], key=answer["probabilities"].get)
|
| 284 |
+
!= max(target["probabilities"], key=target["probabilities"].get)
|
| 285 |
+
)
|
| 286 |
+
differences[qid] = {"changed": changed, "max_probability_delta": delta}
|
| 287 |
+
if answer["type"] == "score":
|
| 288 |
+
differences[qid]["delta"] = target["score"] - answer["score"]
|
| 289 |
+
rows.append(
|
| 290 |
+
{
|
| 291 |
+
"name": item["name"],
|
| 292 |
+
"sha256": item["sha256"],
|
| 293 |
+
"differences": differences,
|
| 294 |
+
"changed": any(x["changed"] for x in differences.values()),
|
| 295 |
+
"max_probability_delta": max(x["max_probability_delta"] for x in differences.values()),
|
| 296 |
+
}
|
| 297 |
+
)
|
| 298 |
+
return sorted(rows, key=lambda x: x["max_probability_delta"], reverse=True)
|
| 299 |
+
|
| 300 |
+
|
| 301 |
+
def export_csv(run):
|
| 302 |
+
schema = run["request"]["questions"]
|
| 303 |
+
columns = ["name", "sha256", "status", "error"] + [
|
| 304 |
+
col for qid in schema for col in ("answer:" + qid, "confidence:" + qid)
|
| 305 |
+
]
|
| 306 |
+
stream = io.StringIO(newline="")
|
| 307 |
+
writer = csv.DictWriter(stream, fieldnames=columns)
|
| 308 |
+
writer.writeheader()
|
| 309 |
+
for item in run["items"]:
|
| 310 |
+
row = {key: item.get(key, "") for key in columns[:4]}
|
| 311 |
+
for qid, answer in item.get("answers", {}).items():
|
| 312 |
+
row["answer:" + qid] = answer.get("choice", answer.get("score", answer.get("noul", "")))
|
| 313 |
+
row["confidence:" + qid] = answer.get("confidence", "")
|
| 314 |
+
for key, value in row.items():
|
| 315 |
+
if isinstance(value, str) and value.lstrip().startswith(("=", "+", "-", "@")):
|
| 316 |
+
row[key] = "'" + value
|
| 317 |
+
writer.writerow(row)
|
| 318 |
+
return stream.getvalue()
|
| 319 |
+
|
| 320 |
+
|
| 321 |
+
def source_manifest(root, model, verify_hashes=False):
|
| 322 |
+
directory = Path(root) / "source" / model
|
| 323 |
+
try:
|
| 324 |
+
manifest = json.loads((directory / "release.json").read_text(encoding="utf-8"))
|
| 325 |
+
except (OSError, ValueError) as exc:
|
| 326 |
+
raise ClefError(f"{model}のBF16元重みは未導入です。aikimi-clef-setup.bat --source --model {model}を実行してください。") from exc
|
| 327 |
+
if manifest.get("revision") != MODELS[model]["revision"] or manifest.get("repo") != MODELS[model]["repo"]:
|
| 328 |
+
raise ClefError("Clefの固定リビジョンが一致しません。セットアップを再実行してください。")
|
| 329 |
+
for item in manifest["files"]:
|
| 330 |
+
path = directory / item["path"]
|
| 331 |
+
if (
|
| 332 |
+
not path.resolve().is_relative_to(directory.resolve())
|
| 333 |
+
or not path.is_file()
|
| 334 |
+
or path.stat().st_size != item["size"]
|
| 335 |
+
):
|
| 336 |
+
raise ClefError(f"Clef配布物が不足・変更されています: {item['path']}")
|
| 337 |
+
if (verify_hashes or path.suffix == ".py") and sha256(path) != item["sha256"]:
|
| 338 |
+
raise ClefError(f"ClefのSHA-256が一致しません: {item['path']}")
|
| 339 |
+
return directory, manifest
|
clef_runtime/runtime.py
ADDED
|
@@ -0,0 +1,203 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Keep the official joint head while moving only vocabulary rows over PCIe."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import gc
|
| 6 |
+
import importlib.util
|
| 7 |
+
import json
|
| 8 |
+
import sys
|
| 9 |
+
from contextlib import contextmanager
|
| 10 |
+
from functools import partial
|
| 11 |
+
|
| 12 |
+
import torch
|
| 13 |
+
|
| 14 |
+
from .bundle import bundle_manifest, read_bundle
|
| 15 |
+
from .core import PROFILES
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
class LexicalRows:
|
| 19 |
+
def __init__(self, weight, *, device, dtype):
|
| 20 |
+
if weight.device.type != "cpu" or weight.is_meta:
|
| 21 |
+
raise ValueError("語彙行列には実体のあるCPUテンソルが必要です。")
|
| 22 |
+
self.weight, self.device, self.dtype = weight, device, dtype
|
| 23 |
+
|
| 24 |
+
def __getitem__(self, token_ids):
|
| 25 |
+
shape = (*token_ids.shape, self.weight.shape[1])
|
| 26 |
+
return self.weight.index_select(0, token_ids.reshape(-1).cpu()).reshape(shape).to(self.device, dtype=self.dtype)
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
class CPUEmbedding(torch.nn.Module):
|
| 30 |
+
def __init__(self, weight, *, device, dtype):
|
| 31 |
+
super().__init__()
|
| 32 |
+
self.rows = LexicalRows(weight, device=device, dtype=dtype)
|
| 33 |
+
self.weight = torch.nn.Parameter(weight, requires_grad=False)
|
| 34 |
+
|
| 35 |
+
def forward(self, tokens):
|
| 36 |
+
return self.rows[tokens]
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def load_options(profile):
|
| 40 |
+
settings = PROFILES[profile]
|
| 41 |
+
result = {"device_map": {"": 0}}
|
| 42 |
+
if settings["cpu_embeddings"]:
|
| 43 |
+
result["device_map"] = {
|
| 44 |
+
"visual": 0,
|
| 45 |
+
"language_model.embed_tokens": "cpu",
|
| 46 |
+
"language_model.layers": 0,
|
| 47 |
+
"language_model.norm": 0,
|
| 48 |
+
"language_model.rotary_emb": 0,
|
| 49 |
+
}
|
| 50 |
+
if settings["precision"] != "bf16":
|
| 51 |
+
quantization = {
|
| 52 |
+
"llm_int8_skip_modules": ["visual", "lm_head"],
|
| 53 |
+
"llm_int8_enable_fp32_cpu_offload": settings["cpu_embeddings"],
|
| 54 |
+
}
|
| 55 |
+
if settings["precision"] == "int8":
|
| 56 |
+
quantization["load_in_8bit"] = True
|
| 57 |
+
else:
|
| 58 |
+
quantization.update(
|
| 59 |
+
load_in_4bit=True,
|
| 60 |
+
bnb_4bit_quant_type="nf4",
|
| 61 |
+
bnb_4bit_use_double_quant=True,
|
| 62 |
+
bnb_4bit_compute_dtype=torch.bfloat16,
|
| 63 |
+
)
|
| 64 |
+
result["quantization"] = quantization
|
| 65 |
+
return result
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
def encode_complete(official, processor, record, max_length):
|
| 69 |
+
# The release silently truncates state. Encode once without truncation and
|
| 70 |
+
# refuse overflow before collating or allocating decoder activations.
|
| 71 |
+
encoded = official.encode_record(processor.tokenizer, record, processor=processor, max_length=1_000_000)
|
| 72 |
+
if len(encoded.input_ids) > max_length:
|
| 73 |
+
raise ValueError(
|
| 74 |
+
f"入力は{len(encoded.input_ids)}トークンです。上限{max_length}を超えます。補足や項目を短くするか処理解像度を下げてください。"
|
| 75 |
+
)
|
| 76 |
+
return encoded
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def tensor_from_source(source, key):
|
| 80 |
+
from safetensors import safe_open
|
| 81 |
+
|
| 82 |
+
index = json.loads((source / "model.safetensors.index.json").read_text())
|
| 83 |
+
with safe_open(source / index["weight_map"][key], framework="pt", device="cpu", backend="pread") as stream:
|
| 84 |
+
return stream.get_tensor(key)
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
@contextmanager
|
| 88 |
+
def stream_weights():
|
| 89 |
+
import transformers.modeling_utils as loading
|
| 90 |
+
|
| 91 |
+
# Windows reserves commit for every copy-on-write mmap, including unread
|
| 92 |
+
# pages. Keep TF's lazy tensor loading without retaining all shard maps.
|
| 93 |
+
original = loading.safe_open
|
| 94 |
+
loading.safe_open = partial(original, backend="pread")
|
| 95 |
+
try:
|
| 96 |
+
yield
|
| 97 |
+
finally:
|
| 98 |
+
loading.safe_open = original
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
class Runner:
|
| 102 |
+
@classmethod
|
| 103 |
+
def from_directory(cls, directory, profile):
|
| 104 |
+
return cls(None, profile, directory=directory)
|
| 105 |
+
|
| 106 |
+
def __init__(self, root, profile, *, directory=None):
|
| 107 |
+
from safetensors.torch import load_file
|
| 108 |
+
from transformers import AutoProcessor, BitsAndBytesConfig, Qwen3_5Model
|
| 109 |
+
|
| 110 |
+
if not torch.cuda.is_available() or not torch.cuda.is_bf16_supported():
|
| 111 |
+
raise RuntimeError("ClefにはBF16対応のNVIDIA CUDA GPUが必要です。")
|
| 112 |
+
self.settings, self.profile = PROFILES[profile], profile
|
| 113 |
+
total = torch.cuda.get_device_properties(0).total_memory / 2**30
|
| 114 |
+
if total < self.settings["min_vram_gib"]:
|
| 115 |
+
raise RuntimeError(
|
| 116 |
+
f"{self.settings['label']}のGPUメモリが不足しています。16GB設定またはFlashを選んでください。"
|
| 117 |
+
)
|
| 118 |
+
source, _ = read_bundle(directory, profile) if directory is not None else bundle_manifest(root, profile)
|
| 119 |
+
spec = importlib.util.spec_from_file_location(
|
| 120 |
+
"clef_release_" + self.settings["model"].replace("-", "_"), source / "joint_schema_model.py"
|
| 121 |
+
)
|
| 122 |
+
self.official = importlib.util.module_from_spec(spec)
|
| 123 |
+
sys.modules[spec.name] = self.official
|
| 124 |
+
spec.loader.exec_module(self.official)
|
| 125 |
+
options = load_options(profile)
|
| 126 |
+
saved = source if self.settings["precision"] != "bf16" else None
|
| 127 |
+
if "quantization" in options:
|
| 128 |
+
options["quantization_config"] = BitsAndBytesConfig(**options.pop("quantization"))
|
| 129 |
+
print(f"{self.settings['label']}: {'保存済み' if saved else '公式重み'}を読み込み中", flush=True) # noqa: T201
|
| 130 |
+
with stream_weights():
|
| 131 |
+
self.backbone, loading = Qwen3_5Model.from_pretrained(
|
| 132 |
+
saved or source,
|
| 133 |
+
dtype=torch.bfloat16,
|
| 134 |
+
local_files_only=True,
|
| 135 |
+
attn_implementation="sdpa",
|
| 136 |
+
output_loading_info=True,
|
| 137 |
+
**options,
|
| 138 |
+
)
|
| 139 |
+
expected_extra = {"lm_head.weight"}
|
| 140 |
+
if loading["missing_keys"] or set(loading["unexpected_keys"]) != expected_extra:
|
| 141 |
+
raise RuntimeError(f"Clef本体のキーが一致しません: {loading}")
|
| 142 |
+
self.backbone.eval()
|
| 143 |
+
self.backbone.config.use_cache = False
|
| 144 |
+
if self.settings["cpu_embeddings"]:
|
| 145 |
+
# Read the bundle's BF16 rows directly; CPU offload may cast to FP32.
|
| 146 |
+
# Discard the old embedding together with its hook. Detaching a
|
| 147 |
+
# hook can temporarily restore the entire vocabulary onto CUDA.
|
| 148 |
+
embedding = tensor_from_source(source, "language_model.embed_tokens.weight")
|
| 149 |
+
self.backbone.language_model.embed_tokens = CPUEmbedding(embedding, device="cuda:0", dtype=torch.bfloat16)
|
| 150 |
+
gc.collect()
|
| 151 |
+
self.lexical = LexicalRows(tensor_from_source(source, "lm_head.weight"), device="cuda:0", dtype=torch.bfloat16)
|
| 152 |
+
self.head = self.official.JointSchemaHead(**json.loads((source / "joint_head_config.json").read_text()))
|
| 153 |
+
self.head.load_state_dict(load_file(source / "joint_head.safetensors", backend="pread"), strict=True)
|
| 154 |
+
self.head = self.head.to(device="cuda:0", dtype=torch.bfloat16).eval()
|
| 155 |
+
self.processor = AutoProcessor.from_pretrained(source, local_files_only=True)
|
| 156 |
+
self.source = source
|
| 157 |
+
self.root, self.saved = root, saved
|
| 158 |
+
|
| 159 |
+
@torch.inference_mode()
|
| 160 |
+
def decide(self, request, image=None):
|
| 161 |
+
record = {
|
| 162 |
+
"model": self.settings["model"],
|
| 163 |
+
"state": request["state"],
|
| 164 |
+
"questions": request["questions"],
|
| 165 |
+
"media_kwargs": {"max_pixels": request["max_pixels"]},
|
| 166 |
+
}
|
| 167 |
+
if image is not None:
|
| 168 |
+
record["images"] = [image]
|
| 169 |
+
encoded = encode_complete(self.official, self.processor, record, request["max_length"])
|
| 170 |
+
batch = self.official.collate_records([encoded], self.processor.tokenizer.pad_token_id, torch.device("cuda:0"))
|
| 171 |
+
torch.cuda.reset_peak_memory_stats()
|
| 172 |
+
media = batch.get("media") or {}
|
| 173 |
+
model = self.backbone if media else self.backbone.language_model
|
| 174 |
+
outputs = model(
|
| 175 |
+
input_ids=batch["input_ids"],
|
| 176 |
+
attention_mask=batch["attention_mask"],
|
| 177 |
+
use_cache=False,
|
| 178 |
+
return_dict=True,
|
| 179 |
+
**media,
|
| 180 |
+
)
|
| 181 |
+
logits = self.head(
|
| 182 |
+
outputs.last_hidden_state, batch["input_ids"], batch["attention_mask"], batch["records"], self.lexical
|
| 183 |
+
)[0]
|
| 184 |
+
answers = {
|
| 185 |
+
q.question_id: self.official.systemone_answer(
|
| 186 |
+
request["questions"][q.question_id],
|
| 187 |
+
dict(zip(q.option_ids, logit.float().softmax(-1).tolist(), strict=True)),
|
| 188 |
+
)
|
| 189 |
+
for q, logit in zip(encoded.questions, logits, strict=True)
|
| 190 |
+
}
|
| 191 |
+
torch.cuda.synchronize()
|
| 192 |
+
return {
|
| 193 |
+
"model": self.settings["model"],
|
| 194 |
+
"answers": answers,
|
| 195 |
+
"usage": {"input_tokens": len(encoded.input_ids), "output_tokens": 0},
|
| 196 |
+
"peak_allocated_gib": torch.cuda.max_memory_allocated() / 2**30,
|
| 197 |
+
"peak_reserved_gib": torch.cuda.max_memory_reserved() / 2**30,
|
| 198 |
+
}
|
| 199 |
+
|
| 200 |
+
def close(self):
|
| 201 |
+
self.backbone = self.head = self.lexical = self.processor = None
|
| 202 |
+
gc.collect()
|
| 203 |
+
torch.cuda.empty_cache()
|
complete.json
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"repo": "Cloudflare/clef-flash",
|
| 3 |
+
"revision": "17f0b0ad64efb65d273590632833508766b2aae6",
|
| 4 |
+
"precision": "int8",
|
| 5 |
+
"format": 1,
|
| 6 |
+
"transformers": "5.10.2",
|
| 7 |
+
"bitsandbytes": "0.50.2",
|
| 8 |
+
"vision": "bf16",
|
| 9 |
+
"bundle_format": 1,
|
| 10 |
+
"files": [
|
| 11 |
+
{
|
| 12 |
+
"path": "LICENSE",
|
| 13 |
+
"size": 11544,
|
| 14 |
+
"sha256": "bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a"
|
| 15 |
+
},
|
| 16 |
+
{
|
| 17 |
+
"path": "README.md",
|
| 18 |
+
"size": 3913,
|
| 19 |
+
"sha256": "8c2553f158a27717a0e5128bebdb576b3e81fa089b821bc79d71a3cfde77a256"
|
| 20 |
+
},
|
| 21 |
+
{
|
| 22 |
+
"path": "chat_template.jinja",
|
| 23 |
+
"size": 7756,
|
| 24 |
+
"sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"
|
| 25 |
+
},
|
| 26 |
+
{
|
| 27 |
+
"path": "clef_runtime/__init__.py",
|
| 28 |
+
"size": 72,
|
| 29 |
+
"sha256": "8252a7c5618dd1324b6c4c307a0c7d574712eb5481d3e129947b99dba14775ca"
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"path": "clef_runtime/bundle.py",
|
| 33 |
+
"size": 3307,
|
| 34 |
+
"sha256": "4abd7c664fbf94e7217b4554657a5ba5ea6e1fea633a43455b2c213cb9b09ba2"
|
| 35 |
+
},
|
| 36 |
+
{
|
| 37 |
+
"path": "clef_runtime/cache.py",
|
| 38 |
+
"size": 1351,
|
| 39 |
+
"sha256": "431f466374141ab80af41d4cc0ddca93dfc9fc74a28de8de792f5be690eed1cb"
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"path": "clef_runtime/core.py",
|
| 43 |
+
"size": 14153,
|
| 44 |
+
"sha256": "15edc6a89e4a011a308bc283ac182c793c7c72227dbe06788d4f9888e377ab36"
|
| 45 |
+
},
|
| 46 |
+
{
|
| 47 |
+
"path": "clef_runtime/runtime.py",
|
| 48 |
+
"size": 8826,
|
| 49 |
+
"sha256": "69df25f5118190fe066199225f0c9c2b25f10b93e1e550ff50890252aafa02b6"
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"path": "config.json",
|
| 53 |
+
"size": 3479,
|
| 54 |
+
"sha256": "ff6de63fbfcd26ce6d0fed3c40039ddcf2c6833c6f07a189473ec2eeb59ac884"
|
| 55 |
+
},
|
| 56 |
+
{
|
| 57 |
+
"path": "inference.py",
|
| 58 |
+
"size": 1521,
|
| 59 |
+
"sha256": "9667ef47476bd4e8dddaa66d7d02f45733d542e815629ebb30ec5fc7639fb135"
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"path": "joint_head.safetensors",
|
| 63 |
+
"size": 243538016,
|
| 64 |
+
"sha256": "19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0"
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"path": "joint_head_config.json",
|
| 68 |
+
"size": 119,
|
| 69 |
+
"sha256": "77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0"
|
| 70 |
+
},
|
| 71 |
+
{
|
| 72 |
+
"path": "joint_schema_model.py",
|
| 73 |
+
"size": 23263,
|
| 74 |
+
"sha256": "0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3"
|
| 75 |
+
},
|
| 76 |
+
{
|
| 77 |
+
"path": "lm-head.safetensors",
|
| 78 |
+
"size": 2034237568,
|
| 79 |
+
"sha256": "dd63614f1dc80dce2d83be3f3e69af1f8a0ebc8add9b8e0abf3910f563e44344"
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"path": "model-00001-of-00005.safetensors",
|
| 83 |
+
"size": 2034237592,
|
| 84 |
+
"sha256": "9c5483aa32a952b3a9a5d75f3c000c9ff934d3cbbb317b228cadff89e25bcffa"
|
| 85 |
+
},
|
| 86 |
+
{
|
| 87 |
+
"path": "model-00002-of-00005.safetensors",
|
| 88 |
+
"size": 1984182833,
|
| 89 |
+
"sha256": "6999510bfc3c9ea6536090fb76b70bb786a0f4bd76f290ea35f35f58aa77e4f6"
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"path": "model-00003-of-00005.safetensors",
|
| 93 |
+
"size": 1983829880,
|
| 94 |
+
"sha256": "efa24fbcede25d84116de286e544554513bd0400c40c66b99c32e846f5096cdd"
|
| 95 |
+
},
|
| 96 |
+
{
|
| 97 |
+
"path": "model-00004-of-00005.safetensors",
|
| 98 |
+
"size": 1983498867,
|
| 99 |
+
"sha256": "7a5dd174473310a090c74abe907de00da6b85c46f8a3b5bcaec20267449fbdbd"
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"path": "model-00005-of-00005.safetensors",
|
| 103 |
+
"size": 1886989356,
|
| 104 |
+
"sha256": "1463c85bac6efd374bb5189d3beda19df628a786a9068b00de74dd3504f12df9"
|
| 105 |
+
},
|
| 106 |
+
{
|
| 107 |
+
"path": "model.safetensors.index.json",
|
| 108 |
+
"size": 112960,
|
| 109 |
+
"sha256": "493691fe586b33a34830abaf788dc940ec43fcf4913139c9ed5dd1ba069542e3"
|
| 110 |
+
},
|
| 111 |
+
{
|
| 112 |
+
"path": "processor_config.json",
|
| 113 |
+
"size": 1191,
|
| 114 |
+
"sha256": "d89ef49ce9cd37fbf510158e13c1ef063d9286411c1ec9049932dbe0487143b1"
|
| 115 |
+
},
|
| 116 |
+
{
|
| 117 |
+
"path": "requirements.txt",
|
| 118 |
+
"size": 194,
|
| 119 |
+
"sha256": "6ea662f95ec818e7f7a17742211ba0e6de4744ac61055565f01214e8a818588a"
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"path": "tokenizer.json",
|
| 123 |
+
"size": 19989325,
|
| 124 |
+
"sha256": "06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
|
| 125 |
+
},
|
| 126 |
+
{
|
| 127 |
+
"path": "tokenizer_config.json",
|
| 128 |
+
"size": 1075,
|
| 129 |
+
"sha256": "91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72"
|
| 130 |
+
}
|
| 131 |
+
]
|
| 132 |
+
}
|
config.json
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3_5Model"
|
| 4 |
+
],
|
| 5 |
+
"dtype": "bfloat16",
|
| 6 |
+
"image_token_id": 248056,
|
| 7 |
+
"model_type": "qwen3_5",
|
| 8 |
+
"quantization_config": {
|
| 9 |
+
"_load_in_4bit": false,
|
| 10 |
+
"_load_in_8bit": true,
|
| 11 |
+
"bnb_4bit_compute_dtype": "float32",
|
| 12 |
+
"bnb_4bit_quant_storage": "uint8",
|
| 13 |
+
"bnb_4bit_quant_type": "fp4",
|
| 14 |
+
"bnb_4bit_use_double_quant": false,
|
| 15 |
+
"llm_int8_enable_fp32_cpu_offload": false,
|
| 16 |
+
"llm_int8_has_fp16_weight": false,
|
| 17 |
+
"llm_int8_skip_modules": [
|
| 18 |
+
"visual",
|
| 19 |
+
"lm_head"
|
| 20 |
+
],
|
| 21 |
+
"llm_int8_threshold": 6.0,
|
| 22 |
+
"load_in_4bit": false,
|
| 23 |
+
"load_in_8bit": true,
|
| 24 |
+
"quant_method": "bitsandbytes"
|
| 25 |
+
},
|
| 26 |
+
"text_config": {
|
| 27 |
+
"attention_bias": false,
|
| 28 |
+
"attention_dropout": 0.0,
|
| 29 |
+
"attn_output_gate": true,
|
| 30 |
+
"bos_token_id": null,
|
| 31 |
+
"dtype": "bfloat16",
|
| 32 |
+
"eos_token_id": 248044,
|
| 33 |
+
"full_attention_interval": 4,
|
| 34 |
+
"head_dim": 256,
|
| 35 |
+
"hidden_act": "silu",
|
| 36 |
+
"hidden_size": 4096,
|
| 37 |
+
"initializer_range": 0.02,
|
| 38 |
+
"intermediate_size": 12288,
|
| 39 |
+
"layer_types": [
|
| 40 |
+
"linear_attention",
|
| 41 |
+
"linear_attention",
|
| 42 |
+
"linear_attention",
|
| 43 |
+
"full_attention",
|
| 44 |
+
"linear_attention",
|
| 45 |
+
"linear_attention",
|
| 46 |
+
"linear_attention",
|
| 47 |
+
"full_attention",
|
| 48 |
+
"linear_attention",
|
| 49 |
+
"linear_attention",
|
| 50 |
+
"linear_attention",
|
| 51 |
+
"full_attention",
|
| 52 |
+
"linear_attention",
|
| 53 |
+
"linear_attention",
|
| 54 |
+
"linear_attention",
|
| 55 |
+
"full_attention",
|
| 56 |
+
"linear_attention",
|
| 57 |
+
"linear_attention",
|
| 58 |
+
"linear_attention",
|
| 59 |
+
"full_attention",
|
| 60 |
+
"linear_attention",
|
| 61 |
+
"linear_attention",
|
| 62 |
+
"linear_attention",
|
| 63 |
+
"full_attention",
|
| 64 |
+
"linear_attention",
|
| 65 |
+
"linear_attention",
|
| 66 |
+
"linear_attention",
|
| 67 |
+
"full_attention",
|
| 68 |
+
"linear_attention",
|
| 69 |
+
"linear_attention",
|
| 70 |
+
"linear_attention",
|
| 71 |
+
"full_attention"
|
| 72 |
+
],
|
| 73 |
+
"linear_conv_kernel_dim": 4,
|
| 74 |
+
"linear_key_head_dim": 128,
|
| 75 |
+
"linear_num_key_heads": 16,
|
| 76 |
+
"linear_num_value_heads": 32,
|
| 77 |
+
"linear_value_head_dim": 128,
|
| 78 |
+
"mamba_ssm_dtype": "float32",
|
| 79 |
+
"max_position_embeddings": 262144,
|
| 80 |
+
"mlp_only_layers": [],
|
| 81 |
+
"model_type": "qwen3_5_text",
|
| 82 |
+
"mtp_num_hidden_layers": 0,
|
| 83 |
+
"mtp_use_dedicated_embeddings": false,
|
| 84 |
+
"num_attention_heads": 16,
|
| 85 |
+
"num_hidden_layers": 32,
|
| 86 |
+
"num_key_value_heads": 4,
|
| 87 |
+
"pad_token_id": null,
|
| 88 |
+
"partial_rotary_factor": 0.25,
|
| 89 |
+
"rms_norm_eps": 1e-06,
|
| 90 |
+
"rope_parameters": {
|
| 91 |
+
"mrope_interleaved": true,
|
| 92 |
+
"mrope_section": [
|
| 93 |
+
11,
|
| 94 |
+
11,
|
| 95 |
+
10
|
| 96 |
+
],
|
| 97 |
+
"partial_rotary_factor": 0.25,
|
| 98 |
+
"rope_theta": 10000000,
|
| 99 |
+
"rope_type": "default"
|
| 100 |
+
},
|
| 101 |
+
"tie_word_embeddings": false,
|
| 102 |
+
"use_cache": true,
|
| 103 |
+
"vocab_size": 248320
|
| 104 |
+
},
|
| 105 |
+
"tie_word_embeddings": false,
|
| 106 |
+
"transformers_version": "5.10.2",
|
| 107 |
+
"use_cache": false,
|
| 108 |
+
"video_token_id": 248057,
|
| 109 |
+
"vision_config": {
|
| 110 |
+
"deepstack_visual_indexes": [],
|
| 111 |
+
"depth": 27,
|
| 112 |
+
"dtype": "bfloat16",
|
| 113 |
+
"hidden_act": "gelu_pytorch_tanh",
|
| 114 |
+
"hidden_size": 1152,
|
| 115 |
+
"in_channels": 3,
|
| 116 |
+
"initializer_range": 0.02,
|
| 117 |
+
"intermediate_size": 4304,
|
| 118 |
+
"model_type": "qwen3_5_vision",
|
| 119 |
+
"num_heads": 16,
|
| 120 |
+
"num_position_embeddings": 2304,
|
| 121 |
+
"out_hidden_size": 4096,
|
| 122 |
+
"patch_size": 16,
|
| 123 |
+
"spatial_merge_size": 2,
|
| 124 |
+
"temporal_patch_size": 2
|
| 125 |
+
},
|
| 126 |
+
"vision_end_token_id": 248054,
|
| 127 |
+
"vision_start_token_id": 248053
|
| 128 |
+
}
|
inference.py
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Run a downloaded Clef quantized bundle, without Forge or source checkpoints."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import argparse
|
| 6 |
+
import json
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def main():
|
| 11 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 12 |
+
parser.add_argument("--profile", choices=["flash-int8", "clef-24gb", "clef-16gb"], required=True)
|
| 13 |
+
parser.add_argument("--image", type=Path)
|
| 14 |
+
parser.add_argument("--questions", type=Path)
|
| 15 |
+
parser.add_argument("--state", default="添付画像を評価してください。")
|
| 16 |
+
parser.add_argument("--max-pixels", type=int, default=262144)
|
| 17 |
+
parser.add_argument("--max-length", type=int, default=2048)
|
| 18 |
+
args = parser.parse_args()
|
| 19 |
+
from clef_runtime.core import TEMPLATES, validate_request
|
| 20 |
+
from clef_runtime.runtime import Runner
|
| 21 |
+
|
| 22 |
+
questions = json.loads(args.questions.read_text(encoding="utf-8")) if args.questions else TEMPLATES["画像の評価"]
|
| 23 |
+
request = validate_request(
|
| 24 |
+
{
|
| 25 |
+
"profile": args.profile,
|
| 26 |
+
"questions": questions,
|
| 27 |
+
"state": args.state,
|
| 28 |
+
"max_pixels": args.max_pixels,
|
| 29 |
+
"max_length": args.max_length,
|
| 30 |
+
}
|
| 31 |
+
)
|
| 32 |
+
runner = Runner.from_directory(Path(__file__).resolve().parent, args.profile)
|
| 33 |
+
try:
|
| 34 |
+
result = runner.decide(request, str(args.image.resolve()) if args.image else None)
|
| 35 |
+
print(json.dumps(result, ensure_ascii=False, indent=2), flush=True) # noqa: T201
|
| 36 |
+
finally:
|
| 37 |
+
runner.close()
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
if __name__ == "__main__":
|
| 41 |
+
main()
|
joint_head.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0
|
| 3 |
+
size 243538016
|
joint_head_config.json
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"hidden_size": 4096,
|
| 3 |
+
"width": 1024,
|
| 4 |
+
"routing_layers": 2,
|
| 5 |
+
"layers": 4,
|
| 6 |
+
"heads": 16,
|
| 7 |
+
"feedforward": 4096
|
| 8 |
+
}
|
joint_schema_model.py
ADDED
|
@@ -0,0 +1,576 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Clef: a multimodal Qwen backbone with a joint schema head for typed decisions.
|
| 2 |
+
|
| 3 |
+
A record provides a ``state`` (any JSON value), optional ``images`` and ``videos``,
|
| 4 |
+
and ``questions``. Each question has a ``type`` (``noul``, ``choice``, or ``score``),
|
| 5 |
+
``instructions``, and, for ``choice`` and ``score``, ``criteria`` describing the
|
| 6 |
+
allowed options. The model returns one logit per allowed option for every question.
|
| 7 |
+
``systemone`` answers a Jev/SystemOne ``/v1/systemone`` request body with the same
|
| 8 |
+
response body.
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
from __future__ import annotations
|
| 12 |
+
|
| 13 |
+
import json
|
| 14 |
+
import math
|
| 15 |
+
from dataclasses import dataclass
|
| 16 |
+
from dataclasses import field as dataclass_field
|
| 17 |
+
from pathlib import Path
|
| 18 |
+
from typing import Any
|
| 19 |
+
|
| 20 |
+
import torch
|
| 21 |
+
import torch.nn.functional as functional
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
SYSTEM_PROMPT = (
|
| 25 |
+
"Read the complete state and schema. Decide every field jointly. Each answer "
|
| 26 |
+
"must be exactly one of that field's allowed options."
|
| 27 |
+
)
|
| 28 |
+
IMAGE_PLACEHOLDER = "<|vision_start|><|image_pad|><|vision_end|>"
|
| 29 |
+
VIDEO_PLACEHOLDER = "<|vision_start|><|video_pad|><|vision_end|>"
|
| 30 |
+
MEDIA_BATCH_KEYS = ("pixel_values", "image_grid_thw", "pixel_values_videos", "video_grid_thw")
|
| 31 |
+
MEDIA_TOKEN_KEYS = ("mm_token_type_ids",)
|
| 32 |
+
QUESTION_TYPES = {"noul": 0, "choice": 1, "score": 2}
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def render(value: Any) -> str:
|
| 36 |
+
if isinstance(value, str):
|
| 37 |
+
return value
|
| 38 |
+
return json.dumps(
|
| 39 |
+
value,
|
| 40 |
+
ensure_ascii=False,
|
| 41 |
+
separators=(",", ":"),
|
| 42 |
+
sort_keys=True,
|
| 43 |
+
)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def question_options(question: dict[str, Any]) -> list[tuple[str, Any]]:
|
| 47 |
+
question_type = str(question["type"])
|
| 48 |
+
if question_type == "noul":
|
| 49 |
+
criteria = {
|
| 50 |
+
"true": "The proposition is true or the answer is yes.",
|
| 51 |
+
"false": "The proposition is false or the answer is no.",
|
| 52 |
+
}
|
| 53 |
+
criteria.update(question.get("criteria") or {})
|
| 54 |
+
return [(key, criteria[key]) for key in ("true", "false")]
|
| 55 |
+
if question_type == "choice":
|
| 56 |
+
return sorted((str(key), value) for key, value in question["criteria"].items())
|
| 57 |
+
return [(str(index), value) for index, value in enumerate(question["criteria"])]
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
@dataclass(frozen=True)
|
| 61 |
+
class EncodedQuestion:
|
| 62 |
+
question_id: str
|
| 63 |
+
question_type: int
|
| 64 |
+
question_span: tuple[int, int]
|
| 65 |
+
option_spans: tuple[tuple[int, int], ...]
|
| 66 |
+
option_ids: tuple[str, ...]
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
@dataclass(frozen=True)
|
| 70 |
+
class EncodedRecord:
|
| 71 |
+
input_ids: tuple[int, ...]
|
| 72 |
+
questions: tuple[EncodedQuestion, ...]
|
| 73 |
+
record_id: str
|
| 74 |
+
media: dict[str, Any] | None = dataclass_field(default=None, compare=False, repr=False)
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def _tokens(tokenizer: Any, text: str) -> list[int]:
|
| 78 |
+
return tokenizer(text, add_special_tokens=False).input_ids
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def _encode_media(processor: Any, record: dict[str, Any]) -> tuple[list[int], dict[str, Any] | None]:
|
| 82 |
+
images = list(record.get("images") or [])
|
| 83 |
+
videos = list(record.get("videos") or [])
|
| 84 |
+
if not images and not videos:
|
| 85 |
+
return [], None
|
| 86 |
+
if processor is None:
|
| 87 |
+
raise ValueError("records with images or videos require a processor")
|
| 88 |
+
text = IMAGE_PLACEHOLDER * len(images) + VIDEO_PLACEHOLDER * len(videos) + "\n"
|
| 89 |
+
encoded = processor(
|
| 90 |
+
text=[text],
|
| 91 |
+
images=images or None,
|
| 92 |
+
videos=videos or None,
|
| 93 |
+
return_tensors="pt",
|
| 94 |
+
**(record.get("media_kwargs") or {}),
|
| 95 |
+
)
|
| 96 |
+
media = {key: encoded[key] for key in MEDIA_BATCH_KEYS if key in encoded}
|
| 97 |
+
for key in MEDIA_TOKEN_KEYS:
|
| 98 |
+
if key in encoded:
|
| 99 |
+
media[key] = encoded[key][0].tolist()
|
| 100 |
+
return encoded["input_ids"][0].tolist(), media
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def encode_record(
|
| 104 |
+
tokenizer: Any,
|
| 105 |
+
record: dict[str, Any],
|
| 106 |
+
max_length: int = 16384,
|
| 107 |
+
max_state_tokens: int | None = None,
|
| 108 |
+
processor: Any | None = None,
|
| 109 |
+
) -> EncodedRecord:
|
| 110 |
+
schema_ids = _tokens(tokenizer, "\n\nSCHEMA FIELDS:\n")
|
| 111 |
+
questions: list[EncodedQuestion] = []
|
| 112 |
+
for question_index, (question_id, question) in enumerate(record["questions"].items()):
|
| 113 |
+
schema_ids.extend(
|
| 114 |
+
_tokens(
|
| 115 |
+
tokenizer,
|
| 116 |
+
f"\nFIELD {question_index + 1}\nID: {question_id}\nTYPE: {question['type']}\nINSTRUCTION: ",
|
| 117 |
+
)
|
| 118 |
+
)
|
| 119 |
+
question_start = len(schema_ids)
|
| 120 |
+
instructions = question.get("instructions")
|
| 121 |
+
if instructions is None or instructions == "":
|
| 122 |
+
instructions = str(question_id)
|
| 123 |
+
schema_ids.extend(_tokens(tokenizer, render(instructions)))
|
| 124 |
+
question_end = len(schema_ids)
|
| 125 |
+
schema_ids.extend(_tokens(tokenizer, "\nALLOWED OPTIONS:\n"))
|
| 126 |
+
|
| 127 |
+
option_spans: list[tuple[int, int]] = []
|
| 128 |
+
option_ids: list[str] = []
|
| 129 |
+
for option_index, (option_id, description) in enumerate(question_options(question)):
|
| 130 |
+
schema_ids.extend(_tokens(tokenizer, f"OPTION {option_index + 1}: "))
|
| 131 |
+
option_start = len(schema_ids)
|
| 132 |
+
semantics = {"option_id": option_id}
|
| 133 |
+
if description is not None:
|
| 134 |
+
semantics["description"] = description
|
| 135 |
+
schema_ids.extend(_tokens(tokenizer, render(semantics)))
|
| 136 |
+
option_spans.append((option_start, len(schema_ids)))
|
| 137 |
+
option_ids.append(option_id)
|
| 138 |
+
schema_ids.extend(_tokens(tokenizer, "\n"))
|
| 139 |
+
schema_ids.extend(_tokens(tokenizer, "END FIELD\n"))
|
| 140 |
+
questions.append(
|
| 141 |
+
EncodedQuestion(
|
| 142 |
+
question_id=str(question_id),
|
| 143 |
+
question_type=QUESTION_TYPES[str(question["type"])],
|
| 144 |
+
question_span=(question_start, question_end),
|
| 145 |
+
option_spans=tuple(option_spans),
|
| 146 |
+
option_ids=tuple(option_ids),
|
| 147 |
+
)
|
| 148 |
+
)
|
| 149 |
+
|
| 150 |
+
prefix_ids = _tokens(
|
| 151 |
+
tokenizer,
|
| 152 |
+
f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n<|im_start|>user\nSTATE:\n",
|
| 153 |
+
)
|
| 154 |
+
suffix_ids = _tokens(
|
| 155 |
+
tokenizer,
|
| 156 |
+
"\n<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nJOINT SCHEMA DECISIONS:",
|
| 157 |
+
)
|
| 158 |
+
media_ids, media = _encode_media(processor, record)
|
| 159 |
+
if media is not None:
|
| 160 |
+
media["token_offset"] = len(prefix_ids)
|
| 161 |
+
prefix_ids = prefix_ids + media_ids
|
| 162 |
+
state_ids = _tokens(tokenizer, render(record["state"]))
|
| 163 |
+
if max_state_tokens is not None:
|
| 164 |
+
state_ids = state_ids[:max_state_tokens]
|
| 165 |
+
fixed_length = len(prefix_ids) + len(schema_ids) + len(suffix_ids)
|
| 166 |
+
if fixed_length > max_length:
|
| 167 |
+
raise ValueError(
|
| 168 |
+
f"schema requires {fixed_length} tokens before state; maximum is {max_length}"
|
| 169 |
+
)
|
| 170 |
+
state_ids = state_ids[: max_length - fixed_length]
|
| 171 |
+
schema_offset = len(prefix_ids) + len(state_ids)
|
| 172 |
+
shifted_questions = tuple(
|
| 173 |
+
EncodedQuestion(
|
| 174 |
+
question_id=question.question_id,
|
| 175 |
+
question_type=question.question_type,
|
| 176 |
+
question_span=(
|
| 177 |
+
question.question_span[0] + schema_offset,
|
| 178 |
+
question.question_span[1] + schema_offset,
|
| 179 |
+
),
|
| 180 |
+
option_spans=tuple(
|
| 181 |
+
(start + schema_offset, end + schema_offset)
|
| 182 |
+
for start, end in question.option_spans
|
| 183 |
+
),
|
| 184 |
+
option_ids=question.option_ids,
|
| 185 |
+
)
|
| 186 |
+
for question in questions
|
| 187 |
+
)
|
| 188 |
+
input_ids = tuple(prefix_ids + state_ids + schema_ids + suffix_ids)
|
| 189 |
+
if not input_ids or not shifted_questions:
|
| 190 |
+
raise ValueError("record produced no model input or questions")
|
| 191 |
+
return EncodedRecord(
|
| 192 |
+
input_ids=input_ids,
|
| 193 |
+
questions=shifted_questions,
|
| 194 |
+
record_id=str(record.get("id", "unknown")),
|
| 195 |
+
media=media,
|
| 196 |
+
)
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
def collate_records(
|
| 200 |
+
records: list[EncodedRecord],
|
| 201 |
+
pad_token_id: int,
|
| 202 |
+
device: torch.device,
|
| 203 |
+
) -> dict[str, Any]:
|
| 204 |
+
maximum_length = max(len(record.input_ids) for record in records)
|
| 205 |
+
input_ids = torch.full(
|
| 206 |
+
(len(records), maximum_length),
|
| 207 |
+
pad_token_id,
|
| 208 |
+
dtype=torch.long,
|
| 209 |
+
device=device,
|
| 210 |
+
)
|
| 211 |
+
attention_mask = torch.zeros(
|
| 212 |
+
(len(records), maximum_length),
|
| 213 |
+
dtype=torch.long,
|
| 214 |
+
device=device,
|
| 215 |
+
)
|
| 216 |
+
for index, record in enumerate(records):
|
| 217 |
+
length = len(record.input_ids)
|
| 218 |
+
input_ids[index, :length] = torch.tensor(record.input_ids, device=device)
|
| 219 |
+
attention_mask[index, :length] = 1
|
| 220 |
+
media: dict[str, torch.Tensor] = {}
|
| 221 |
+
for key in MEDIA_BATCH_KEYS:
|
| 222 |
+
values = [record.media[key] for record in records if record.media and key in record.media]
|
| 223 |
+
if values:
|
| 224 |
+
media[key] = torch.cat(values, dim=0).to(device)
|
| 225 |
+
for key in MEDIA_TOKEN_KEYS:
|
| 226 |
+
if any(record.media and key in record.media for record in records):
|
| 227 |
+
token_values = torch.zeros((len(records), maximum_length), dtype=torch.long, device=device)
|
| 228 |
+
for index, record in enumerate(records):
|
| 229 |
+
if record.media and key in record.media:
|
| 230 |
+
offset = record.media["token_offset"]
|
| 231 |
+
values = torch.tensor(record.media[key], dtype=torch.long, device=device)
|
| 232 |
+
token_values[index, offset : offset + len(values)] = values
|
| 233 |
+
media[key] = token_values
|
| 234 |
+
return {
|
| 235 |
+
"input_ids": input_ids,
|
| 236 |
+
"attention_mask": attention_mask,
|
| 237 |
+
"records": records,
|
| 238 |
+
"media": media,
|
| 239 |
+
}
|
| 240 |
+
|
| 241 |
+
|
| 242 |
+
class EvidenceRoutingLayer(torch.nn.Module):
|
| 243 |
+
def __init__(
|
| 244 |
+
self,
|
| 245 |
+
width: int,
|
| 246 |
+
heads: int,
|
| 247 |
+
feedforward: int,
|
| 248 |
+
dropout: float = 0.0,
|
| 249 |
+
) -> None:
|
| 250 |
+
super().__init__()
|
| 251 |
+
self.query_norm = torch.nn.LayerNorm(width)
|
| 252 |
+
self.memory_norm = torch.nn.LayerNorm(width)
|
| 253 |
+
self.attention = torch.nn.MultiheadAttention(
|
| 254 |
+
width,
|
| 255 |
+
heads,
|
| 256 |
+
dropout=dropout,
|
| 257 |
+
batch_first=True,
|
| 258 |
+
)
|
| 259 |
+
self.attention_dropout = torch.nn.Dropout(dropout)
|
| 260 |
+
self.feedforward_norm = torch.nn.LayerNorm(width)
|
| 261 |
+
self.feedforward = torch.nn.Sequential(
|
| 262 |
+
torch.nn.Linear(width, feedforward),
|
| 263 |
+
torch.nn.GELU(),
|
| 264 |
+
torch.nn.Dropout(dropout),
|
| 265 |
+
torch.nn.Linear(feedforward, width),
|
| 266 |
+
torch.nn.Dropout(dropout),
|
| 267 |
+
)
|
| 268 |
+
|
| 269 |
+
def forward(self, queries: torch.Tensor, memory: torch.Tensor) -> torch.Tensor:
|
| 270 |
+
normalized_queries = self.query_norm(queries)
|
| 271 |
+
routed, _ = self.attention(
|
| 272 |
+
normalized_queries,
|
| 273 |
+
self.memory_norm(memory),
|
| 274 |
+
self.memory_norm(memory),
|
| 275 |
+
need_weights=False,
|
| 276 |
+
)
|
| 277 |
+
queries = queries + self.attention_dropout(routed)
|
| 278 |
+
return queries + self.feedforward(self.feedforward_norm(queries))
|
| 279 |
+
|
| 280 |
+
|
| 281 |
+
class JointSchemaHead(torch.nn.Module):
|
| 282 |
+
def __init__(
|
| 283 |
+
self,
|
| 284 |
+
hidden_size: int,
|
| 285 |
+
width: int,
|
| 286 |
+
routing_layers: int,
|
| 287 |
+
layers: int,
|
| 288 |
+
heads: int,
|
| 289 |
+
feedforward: int,
|
| 290 |
+
dropout: float = 0.0,
|
| 291 |
+
) -> None:
|
| 292 |
+
super().__init__()
|
| 293 |
+
self.hidden_norm = torch.nn.LayerNorm(hidden_size)
|
| 294 |
+
self.memory_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 295 |
+
self.question_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 296 |
+
self.option_question_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 297 |
+
self.global_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 298 |
+
self.option_context_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 299 |
+
self.option_lexical_projection = torch.nn.Linear(hidden_size, width, bias=False)
|
| 300 |
+
self.type_embedding = torch.nn.Embedding(3, width)
|
| 301 |
+
self.evidence_layers = torch.nn.ModuleList(
|
| 302 |
+
[
|
| 303 |
+
EvidenceRoutingLayer(
|
| 304 |
+
width=width,
|
| 305 |
+
heads=heads,
|
| 306 |
+
feedforward=feedforward,
|
| 307 |
+
dropout=dropout,
|
| 308 |
+
)
|
| 309 |
+
for _ in range(routing_layers)
|
| 310 |
+
]
|
| 311 |
+
)
|
| 312 |
+
self.option_summary_norm = torch.nn.LayerNorm(width)
|
| 313 |
+
self.layers = torch.nn.ModuleList(
|
| 314 |
+
[
|
| 315 |
+
torch.nn.TransformerDecoderLayer(
|
| 316 |
+
d_model=width,
|
| 317 |
+
nhead=heads,
|
| 318 |
+
dim_feedforward=feedforward,
|
| 319 |
+
dropout=dropout,
|
| 320 |
+
activation="gelu",
|
| 321 |
+
batch_first=True,
|
| 322 |
+
norm_first=True,
|
| 323 |
+
)
|
| 324 |
+
for _ in range(layers)
|
| 325 |
+
]
|
| 326 |
+
)
|
| 327 |
+
self.field_norm = torch.nn.LayerNorm(width)
|
| 328 |
+
self.option_norm = torch.nn.LayerNorm(width)
|
| 329 |
+
self.residual_scorer = torch.nn.Sequential(
|
| 330 |
+
torch.nn.Linear(width * 4, width),
|
| 331 |
+
torch.nn.GELU(),
|
| 332 |
+
torch.nn.Dropout(dropout),
|
| 333 |
+
torch.nn.Linear(width, 1),
|
| 334 |
+
)
|
| 335 |
+
self.prior_logit_scale = torch.nn.Parameter(torch.zeros(()))
|
| 336 |
+
self.joint_logit_scale = torch.nn.Parameter(torch.zeros(()))
|
| 337 |
+
self.residual_gate = torch.nn.Parameter(torch.zeros(()))
|
| 338 |
+
|
| 339 |
+
@staticmethod
|
| 340 |
+
def _mean_span(values: torch.Tensor, span: tuple[int, int]) -> torch.Tensor:
|
| 341 |
+
start, end = span
|
| 342 |
+
return values[start:end].mean(dim=0)
|
| 343 |
+
|
| 344 |
+
def forward(
|
| 345 |
+
self,
|
| 346 |
+
hidden_states: torch.Tensor,
|
| 347 |
+
input_ids: torch.Tensor,
|
| 348 |
+
attention_mask: torch.Tensor,
|
| 349 |
+
records: list[EncodedRecord],
|
| 350 |
+
output_embedding_weight: torch.Tensor,
|
| 351 |
+
) -> list[list[torch.Tensor]]:
|
| 352 |
+
results: list[list[torch.Tensor]] = []
|
| 353 |
+
normalized_hidden = self.hidden_norm(hidden_states)
|
| 354 |
+
for batch_index, record in enumerate(records):
|
| 355 |
+
sequence_length = int(attention_mask[batch_index].sum().item())
|
| 356 |
+
sequence_hidden = normalized_hidden[batch_index, :sequence_length]
|
| 357 |
+
memory = self.memory_projection(sequence_hidden).unsqueeze(0)
|
| 358 |
+
global_vector = sequence_hidden[-1]
|
| 359 |
+
question_vectors = torch.stack(
|
| 360 |
+
[
|
| 361 |
+
self._mean_span(sequence_hidden, question.question_span)
|
| 362 |
+
for question in record.questions
|
| 363 |
+
]
|
| 364 |
+
)
|
| 365 |
+
type_ids = torch.tensor(
|
| 366 |
+
[question.question_type for question in record.questions],
|
| 367 |
+
device=hidden_states.device,
|
| 368 |
+
)
|
| 369 |
+
option_contexts: list[torch.Tensor] = []
|
| 370 |
+
lexical_options: list[torch.Tensor] = []
|
| 371 |
+
option_counts = []
|
| 372 |
+
for question in record.questions:
|
| 373 |
+
context_vectors = torch.stack(
|
| 374 |
+
[
|
| 375 |
+
self._mean_span(sequence_hidden, span)
|
| 376 |
+
for span in question.option_spans
|
| 377 |
+
]
|
| 378 |
+
)
|
| 379 |
+
lexical_vectors = []
|
| 380 |
+
for start, end in question.option_spans:
|
| 381 |
+
token_ids = input_ids[batch_index, start:end]
|
| 382 |
+
lexical_vectors.append(output_embedding_weight[token_ids].mean(dim=0))
|
| 383 |
+
lexical = torch.stack(lexical_vectors)
|
| 384 |
+
option_contexts.append(context_vectors)
|
| 385 |
+
lexical_options.append(lexical)
|
| 386 |
+
option_counts.append(len(question.option_spans))
|
| 387 |
+
|
| 388 |
+
option_queries = []
|
| 389 |
+
for question_index, (context_vectors, lexical) in enumerate(
|
| 390 |
+
zip(option_contexts, lexical_options)
|
| 391 |
+
):
|
| 392 |
+
option_queries.append(
|
| 393 |
+
self.option_context_projection(context_vectors)
|
| 394 |
+
+ self.option_lexical_projection(lexical)
|
| 395 |
+
+ self.option_question_projection(
|
| 396 |
+
question_vectors[question_index]
|
| 397 |
+
).unsqueeze(0)
|
| 398 |
+
)
|
| 399 |
+
routed_options = torch.cat(option_queries, dim=0).unsqueeze(0)
|
| 400 |
+
for layer in self.evidence_layers:
|
| 401 |
+
routed_options = layer(routed_options, memory)
|
| 402 |
+
routed_options = routed_options[0]
|
| 403 |
+
split_options = list(torch.split(routed_options, option_counts, dim=0))
|
| 404 |
+
|
| 405 |
+
base_fields = self.question_projection(question_vectors)
|
| 406 |
+
option_summaries = []
|
| 407 |
+
for field, options in zip(base_fields, split_options):
|
| 408 |
+
routing_weights = torch.softmax(
|
| 409 |
+
torch.matmul(options, field) / math.sqrt(options.shape[-1]),
|
| 410 |
+
dim=0,
|
| 411 |
+
)
|
| 412 |
+
option_summaries.append(
|
| 413 |
+
torch.sum(routing_weights.unsqueeze(-1) * options, dim=0)
|
| 414 |
+
)
|
| 415 |
+
fields = (
|
| 416 |
+
base_fields
|
| 417 |
+
+ self.option_summary_norm(torch.stack(option_summaries))
|
| 418 |
+
+ self.global_projection(global_vector).unsqueeze(0)
|
| 419 |
+
+ self.type_embedding(type_ids)
|
| 420 |
+
)
|
| 421 |
+
fields = fields.unsqueeze(0)
|
| 422 |
+
for layer in self.layers:
|
| 423 |
+
fields = layer(fields, memory)
|
| 424 |
+
fields = self.field_norm(fields[0])
|
| 425 |
+
|
| 426 |
+
record_logits: list[torch.Tensor] = []
|
| 427 |
+
for field, question, lexical, routed in zip(
|
| 428 |
+
fields,
|
| 429 |
+
record.questions,
|
| 430 |
+
lexical_options,
|
| 431 |
+
split_options,
|
| 432 |
+
):
|
| 433 |
+
anchor = functional.normalize(
|
| 434 |
+
question_vectors[len(record_logits)] + global_vector,
|
| 435 |
+
dim=-1,
|
| 436 |
+
)
|
| 437 |
+
lexical_anchor = functional.normalize(lexical, dim=-1)
|
| 438 |
+
prior_scale = self.prior_logit_scale.clamp(max=math.log(100.0)).exp()
|
| 439 |
+
prior = prior_scale * torch.matmul(lexical_anchor, anchor)
|
| 440 |
+
options = self.option_norm(routed)
|
| 441 |
+
repeated_field = field.unsqueeze(0).expand_as(options)
|
| 442 |
+
cosine = functional.cosine_similarity(repeated_field, options, dim=-1)
|
| 443 |
+
features = torch.cat(
|
| 444 |
+
[
|
| 445 |
+
repeated_field,
|
| 446 |
+
options,
|
| 447 |
+
repeated_field * options,
|
| 448 |
+
torch.abs(repeated_field - options),
|
| 449 |
+
],
|
| 450 |
+
dim=-1,
|
| 451 |
+
)
|
| 452 |
+
residual = self.residual_scorer(features).squeeze(-1)
|
| 453 |
+
joint_scale = self.joint_logit_scale.clamp(max=math.log(100.0)).exp()
|
| 454 |
+
joint = joint_scale * cosine + residual
|
| 455 |
+
record_logits.append(
|
| 456 |
+
prior + torch.sigmoid(self.residual_gate) * joint
|
| 457 |
+
)
|
| 458 |
+
results.append(record_logits)
|
| 459 |
+
return results
|
| 460 |
+
|
| 461 |
+
|
| 462 |
+
class ClefModel(torch.nn.Module):
|
| 463 |
+
def __init__(self, language_model: Any, head: JointSchemaHead) -> None:
|
| 464 |
+
super().__init__()
|
| 465 |
+
self.language_model = language_model
|
| 466 |
+
self.head = head
|
| 467 |
+
|
| 468 |
+
def forward(self, batch: dict[str, Any]) -> list[list[torch.Tensor]]:
|
| 469 |
+
base_model = (
|
| 470 |
+
self.language_model.get_base_model()
|
| 471 |
+
if hasattr(self.language_model, "get_base_model")
|
| 472 |
+
else self.language_model
|
| 473 |
+
)
|
| 474 |
+
media = batch.get("media") or {}
|
| 475 |
+
text_model = base_model.model
|
| 476 |
+
if not media and hasattr(text_model, "language_model"):
|
| 477 |
+
text_model = text_model.language_model
|
| 478 |
+
outputs = text_model(
|
| 479 |
+
input_ids=batch["input_ids"],
|
| 480 |
+
attention_mask=batch["attention_mask"],
|
| 481 |
+
use_cache=False,
|
| 482 |
+
return_dict=True,
|
| 483 |
+
**media,
|
| 484 |
+
)
|
| 485 |
+
return self.head(
|
| 486 |
+
outputs.last_hidden_state,
|
| 487 |
+
batch["input_ids"],
|
| 488 |
+
batch["attention_mask"],
|
| 489 |
+
batch["records"],
|
| 490 |
+
base_model.get_output_embeddings().weight,
|
| 491 |
+
)
|
| 492 |
+
|
| 493 |
+
|
| 494 |
+
def load_release_model(
|
| 495 |
+
model_path: str | Path,
|
| 496 |
+
device: str | torch.device = "cuda",
|
| 497 |
+
dtype: torch.dtype = torch.bfloat16,
|
| 498 |
+
**from_pretrained_kwargs: Any,
|
| 499 |
+
) -> tuple[ClefModel, Any]:
|
| 500 |
+
"""Load a Clef release (merged backbone, joint schema head, and processor)."""
|
| 501 |
+
from huggingface_hub import snapshot_download
|
| 502 |
+
from safetensors.torch import load_file
|
| 503 |
+
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
|
| 504 |
+
|
| 505 |
+
path = Path(model_path)
|
| 506 |
+
if not path.is_dir():
|
| 507 |
+
path = Path(snapshot_download(str(model_path)))
|
| 508 |
+
backbone = Qwen3_5ForConditionalGeneration.from_pretrained(
|
| 509 |
+
path,
|
| 510 |
+
dtype=dtype,
|
| 511 |
+
device_map={"": str(device)},
|
| 512 |
+
**from_pretrained_kwargs,
|
| 513 |
+
)
|
| 514 |
+
backbone.config.use_cache = False
|
| 515 |
+
head_config = json.loads((path / "joint_head_config.json").read_text())
|
| 516 |
+
head = JointSchemaHead(**head_config)
|
| 517 |
+
head.load_state_dict(load_file(path / "joint_head.safetensors"), strict=True)
|
| 518 |
+
head = head.to(device=device, dtype=dtype)
|
| 519 |
+
processor = AutoProcessor.from_pretrained(path)
|
| 520 |
+
return ClefModel(backbone, head).eval(), processor
|
| 521 |
+
|
| 522 |
+
|
| 523 |
+
def systemone_answer(question: dict[str, Any], probabilities: dict[str, float]) -> dict[str, Any]:
|
| 524 |
+
"""Convert per-option probabilities for one question into a SystemOne answer."""
|
| 525 |
+
if question["type"] == "noul":
|
| 526 |
+
return {"type": "noul", "noul": round(probabilities["true"], 4)}
|
| 527 |
+
if question["type"] == "choice":
|
| 528 |
+
options = [str(option) for option in question["criteria"]]
|
| 529 |
+
choice = max(options, key=probabilities.__getitem__)
|
| 530 |
+
return {
|
| 531 |
+
"type": "choice",
|
| 532 |
+
"choice": choice,
|
| 533 |
+
"confidence": round(probabilities[choice], 4),
|
| 534 |
+
"probabilities": {option: round(probabilities[option], 4) for option in options},
|
| 535 |
+
}
|
| 536 |
+
levels = [str(index) for index in range(len(question["criteria"]))]
|
| 537 |
+
return {
|
| 538 |
+
"type": "score",
|
| 539 |
+
"score": round(sum(index * probabilities[level] for index, level in enumerate(levels)), 4),
|
| 540 |
+
"confidence": round(max(probabilities[level] for level in levels), 4),
|
| 541 |
+
"legend": dict(zip(levels, question["criteria"])),
|
| 542 |
+
"probabilities": {level: round(probabilities[level], 4) for level in levels},
|
| 543 |
+
}
|
| 544 |
+
|
| 545 |
+
|
| 546 |
+
@torch.inference_mode()
|
| 547 |
+
def systemone(model: ClefModel, processor: Any, request: dict[str, Any], max_length: int = 16384) -> dict[str, Any]:
|
| 548 |
+
"""Answer a Jev/SystemOne ``/v1/systemone`` request body with a SystemOne response body.
|
| 549 |
+
|
| 550 |
+
The request has ``model``, ``state``, and ``questions``, plus optional ``images`` and ``videos``.
|
| 551 |
+
"""
|
| 552 |
+
questions = request.get("questions")
|
| 553 |
+
if not isinstance(request.get("model"), str) or "state" not in request:
|
| 554 |
+
raise ValueError("model and state are required")
|
| 555 |
+
if not isinstance(questions, dict) or not questions:
|
| 556 |
+
raise ValueError("at least one question is required")
|
| 557 |
+
for question_id, question in questions.items():
|
| 558 |
+
if question.get("type") not in QUESTION_TYPES:
|
| 559 |
+
raise ValueError(f"{question_id}: type must be noul, choice, or score")
|
| 560 |
+
if question["type"] != "noul" and not question.get("criteria"):
|
| 561 |
+
raise ValueError(f"{question_id}: criteria must not be empty")
|
| 562 |
+
encoded = encode_record(processor.tokenizer, request, max_length=max_length, processor=processor)
|
| 563 |
+
device = next(model.parameters()).device
|
| 564 |
+
logits = model(collate_records([encoded], processor.tokenizer.pad_token_id, device))[0]
|
| 565 |
+
answers = {
|
| 566 |
+
question.question_id: systemone_answer(
|
| 567 |
+
questions[question.question_id],
|
| 568 |
+
dict(zip(question.option_ids, question_logits.float().softmax(-1).tolist())),
|
| 569 |
+
)
|
| 570 |
+
for question, question_logits in zip(encoded.questions, logits)
|
| 571 |
+
}
|
| 572 |
+
return {
|
| 573 |
+
"model": request["model"],
|
| 574 |
+
"answers": answers,
|
| 575 |
+
"usage": {"input_tokens": len(encoded.input_ids), "output_tokens": 0},
|
| 576 |
+
}
|
lm-head.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dd63614f1dc80dce2d83be3f3e69af1f8a0ebc8add9b8e0abf3910f563e44344
|
| 3 |
+
size 2034237568
|
model-00001-of-00005.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9c5483aa32a952b3a9a5d75f3c000c9ff934d3cbbb317b228cadff89e25bcffa
|
| 3 |
+
size 2034237592
|
model-00002-of-00005.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6999510bfc3c9ea6536090fb76b70bb786a0f4bd76f290ea35f35f58aa77e4f6
|
| 3 |
+
size 1984182833
|
model-00003-of-00005.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efa24fbcede25d84116de286e544554513bd0400c40c66b99c32e846f5096cdd
|
| 3 |
+
size 1983829880
|
model-00004-of-00005.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7a5dd174473310a090c74abe907de00da6b85c46f8a3b5bcaec20267449fbdbd
|
| 3 |
+
size 1983498867
|
model-00005-of-00005.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1463c85bac6efd374bb5189d3beda19df628a786a9068b00de74dd3504f12df9
|
| 3 |
+
size 1886989356
|
model.safetensors.index.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
processor_config.json
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"image_processor": {
|
| 3 |
+
"do_convert_rgb": true,
|
| 4 |
+
"do_normalize": true,
|
| 5 |
+
"do_rescale": true,
|
| 6 |
+
"do_resize": true,
|
| 7 |
+
"image_mean": [
|
| 8 |
+
0.5,
|
| 9 |
+
0.5,
|
| 10 |
+
0.5
|
| 11 |
+
],
|
| 12 |
+
"image_processor_type": "Qwen2VLImageProcessor",
|
| 13 |
+
"image_std": [
|
| 14 |
+
0.5,
|
| 15 |
+
0.5,
|
| 16 |
+
0.5
|
| 17 |
+
],
|
| 18 |
+
"merge_size": 2,
|
| 19 |
+
"patch_size": 16,
|
| 20 |
+
"resample": 3,
|
| 21 |
+
"rescale_factor": 0.00392156862745098,
|
| 22 |
+
"size": {
|
| 23 |
+
"longest_edge": 16777216,
|
| 24 |
+
"shortest_edge": 65536
|
| 25 |
+
},
|
| 26 |
+
"temporal_patch_size": 2
|
| 27 |
+
},
|
| 28 |
+
"processor_class": "Qwen3VLProcessor",
|
| 29 |
+
"video_processor": {
|
| 30 |
+
"do_convert_rgb": true,
|
| 31 |
+
"do_normalize": true,
|
| 32 |
+
"do_rescale": true,
|
| 33 |
+
"do_resize": true,
|
| 34 |
+
"do_sample_frames": true,
|
| 35 |
+
"fps": 2,
|
| 36 |
+
"image_mean": [
|
| 37 |
+
0.5,
|
| 38 |
+
0.5,
|
| 39 |
+
0.5
|
| 40 |
+
],
|
| 41 |
+
"image_std": [
|
| 42 |
+
0.5,
|
| 43 |
+
0.5,
|
| 44 |
+
0.5
|
| 45 |
+
],
|
| 46 |
+
"max_frames": 768,
|
| 47 |
+
"merge_size": 2,
|
| 48 |
+
"min_frames": 4,
|
| 49 |
+
"patch_size": 16,
|
| 50 |
+
"resample": 3,
|
| 51 |
+
"rescale_factor": 0.00392156862745098,
|
| 52 |
+
"return_metadata": false,
|
| 53 |
+
"size": {
|
| 54 |
+
"longest_edge": 25165824,
|
| 55 |
+
"shortest_edge": 4096
|
| 56 |
+
},
|
| 57 |
+
"temporal_patch_size": 2,
|
| 58 |
+
"video_processor_type": "Qwen3VLVideoProcessor"
|
| 59 |
+
}
|
| 60 |
+
}
|
requirements.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Isolated from Forge and the diffusion-model runtimes.
|
| 2 |
+
transformers==5.10.2
|
| 3 |
+
bitsandbytes==0.50.2
|
| 4 |
+
accelerate==1.15.0
|
| 5 |
+
huggingface_hub==1.5.0
|
| 6 |
+
safetensors==0.8.0
|
| 7 |
+
Pillow==12.3.0
|
| 8 |
+
torchvision==0.28.0
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
|
| 3 |
+
size 19989325
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"audio_bos_token": "<|audio_start|>",
|
| 4 |
+
"audio_eos_token": "<|audio_end|>",
|
| 5 |
+
"audio_token": "<|audio_pad|>",
|
| 6 |
+
"backend": "tokenizers",
|
| 7 |
+
"bos_token": null,
|
| 8 |
+
"clean_up_tokenization_spaces": false,
|
| 9 |
+
"eos_token": "<|im_end|>",
|
| 10 |
+
"errors": "replace",
|
| 11 |
+
"image_token": "<|image_pad|>",
|
| 12 |
+
"model_max_length": 262144,
|
| 13 |
+
"model_specific_special_tokens": {
|
| 14 |
+
"audio_bos_token": "<|audio_start|>",
|
| 15 |
+
"audio_eos_token": "<|audio_end|>",
|
| 16 |
+
"audio_token": "<|audio_pad|>",
|
| 17 |
+
"image_token": "<|image_pad|>",
|
| 18 |
+
"video_token": "<|video_pad|>",
|
| 19 |
+
"vision_bos_token": "<|vision_start|>",
|
| 20 |
+
"vision_eos_token": "<|vision_end|>"
|
| 21 |
+
},
|
| 22 |
+
"pad_token": "<|endoftext|>",
|
| 23 |
+
"pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
|
| 24 |
+
"split_special_tokens": false,
|
| 25 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 26 |
+
"unk_token": null,
|
| 27 |
+
"video_token": "<|video_pad|>",
|
| 28 |
+
"vision_bos_token": "<|vision_start|>",
|
| 29 |
+
"vision_eos_token": "<|vision_end|>"
|
| 30 |
+
}
|