Image Hopper initial release
Browse files- BASE-MODEL-LICENSE +201 -0
- Dockerfile +12 -0
- LICENSE +201 -0
- NOTICE +8 -0
- README.md +90 -0
- RELEASE-MANIFEST.json +54 -0
- adapter/README.md +207 -0
- adapter/adapter_config.json +56 -0
- adapter/adapter_model.safetensors +3 -0
- base-model-files.json +24 -0
- calibration.json +39 -0
- open_decisions/__init__.py +0 -0
- open_decisions/image_jev/__init__.py +0 -0
- open_decisions/image_jev/release/__init__.py +7 -0
- open_decisions/image_jev/release/predictor.py +586 -0
- open_decisions/image_jev/release/server.py +118 -0
- open_decisions/image_jev/release/smoke.py +223 -0
- open_decisions/image_jev/router.py +121 -0
- open_decisions/image_jev/speed.py +164 -0
- open_decisions/image_jev/vision.py +521 -0
- open_decisions/scoring/__init__.py +0 -0
- open_decisions/scoring/prompt.py +37 -0
- requirements.lock +93 -0
- requirements.lock.sha256 +1 -0
- requirements.txt +18 -0
BASE-MODEL-LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 6 |
+
|
| 7 |
+
1. Definitions.
|
| 8 |
+
|
| 9 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 10 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 11 |
+
|
| 12 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 13 |
+
the copyright owner that is granting the License.
|
| 14 |
+
|
| 15 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 16 |
+
other entities that control, are controlled by, or are under common
|
| 17 |
+
control with that entity. For the purposes of this definition,
|
| 18 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 19 |
+
direction or management of such entity, whether by contract or
|
| 20 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 21 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 22 |
+
|
| 23 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 24 |
+
exercising permissions granted by this License.
|
| 25 |
+
|
| 26 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 27 |
+
including but not limited to software source code, documentation
|
| 28 |
+
source, and configuration files.
|
| 29 |
+
|
| 30 |
+
"Object" form shall mean any form resulting from mechanical
|
| 31 |
+
transformation or translation of a Source form, including but
|
| 32 |
+
not limited to compiled object code, generated documentation,
|
| 33 |
+
and conversions to other media types.
|
| 34 |
+
|
| 35 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 36 |
+
Object form, made available under the License, as indicated by a
|
| 37 |
+
copyright notice that is included in or attached to the work
|
| 38 |
+
(an example is provided in the Appendix below).
|
| 39 |
+
|
| 40 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 41 |
+
form, that is based on (or derived from) the Work and for which the
|
| 42 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 43 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 44 |
+
of this License, Derivative Works shall not include works that remain
|
| 45 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 46 |
+
the Work and Derivative Works thereof.
|
| 47 |
+
|
| 48 |
+
"Contribution" shall mean any work of authorship, including
|
| 49 |
+
the original version of the Work and any modifications or additions
|
| 50 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 51 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 52 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 53 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 54 |
+
means any form of electronic, verbal, or written communication sent
|
| 55 |
+
to the Licensor or its representatives, including but not limited to
|
| 56 |
+
communication on electronic mailing lists, source code control systems,
|
| 57 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 58 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 59 |
+
excluding communication that is conspicuously marked or otherwise
|
| 60 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 61 |
+
|
| 62 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 63 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 64 |
+
subsequently incorporated within the Work.
|
| 65 |
+
|
| 66 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 67 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 68 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 69 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 70 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 71 |
+
Work and such Derivative Works in Source or Object form.
|
| 72 |
+
|
| 73 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 74 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 75 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 76 |
+
(except as stated in this section) patent license to make, have made,
|
| 77 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 78 |
+
where such license applies only to those patent claims licensable
|
| 79 |
+
by such Contributor that are necessarily infringed by their
|
| 80 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 81 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 82 |
+
institute patent litigation against any entity (including a
|
| 83 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 84 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 85 |
+
or contributory patent infringement, then any patent licenses
|
| 86 |
+
granted to You under this License for that Work shall terminate
|
| 87 |
+
as of the date such litigation is filed.
|
| 88 |
+
|
| 89 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 90 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 91 |
+
modifications, and in Source or Object form, provided that You
|
| 92 |
+
meet the following conditions:
|
| 93 |
+
|
| 94 |
+
(a) You must give any other recipients of the Work or
|
| 95 |
+
Derivative Works a copy of this License; and
|
| 96 |
+
|
| 97 |
+
(b) You must cause any modified files to carry prominent notices
|
| 98 |
+
stating that You changed the files; and
|
| 99 |
+
|
| 100 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 101 |
+
that You distribute, all copyright, patent, trademark, and
|
| 102 |
+
attribution notices from the Source form of the Work,
|
| 103 |
+
excluding those notices that do not pertain to any part of
|
| 104 |
+
the Derivative Works; and
|
| 105 |
+
|
| 106 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 107 |
+
distribution, then any Derivative Works that You distribute must
|
| 108 |
+
include a readable copy of the attribution notices contained
|
| 109 |
+
within such NOTICE file, excluding those notices that do not
|
| 110 |
+
pertain to any part of the Derivative Works, in at least one
|
| 111 |
+
of the following places: within a NOTICE text file distributed
|
| 112 |
+
as part of the Derivative Works; within the Source form or
|
| 113 |
+
documentation, if provided along with the Derivative Works; or,
|
| 114 |
+
within a display generated by the Derivative Works, if and
|
| 115 |
+
wherever such third-party notices normally appear. The contents
|
| 116 |
+
of the NOTICE file are for informational purposes only and
|
| 117 |
+
do not modify the License. You may add Your own attribution
|
| 118 |
+
notices within Derivative Works that You distribute, alongside
|
| 119 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 120 |
+
that such additional attribution notices cannot be construed
|
| 121 |
+
as modifying the License.
|
| 122 |
+
|
| 123 |
+
You may add Your own copyright statement to Your modifications and
|
| 124 |
+
may provide additional or different license terms and conditions
|
| 125 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 126 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 127 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 128 |
+
the conditions stated in this License.
|
| 129 |
+
|
| 130 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 131 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 132 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 133 |
+
this License, without any additional terms or conditions.
|
| 134 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 135 |
+
the terms of any separate license agreement you may have executed
|
| 136 |
+
with Licensor regarding such Contributions.
|
| 137 |
+
|
| 138 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 139 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 140 |
+
except as required for reasonable and customary use in describing the
|
| 141 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 142 |
+
|
| 143 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 144 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 145 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 146 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 147 |
+
implied, including, without limitation, any warranties or conditions
|
| 148 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 149 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 150 |
+
appropriateness of using or redistributing the Work and assume any
|
| 151 |
+
risks associated with Your exercise of permissions under this License.
|
| 152 |
+
|
| 153 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 154 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 155 |
+
unless required by applicable law (such as deliberate and grossly
|
| 156 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 157 |
+
liable to You for damages, including any direct, indirect, special,
|
| 158 |
+
incidental, or consequential damages of any character arising as a
|
| 159 |
+
result of this License or out of the use or inability to use the
|
| 160 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 161 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 162 |
+
other commercial damages or losses), even if such Contributor
|
| 163 |
+
has been advised of the possibility of such damages.
|
| 164 |
+
|
| 165 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 166 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 167 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 168 |
+
or other liability obligations and/or rights consistent with this
|
| 169 |
+
License. However, in accepting such obligations, You may act only
|
| 170 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 171 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 172 |
+
defend, and hold each Contributor harmless for any liability
|
| 173 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 174 |
+
of your accepting any such warranty or additional liability.
|
| 175 |
+
|
| 176 |
+
END OF TERMS AND CONDITIONS
|
| 177 |
+
|
| 178 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 179 |
+
|
| 180 |
+
To apply the Apache License to your work, attach the following
|
| 181 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 182 |
+
replaced with your own identifying information. (Don't include
|
| 183 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 184 |
+
comment syntax for the file format. We also recommend that a
|
| 185 |
+
file or class name and description of purpose be included on the
|
| 186 |
+
same "printed page" as the copyright notice for easier
|
| 187 |
+
identification within third-party archives.
|
| 188 |
+
|
| 189 |
+
Copyright [yyyy] [name of copyright owner]
|
| 190 |
+
|
| 191 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 192 |
+
you may not use this file except in compliance with the License.
|
| 193 |
+
You may obtain a copy of the License at
|
| 194 |
+
|
| 195 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 196 |
+
|
| 197 |
+
Unless required by applicable law or agreed to in writing, software
|
| 198 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 199 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 200 |
+
See the License for the specific language governing permissions and
|
| 201 |
+
limitations under the License.
|
Dockerfile
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CUDA 12.8.1 base matching torch 2.8.0+cu128, pinned by digest.
|
| 2 |
+
FROM nvidia/cuda:12.8.1-cudnn-devel-ubuntu24.04@sha256:24c8e3581ea6330038b0d374920721983312627f8adbfcf390bdb4b399d280ed
|
| 3 |
+
ENV DEBIAN_FRONTEND=noninteractive PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 TOKENIZERS_PARALLELISM=false HF_HOME=/models HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1
|
| 4 |
+
RUN apt-get update && apt-get install -y --no-install-recommends build-essential python3.12 python3.12-venv python3.12-dev libgl1 libglib2.0-0 && rm -rf /var/lib/apt/lists/*
|
| 5 |
+
RUN python3.12 -m venv /opt/venv
|
| 6 |
+
ENV PATH=/opt/venv/bin:$PATH VIRTUAL_ENV=/opt/venv
|
| 7 |
+
WORKDIR /app
|
| 8 |
+
COPY requirements.lock requirements.lock.sha256 /app/
|
| 9 |
+
RUN sha256sum -c requirements.lock.sha256 && python -m pip install --no-cache-dir -r requirements.lock
|
| 10 |
+
COPY . /app
|
| 11 |
+
EXPOSE 8080
|
| 12 |
+
ENTRYPOINT ["python", "-m", "open_decisions.image_jev.release.server", "--host", "0.0.0.0", "--port", "8080", "--offline", "--adapter", "adapter"]
|
LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 6 |
+
|
| 7 |
+
1. Definitions.
|
| 8 |
+
|
| 9 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 10 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 11 |
+
|
| 12 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 13 |
+
the copyright owner that is granting the License.
|
| 14 |
+
|
| 15 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 16 |
+
other entities that control, are controlled by, or are under common
|
| 17 |
+
control with that entity. For the purposes of this definition,
|
| 18 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 19 |
+
direction or management of such entity, whether by contract or
|
| 20 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 21 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 22 |
+
|
| 23 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 24 |
+
exercising permissions granted by this License.
|
| 25 |
+
|
| 26 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 27 |
+
including but not limited to software source code, documentation
|
| 28 |
+
source, and configuration files.
|
| 29 |
+
|
| 30 |
+
"Object" form shall mean any form resulting from mechanical
|
| 31 |
+
transformation or translation of a Source form, including but
|
| 32 |
+
not limited to compiled object code, generated documentation,
|
| 33 |
+
and conversions to other media types.
|
| 34 |
+
|
| 35 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 36 |
+
Object form, made available under the License, as indicated by a
|
| 37 |
+
copyright notice that is included in or attached to the work
|
| 38 |
+
(an example is provided in the Appendix below).
|
| 39 |
+
|
| 40 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 41 |
+
form, that is based on (or derived from) the Work and for which the
|
| 42 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 43 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 44 |
+
of this License, Derivative Works shall not include works that remain
|
| 45 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 46 |
+
the Work and Derivative Works thereof.
|
| 47 |
+
|
| 48 |
+
"Contribution" shall mean any work of authorship, including
|
| 49 |
+
the original version of the Work and any modifications or additions
|
| 50 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 51 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 52 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 53 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 54 |
+
means any form of electronic, verbal, or written communication sent
|
| 55 |
+
to the Licensor or its representatives, including but not limited to
|
| 56 |
+
communication on electronic mailing lists, source code control systems,
|
| 57 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 58 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 59 |
+
excluding communication that is conspicuously marked or otherwise
|
| 60 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 61 |
+
|
| 62 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 63 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 64 |
+
subsequently incorporated within the Work.
|
| 65 |
+
|
| 66 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 67 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 68 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 69 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 70 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 71 |
+
Work and such Derivative Works in Source or Object form.
|
| 72 |
+
|
| 73 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 74 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 75 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 76 |
+
(except as stated in this section) patent license to make, have made,
|
| 77 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 78 |
+
where such license applies only to those patent claims licensable
|
| 79 |
+
by such Contributor that are necessarily infringed by their
|
| 80 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 81 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 82 |
+
institute patent litigation against any entity (including a
|
| 83 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 84 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 85 |
+
or contributory patent infringement, then any patent licenses
|
| 86 |
+
granted to You under this License for that Work shall terminate
|
| 87 |
+
as of the date such litigation is filed.
|
| 88 |
+
|
| 89 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 90 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 91 |
+
modifications, and in Source or Object form, provided that You
|
| 92 |
+
meet the following conditions:
|
| 93 |
+
|
| 94 |
+
(a) You must give any other recipients of the Work or
|
| 95 |
+
Derivative Works a copy of this License; and
|
| 96 |
+
|
| 97 |
+
(b) You must cause any modified files to carry prominent notices
|
| 98 |
+
stating that You changed the files; and
|
| 99 |
+
|
| 100 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 101 |
+
that You distribute, all copyright, patent, trademark, and
|
| 102 |
+
attribution notices from the Source form of the Work,
|
| 103 |
+
excluding those notices that do not pertain to any part of
|
| 104 |
+
the Derivative Works; and
|
| 105 |
+
|
| 106 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 107 |
+
distribution, then any Derivative Works that You distribute must
|
| 108 |
+
include a readable copy of the attribution notices contained
|
| 109 |
+
within such NOTICE file, excluding those notices that do not
|
| 110 |
+
pertain to any part of the Derivative Works, in at least one
|
| 111 |
+
of the following places: within a NOTICE text file distributed
|
| 112 |
+
as part of the Derivative Works; within the Source form or
|
| 113 |
+
documentation, if provided along with the Derivative Works; or,
|
| 114 |
+
within a display generated by the Derivative Works, if and
|
| 115 |
+
wherever such third-party notices normally appear. The contents
|
| 116 |
+
of the NOTICE file are for informational purposes only and
|
| 117 |
+
do not modify the License. You may add Your own attribution
|
| 118 |
+
notices within Derivative Works that You distribute, alongside
|
| 119 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 120 |
+
that such additional attribution notices cannot be construed
|
| 121 |
+
as modifying the License.
|
| 122 |
+
|
| 123 |
+
You may add Your own copyright statement to Your modifications and
|
| 124 |
+
may provide additional or different license terms and conditions
|
| 125 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 126 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 127 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 128 |
+
the conditions stated in this License.
|
| 129 |
+
|
| 130 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 131 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 132 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 133 |
+
this License, without any additional terms or conditions.
|
| 134 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 135 |
+
the terms of any separate license agreement you may have executed
|
| 136 |
+
with Licensor regarding such Contributions.
|
| 137 |
+
|
| 138 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 139 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 140 |
+
except as required for reasonable and customary use in describing the
|
| 141 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 142 |
+
|
| 143 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 144 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 145 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 146 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 147 |
+
implied, including, without limitation, any warranties or conditions
|
| 148 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 149 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 150 |
+
appropriateness of using or redistributing the Work and assume any
|
| 151 |
+
risks associated with Your exercise of permissions under this License.
|
| 152 |
+
|
| 153 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 154 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 155 |
+
unless required by applicable law (such as deliberate and grossly
|
| 156 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 157 |
+
liable to You for damages, including any direct, indirect, special,
|
| 158 |
+
incidental, or consequential damages of any character arising as a
|
| 159 |
+
result of this License or out of the use or inability to use the
|
| 160 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 161 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 162 |
+
other commercial damages or losses), even if such Contributor
|
| 163 |
+
has been advised of the possibility of such damages.
|
| 164 |
+
|
| 165 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 166 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 167 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 168 |
+
or other liability obligations and/or rights consistent with this
|
| 169 |
+
License. However, in accepting such obligations, You may act only
|
| 170 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 171 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 172 |
+
defend, and hold each Contributor harmless for any liability
|
| 173 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 174 |
+
of your accepting any such warranty or additional liability.
|
| 175 |
+
|
| 176 |
+
END OF TERMS AND CONDITIONS
|
| 177 |
+
|
| 178 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 179 |
+
|
| 180 |
+
To apply the Apache License to your work, attach the following
|
| 181 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 182 |
+
replaced with your own identifying information. (Don't include
|
| 183 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 184 |
+
comment syntax for the file format. We also recommend that a
|
| 185 |
+
file or class name and description of purpose be included on the
|
| 186 |
+
same "printed page" as the copyright notice for easier
|
| 187 |
+
identification within third-party archives.
|
| 188 |
+
|
| 189 |
+
Copyright [yyyy] [name of copyright owner]
|
| 190 |
+
|
| 191 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 192 |
+
you may not use this file except in compliance with the License.
|
| 193 |
+
You may obtain a copy of the License at
|
| 194 |
+
|
| 195 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 196 |
+
|
| 197 |
+
Unless required by applicable law or agreed to in writing, software
|
| 198 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 199 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 200 |
+
See the License for the specific language governing permissions and
|
| 201 |
+
limitations under the License.
|
NOTICE
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Image Hopper
|
| 2 |
+
|
| 3 |
+
This release uses the Qwen/Qwen3.5-9B base model at the revision recorded in
|
| 4 |
+
RELEASE-MANIFEST.json. The base model is made available under the Apache
|
| 5 |
+
License, Version 2.0. Its weights are not redistributed in this bundle.
|
| 6 |
+
|
| 7 |
+
The Image Hopper server code and calibration artifact are made available under
|
| 8 |
+
the Apache License, Version 2.0.
|
README.md
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3.5-9B
|
| 4 |
+
pipeline_tag: image-text-to-text
|
| 5 |
+
tags:
|
| 6 |
+
- image-decisions
|
| 7 |
+
- calibration
|
| 8 |
+
- multiple-choice
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Image Hopper
|
| 12 |
+
|
| 13 |
+
Image Hopper answers multiple-choice and yes/no questions about images with a calibrated probability for every option,
|
| 14 |
+
in one forward pass and without generating text. It is built on `Qwen/Qwen3.5-9B`
|
| 15 |
+
(revision `c202236235762e1c871ad0ccb60c8ee5ba337b9a`) and is designed for the Image JevBench `/v1/systemone` contract.
|
| 16 |
+
|
| 17 |
+
## How it works
|
| 18 |
+
|
| 19 |
+
- **One base, two routes.** A fixed text rule (`router-rules/v1`) reads only the question and option text. Questions
|
| 20 |
+
about on-screen elements (markers, buttons, menus, windows, …) or geometry (angles, triangles, circles, lengths, …)
|
| 21 |
+
use a LoRA adapter; every other question uses the unmodified base model. Both routes share one loaded model and one
|
| 22 |
+
forward pass.
|
| 23 |
+
- **Readout.** Options are lettered A, B, C, …; the answer distribution is the FP32 softmax of the next-token logits
|
| 24 |
+
for those letters at the last prompt position.
|
| 25 |
+
- **Calibration.** Logits are divided by a temperature chosen by route and answer type (fitted by maximum likelihood
|
| 26 |
+
with shrinkage toward a global temperature). Calibration never changes which option ranks first.
|
| 27 |
+
- **Images** use the base processor's native resolution route; up to four images per request.
|
| 28 |
+
|
| 29 |
+
| Route / answer type | Temperature |
|
| 30 |
+
|---|---:|
|
| 31 |
+
| default / 4-option | 1.2338 |
|
| 32 |
+
| default / 5-marker | 1.9765 |
|
| 33 |
+
| default / yes/no | 1.3353 |
|
| 34 |
+
| screen_geometry / 4-option | 3.1396 |
|
| 35 |
+
| screen_geometry / 5-marker | 1.3292 |
|
| 36 |
+
| screen_geometry / yes/no | 0.6572 |
|
| 37 |
+
|
| 38 |
+
## Training and calibration data
|
| 39 |
+
|
| 40 |
+
The adapter (LoRA r64, one epoch, about 5,900 examples) was trained on real images with their datasets' own labels —
|
| 41 |
+
desktop screenshots from GroundCUA (training applications only), document pages from DocLayNet, industrial-defect
|
| 42 |
+
photos from VisA and openly licensed photo collections — plus original procedural renders (charts, documents, tables,
|
| 43 |
+
geometry, interface mock-ups). No benchmark items were used for training or calibration, and every training image was
|
| 44 |
+
checked against the benchmark's reference images for near-duplicates.
|
| 45 |
+
|
| 46 |
+
| Source | Licence of admitted material |
|
| 47 |
+
|---|---|
|
| 48 |
+
| GroundCUA (training applications) | MIT |
|
| 49 |
+
| DocLayNet | CDLA-Permissive-1.0 |
|
| 50 |
+
| Visual Anomaly (VisA) | CC BY 4.0 |
|
| 51 |
+
| Amazon Berkeley Objects | CC BY 4.0 |
|
| 52 |
+
| Public Domain 12M | CDLA-Permissive-2.0 metadata; admitted images public domain or CC0 |
|
| 53 |
+
| AgriFreshNET | CC BY 4.0 |
|
| 54 |
+
| SHARD shelf-management dataset | CC BY 4.0 |
|
| 55 |
+
| PHELE physical-hazards dataset | CC BY 4.0 |
|
| 56 |
+
| Original procedural renders | Apache-2.0 (ours) |
|
| 57 |
+
|
| 58 |
+
The default-route temperatures were fitted on a held-out calibration split of our own data; the screen/geometry
|
| 59 |
+
temperatures on held-out real screenshots and geometry diagrams that were never used for training.
|
| 60 |
+
|
| 61 |
+
## Usage
|
| 62 |
+
|
| 63 |
+
pip install -r requirements.txt
|
| 64 |
+
python -m open_decisions.image_jev.release.server --offline --adapter adapter --port 8080
|
| 65 |
+
|
| 66 |
+
`POST /v1/systemone`:
|
| 67 |
+
|
| 68 |
+
{"images": ["data:image/png;base64,..."],
|
| 69 |
+
"questions": {"decision": {"type": "choice", "instructions": "Which shelf is empty?",
|
| 70 |
+
"criteria": {"top": "Top shelf", "bottom": "Bottom shelf"}}}}
|
| 71 |
+
|
| 72 |
+
The response has `model`, token `usage` and `answers`; a `choice` answer holds the selected option and a probability
|
| 73 |
+
per option, a `noul` answer holds P(yes).
|
| 74 |
+
|
| 75 |
+
## Evaluation
|
| 76 |
+
|
| 77 |
+
Image JevBench results will be added here once the benchmark maintainer publishes them.
|
| 78 |
+
|
| 79 |
+
## Intended use and limitations
|
| 80 |
+
|
| 81 |
+
For research and evaluation of image-grounded decisions where callers supply the options. Not a safety system or a
|
| 82 |
+
substitute for human review in high-impact decisions. Weak at dense counting; the adapter route is specialised for
|
| 83 |
+
screens and geometry; probabilities cover only the supplied options (no abstain answer); calibration reflects our
|
| 84 |
+
data mix and may transfer imperfectly; evaluated on English questions with thinking mode off. Native-resolution cost
|
| 85 |
+
and latency grow with image size.
|
| 86 |
+
|
| 87 |
+
## Licence
|
| 88 |
+
|
| 89 |
+
Code, adapter and calibration: Apache-2.0. The base model `Qwen/Qwen3.5-9B` is Apache-2.0 and is not redistributed
|
| 90 |
+
here. See `LICENSE`, `BASE-MODEL-LICENSE` and `NOTICE`; third-party data licences as listed above.
|
RELEASE-MANIFEST.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"adapter": {
|
| 3 |
+
"path": "adapter",
|
| 4 |
+
"tree_sha256": "1e87a9966f77622d9b70b874364adc7ac1990b78735b5e08168d68106715d341"
|
| 5 |
+
},
|
| 6 |
+
"base_model": {
|
| 7 |
+
"fine_tuned": true,
|
| 8 |
+
"repo": "Qwen/Qwen3.5-9B",
|
| 9 |
+
"revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a"
|
| 10 |
+
},
|
| 11 |
+
"calibration": {
|
| 12 |
+
"artifact_sha256": "fc29b120cf2ac00d1917c3712fcd2293b5be3c4746de7d4640b41c3cce251501",
|
| 13 |
+
"fit": null,
|
| 14 |
+
"identity_sha256": "9ca8772bb7207d3dbbedc42f7cbd933f30fe7c2367c682e5a01acd2bf8303c78"
|
| 15 |
+
},
|
| 16 |
+
"code": {
|
| 17 |
+
"dirty": false,
|
| 18 |
+
"git_commit": "b94f0c4507c2463f6569974cfbb36a05ccdbb764"
|
| 19 |
+
},
|
| 20 |
+
"files": {
|
| 21 |
+
"BASE-MODEL-LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
|
| 22 |
+
"Dockerfile": "904c4630d2b50f577e3e6cf0e9d71f25c06e4fb64ef1a41238b299ba0c8a5d7a",
|
| 23 |
+
"LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
|
| 24 |
+
"NOTICE": "1d5e63ea05ed3c846593f1f4185b4355a25b2b25682bccac24995e094141b6c7",
|
| 25 |
+
"README.md": "2b1b313bcf80dff7aaeea5904c82d3adec8e0c8079a42a28093f7538c794b8b9",
|
| 26 |
+
"adapter/README.md": "bb29528044c9409e183ce924e4ea6f29f2b705d749287781b8ff2777b5209a55",
|
| 27 |
+
"adapter/adapter_config.json": "4f6e55c1d151070a6e8444feff44bdcfbe4b87e14c3fde855f69159066162487",
|
| 28 |
+
"adapter/adapter_model.safetensors": "6201fcb1c08144e465a03fd6f5355c4394b3bffaad659440fa64063dd4f71509",
|
| 29 |
+
"base-model-files.json": "d5009e4769f367557296b7b233824b3045e3cebee1a703f6419f2ff23ad53318",
|
| 30 |
+
"calibration.json": "bfb674b8ae7b2ba5293fd7c87706d1628d0455505794091d3e50d32a5378e685",
|
| 31 |
+
"open_decisions/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 32 |
+
"open_decisions/image_jev/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 33 |
+
"open_decisions/image_jev/release/__init__.py": "4a29d6b23b6021c0cc27d3fa5163ebbede7ab531b1a13814dfddf1fedcd71af4",
|
| 34 |
+
"open_decisions/image_jev/release/predictor.py": "b4a469714f2cfb5b6cf8551975dc76e2b6139b6dae61b1ebcc4f31335dddb407",
|
| 35 |
+
"open_decisions/image_jev/release/server.py": "3815ab07ca929b9e226b0a73521b2dc6cc6c426da1d0d8942ab9e372163f3ca9",
|
| 36 |
+
"open_decisions/image_jev/release/smoke.py": "80c6ed250089724a332ea0341ad6324757b19a5407d219f539d28bcdf7864ea4",
|
| 37 |
+
"open_decisions/image_jev/router.py": "e99db6dde898e7f7be4babcec2f4e545eedff51b04db8c7fe98e47e884a27c47",
|
| 38 |
+
"open_decisions/image_jev/speed.py": "5671a56aa16d4647dca8f3c360e83f98747641d7722ae184110806375448c5cb",
|
| 39 |
+
"open_decisions/image_jev/vision.py": "f9e988cae5184b09b0ebacec7f9baeb3db06e971862c18229dea1ee5034e2be2",
|
| 40 |
+
"open_decisions/scoring/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 41 |
+
"open_decisions/scoring/prompt.py": "131916c014979500e5d2daceb94817a8942de8148966d3309c8ecbc8547027ee",
|
| 42 |
+
"requirements.lock": "974e3a19773d67b4de84842ffa4a3a537e2886240cdfb97d94216159804aa7e2",
|
| 43 |
+
"requirements.lock.sha256": "e2c447dea2405d458599f08d91d6d8ae774f85b7074de331a431a1ef43b7829a",
|
| 44 |
+
"requirements.txt": "321bb81a1bd43b4858cf724d22944ac3e5e93504a9bf231be88b6752b437a63c"
|
| 45 |
+
},
|
| 46 |
+
"image_route": "native",
|
| 47 |
+
"manifest_sha256": "1403eee576ec886cf6c3bcd2dbb02761fd9cd9200af118d2fff05da6eb4ecbd3",
|
| 48 |
+
"measurement_artifact": null,
|
| 49 |
+
"name": "HopitAI/image-hopper",
|
| 50 |
+
"scorer": "one-prefill-option-letter",
|
| 51 |
+
"status": "owner-review",
|
| 52 |
+
"system_kind": "routed",
|
| 53 |
+
"version": "image-jev-release/v1"
|
| 54 |
+
}
|
adapter/README.md
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: Qwen/Qwen3.5-9B
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
tags:
|
| 6 |
+
- base_model:adapter:Qwen/Qwen3.5-9B
|
| 7 |
+
- lora
|
| 8 |
+
- transformers
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for Model ID
|
| 12 |
+
|
| 13 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Model Details
|
| 18 |
+
|
| 19 |
+
### Model Description
|
| 20 |
+
|
| 21 |
+
<!-- Provide a longer summary of what this model is. -->
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
- **Developed by:** [More Information Needed]
|
| 26 |
+
- **Funded by [optional]:** [More Information Needed]
|
| 27 |
+
- **Shared by [optional]:** [More Information Needed]
|
| 28 |
+
- **Model type:** [More Information Needed]
|
| 29 |
+
- **Language(s) (NLP):** [More Information Needed]
|
| 30 |
+
- **License:** [More Information Needed]
|
| 31 |
+
- **Finetuned from model [optional]:** [More Information Needed]
|
| 32 |
+
|
| 33 |
+
### Model Sources [optional]
|
| 34 |
+
|
| 35 |
+
<!-- Provide the basic links for the model. -->
|
| 36 |
+
|
| 37 |
+
- **Repository:** [More Information Needed]
|
| 38 |
+
- **Paper [optional]:** [More Information Needed]
|
| 39 |
+
- **Demo [optional]:** [More Information Needed]
|
| 40 |
+
|
| 41 |
+
## Uses
|
| 42 |
+
|
| 43 |
+
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 44 |
+
|
| 45 |
+
### Direct Use
|
| 46 |
+
|
| 47 |
+
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
| 48 |
+
|
| 49 |
+
[More Information Needed]
|
| 50 |
+
|
| 51 |
+
### Downstream Use [optional]
|
| 52 |
+
|
| 53 |
+
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 54 |
+
|
| 55 |
+
[More Information Needed]
|
| 56 |
+
|
| 57 |
+
### Out-of-Scope Use
|
| 58 |
+
|
| 59 |
+
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
| 60 |
+
|
| 61 |
+
[More Information Needed]
|
| 62 |
+
|
| 63 |
+
## Bias, Risks, and Limitations
|
| 64 |
+
|
| 65 |
+
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
| 66 |
+
|
| 67 |
+
[More Information Needed]
|
| 68 |
+
|
| 69 |
+
### Recommendations
|
| 70 |
+
|
| 71 |
+
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
| 72 |
+
|
| 73 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 74 |
+
|
| 75 |
+
## How to Get Started with the Model
|
| 76 |
+
|
| 77 |
+
Use the code below to get started with the model.
|
| 78 |
+
|
| 79 |
+
[More Information Needed]
|
| 80 |
+
|
| 81 |
+
## Training Details
|
| 82 |
+
|
| 83 |
+
### Training Data
|
| 84 |
+
|
| 85 |
+
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 86 |
+
|
| 87 |
+
[More Information Needed]
|
| 88 |
+
|
| 89 |
+
### Training Procedure
|
| 90 |
+
|
| 91 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 92 |
+
|
| 93 |
+
#### Preprocessing [optional]
|
| 94 |
+
|
| 95 |
+
[More Information Needed]
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
#### Training Hyperparameters
|
| 99 |
+
|
| 100 |
+
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 101 |
+
|
| 102 |
+
#### Speeds, Sizes, Times [optional]
|
| 103 |
+
|
| 104 |
+
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 105 |
+
|
| 106 |
+
[More Information Needed]
|
| 107 |
+
|
| 108 |
+
## Evaluation
|
| 109 |
+
|
| 110 |
+
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 111 |
+
|
| 112 |
+
### Testing Data, Factors & Metrics
|
| 113 |
+
|
| 114 |
+
#### Testing Data
|
| 115 |
+
|
| 116 |
+
<!-- This should link to a Dataset Card if possible. -->
|
| 117 |
+
|
| 118 |
+
[More Information Needed]
|
| 119 |
+
|
| 120 |
+
#### Factors
|
| 121 |
+
|
| 122 |
+
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 123 |
+
|
| 124 |
+
[More Information Needed]
|
| 125 |
+
|
| 126 |
+
#### Metrics
|
| 127 |
+
|
| 128 |
+
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 129 |
+
|
| 130 |
+
[More Information Needed]
|
| 131 |
+
|
| 132 |
+
### Results
|
| 133 |
+
|
| 134 |
+
[More Information Needed]
|
| 135 |
+
|
| 136 |
+
#### Summary
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
## Model Examination [optional]
|
| 141 |
+
|
| 142 |
+
<!-- Relevant interpretability work for the model goes here -->
|
| 143 |
+
|
| 144 |
+
[More Information Needed]
|
| 145 |
+
|
| 146 |
+
## Environmental Impact
|
| 147 |
+
|
| 148 |
+
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 149 |
+
|
| 150 |
+
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 151 |
+
|
| 152 |
+
- **Hardware Type:** [More Information Needed]
|
| 153 |
+
- **Hours used:** [More Information Needed]
|
| 154 |
+
- **Cloud Provider:** [More Information Needed]
|
| 155 |
+
- **Compute Region:** [More Information Needed]
|
| 156 |
+
- **Carbon Emitted:** [More Information Needed]
|
| 157 |
+
|
| 158 |
+
## Technical Specifications [optional]
|
| 159 |
+
|
| 160 |
+
### Model Architecture and Objective
|
| 161 |
+
|
| 162 |
+
[More Information Needed]
|
| 163 |
+
|
| 164 |
+
### Compute Infrastructure
|
| 165 |
+
|
| 166 |
+
[More Information Needed]
|
| 167 |
+
|
| 168 |
+
#### Hardware
|
| 169 |
+
|
| 170 |
+
[More Information Needed]
|
| 171 |
+
|
| 172 |
+
#### Software
|
| 173 |
+
|
| 174 |
+
[More Information Needed]
|
| 175 |
+
|
| 176 |
+
## Citation [optional]
|
| 177 |
+
|
| 178 |
+
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 179 |
+
|
| 180 |
+
**BibTeX:**
|
| 181 |
+
|
| 182 |
+
[More Information Needed]
|
| 183 |
+
|
| 184 |
+
**APA:**
|
| 185 |
+
|
| 186 |
+
[More Information Needed]
|
| 187 |
+
|
| 188 |
+
## Glossary [optional]
|
| 189 |
+
|
| 190 |
+
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 191 |
+
|
| 192 |
+
[More Information Needed]
|
| 193 |
+
|
| 194 |
+
## More Information [optional]
|
| 195 |
+
|
| 196 |
+
[More Information Needed]
|
| 197 |
+
|
| 198 |
+
## Model Card Authors [optional]
|
| 199 |
+
|
| 200 |
+
[More Information Needed]
|
| 201 |
+
|
| 202 |
+
## Model Card Contact
|
| 203 |
+
|
| 204 |
+
[More Information Needed]
|
| 205 |
+
### Framework versions
|
| 206 |
+
|
| 207 |
+
- PEFT 0.21.0
|
adapter/adapter_config.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "Qwen/Qwen3.5-9B",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 128,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.0,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.0",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 64,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"down_proj",
|
| 36 |
+
"q_proj",
|
| 37 |
+
"up_proj",
|
| 38 |
+
"out_proj",
|
| 39 |
+
"o_proj",
|
| 40 |
+
"gate_proj",
|
| 41 |
+
"in_proj_z",
|
| 42 |
+
"in_proj_a",
|
| 43 |
+
"in_proj_b",
|
| 44 |
+
"in_proj_qkv",
|
| 45 |
+
"k_proj",
|
| 46 |
+
"v_proj"
|
| 47 |
+
],
|
| 48 |
+
"target_parameters": null,
|
| 49 |
+
"task_type": "CAUSAL_LM",
|
| 50 |
+
"trainable_token_indices": null,
|
| 51 |
+
"use_bdlora": null,
|
| 52 |
+
"use_dora": false,
|
| 53 |
+
"use_qalora": false,
|
| 54 |
+
"use_rslora": false,
|
| 55 |
+
"velora_config": null
|
| 56 |
+
}
|
adapter/adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6201fcb1c08144e465a03fd6f5355c4394b3bffaad659440fa64063dd4f71509
|
| 3 |
+
size 692529024
|
base-model-files.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"files": {
|
| 3 |
+
".gitattributes": "34448b82c17d60fec9b65b1f093c115ddbaadc04beb1b0140b6bfed2e012a930",
|
| 4 |
+
"LICENSE": "bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a",
|
| 5 |
+
"README.md": "c5f5a8c2dddab69cfbf05279235aa5fddb137939a06539c4c7637aa900fef6d0",
|
| 6 |
+
"chat_template.jinja": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715",
|
| 7 |
+
"config.json": "d0883072e01861ed0b2d47be3c16c36a8e81c224c7ffaa310c6558fb3f932b05",
|
| 8 |
+
"merges.txt": "a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d",
|
| 9 |
+
"model.safetensors-00001-of-00004.safetensors": "db6f444b43d318c92f360a13a25561a6a65b10c0631b8ed305a426dbaa6c380e",
|
| 10 |
+
"model.safetensors-00002-of-00004.safetensors": "31c7d7e2dd5d207840b31cc59083c8f4c4718959149e0358c0364052bb9a0330",
|
| 11 |
+
"model.safetensors-00003-of-00004.safetensors": "7ec36ba3a4176a44c3c0876ad80c56a2f70c84bf008d82e9501df642f17dadec",
|
| 12 |
+
"model.safetensors-00004-of-00004.safetensors": "b62b0c4cd7e44edee103ee8f4fe225f246d5e768e07bfd5f25b63a8aa1fdd0c6",
|
| 13 |
+
"model.safetensors.index.json": "26d3539b516be613f39563617cb9d33b3f83d401298125be392c80cefb8f7fe5",
|
| 14 |
+
"preprocessor_config.json": "27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516",
|
| 15 |
+
"tokenizer.json": "5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42",
|
| 16 |
+
"tokenizer_config.json": "316230d6a809701f4db5ea8f8fc862bc3a6f3229c937c174e674ff3ca0a64ac8",
|
| 17 |
+
"video_preprocessor_config.json": "7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13",
|
| 18 |
+
"vocab.json": "ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003"
|
| 19 |
+
},
|
| 20 |
+
"model_id": "Qwen/Qwen3.5-9B",
|
| 21 |
+
"revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a",
|
| 22 |
+
"source_manifest_sha256": "2b890ff314357c061b6fb1e69dcf530344bc036d3dd35a40ce234e4de5d4670e",
|
| 23 |
+
"source_manifest_version": "image-jev/read-results/v1"
|
| 24 |
+
}
|
calibration.json
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"adapter_sha256": "1e87a9966f77622d9b70b874364adc7ac1990b78735b5e08168d68106715d341",
|
| 3 |
+
"artifact_sha256": "fc29b120cf2ac00d1917c3712fcd2293b5be3c4746de7d4640b41c3cce251501",
|
| 4 |
+
"base": {
|
| 5 |
+
"repo": "Qwen/Qwen3.5-9B",
|
| 6 |
+
"revision": "c202236235762e1c871ad0ccb60c8ee5ba337b9a"
|
| 7 |
+
},
|
| 8 |
+
"global_temperature": 1.474397718906407,
|
| 9 |
+
"image_route": "native",
|
| 10 |
+
"image_route_config_sha256": "ba61089ca8f811596a21354e679df307d8ad73f5593e7bd904e581d36c1fcb1c",
|
| 11 |
+
"regularization": 8.0,
|
| 12 |
+
"route_type_counts": {
|
| 13 |
+
"default": {
|
| 14 |
+
"4-option": 233,
|
| 15 |
+
"5-marker": 114,
|
| 16 |
+
"yes/no": 253
|
| 17 |
+
},
|
| 18 |
+
"screen_geometry": {
|
| 19 |
+
"4-option": 101,
|
| 20 |
+
"5-marker": 230,
|
| 21 |
+
"yes/no": 0
|
| 22 |
+
}
|
| 23 |
+
},
|
| 24 |
+
"route_type_temperatures": {
|
| 25 |
+
"default": {
|
| 26 |
+
"4-option": 1.2338173243557795,
|
| 27 |
+
"5-marker": 1.976517229898893,
|
| 28 |
+
"yes/no": 1.3352577388514966
|
| 29 |
+
},
|
| 30 |
+
"screen_geometry": {
|
| 31 |
+
"4-option": 3.1395652047002547,
|
| 32 |
+
"5-marker": 1.3292399584473866,
|
| 33 |
+
"yes/no": 0.6572106043305197
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"rules_sha256": "dabf0df7a787f305d272b061b6e0fe01fde605c7c8ddbca2c3da9e2e162e5c75",
|
| 37 |
+
"rules_version": "router-rules/v1",
|
| 38 |
+
"version": "image-jev/router-calibration/v1"
|
| 39 |
+
}
|
open_decisions/__init__.py
ADDED
|
File without changes
|
open_decisions/image_jev/__init__.py
ADDED
|
File without changes
|
open_decisions/image_jev/release/__init__.py
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Fallback release of the frozen base model with route-bound temperature calibration.
|
| 2 |
+
|
| 3 |
+
``predictor`` is the request contract and readout, ``server`` the HTTP wrapper,
|
| 4 |
+
``bundle`` the release directory generator, ``measure_body`` the board-style
|
| 5 |
+
measurement manifest body builder. Only ``predictor``, ``server`` and ``smoke``
|
| 6 |
+
are shipped inside the release bundle.
|
| 7 |
+
"""
|
open_decisions/image_jev/release/predictor.py
ADDED
|
@@ -0,0 +1,586 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Request contract, calibration and readout for Image Hopper releases.
|
| 2 |
+
|
| 3 |
+
The model path is exactly the one evaluated in stage 2 (``deploy/modal_image_jev.py``
|
| 4 |
+
``TorchPredictor`` on the native arm): :func:`vision.prepare_native` (processor-default
|
| 5 |
+
resolution, no pixel budget), one prefill with a last-position vocabulary projection
|
| 6 |
+
(:func:`vision.last_position_logits`), an FP32 softmax over the displayed option letters, and
|
| 7 |
+
a per-answer-type (or route x answer-type) temperature from the bound calibration artifact.
|
| 8 |
+
The routed system keeps its LoRA attached and enables it only for the registered text cues.
|
| 9 |
+
Temperature scaling never changes the selected option.
|
| 10 |
+
|
| 11 |
+
Wire format (TypeSafe ``/v1/systemone`` style, as used by published image submissions)::
|
| 12 |
+
|
| 13 |
+
{"state": <string or object, optional>,
|
| 14 |
+
"images": ["data:image/png;base64,...", ...], # or bare base64
|
| 15 |
+
"questions": {"decision": {"type": "choice", "instructions": "...",
|
| 16 |
+
"criteria": {"<name>": "<description>", ...}}}}
|
| 17 |
+
|
| 18 |
+
Response::
|
| 19 |
+
|
| 20 |
+
{"model": NAME, "usage": {"input_tokens": n, "output_tokens": 0},
|
| 21 |
+
"answers": {"decision": {"type": "choice", "choice": "<name>",
|
| 22 |
+
"probabilities": {"<name>": p, ...}}}}
|
| 23 |
+
|
| 24 |
+
``noul`` (yes/no) questions answer ``{"type": "noul", "noul": P(yes)}``.
|
| 25 |
+
"""
|
| 26 |
+
|
| 27 |
+
from __future__ import annotations
|
| 28 |
+
|
| 29 |
+
import base64
|
| 30 |
+
import binascii
|
| 31 |
+
import hashlib
|
| 32 |
+
import json
|
| 33 |
+
import math
|
| 34 |
+
from contextlib import nullcontext
|
| 35 |
+
from dataclasses import dataclass
|
| 36 |
+
from pathlib import Path
|
| 37 |
+
from typing import Any, Callable, Mapping, Sequence
|
| 38 |
+
|
| 39 |
+
from open_decisions.image_jev import router, vision
|
| 40 |
+
from open_decisions.image_jev.speed import AdapterSwitch, NativeCache, PhaseTimer
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
BASE_REPO = "Qwen/Qwen3.5-9B"
|
| 44 |
+
BASE_REVISION = "c202236235762e1c871ad0ccb60c8ee5ba337b9a"
|
| 45 |
+
IMAGE_ROUTE = "native"
|
| 46 |
+
CALIBRATION_VERSION = "image-jev/calibration/v2"
|
| 47 |
+
# Same buckets as train.calibration.answer_type for requests without an explicit answer type
|
| 48 |
+
# (two options -> yes/no, four -> 4-option, five -> 5-marker); other counts use the global
|
| 49 |
+
# temperature. Every evaluated item had 2, 4 or 5 options.
|
| 50 |
+
TYPE_BUCKET_BY_OPTION_COUNT = {2: "yes/no", 4: "4-option", 5: "5-marker"}
|
| 51 |
+
MAX_IMAGES = 4
|
| 52 |
+
NOUL_OPTIONS = ({"name": "yes", "description": "Yes."}, {"name": "no", "description": "No."})
|
| 53 |
+
CONTEXT_POLICIES = ("prepend", "ignore")
|
| 54 |
+
_CALIBRATION_FIELDS = {
|
| 55 |
+
"artifact_sha256", "checkpoint_sha256", "global_temperature", "image_route",
|
| 56 |
+
"image_route_config_sha256", "regularization", "routing_config_sha256", "type_counts",
|
| 57 |
+
"type_temperatures", "validation", "version",
|
| 58 |
+
}
|
| 59 |
+
_ROUTED_CALIBRATION_FIELDS = {
|
| 60 |
+
"adapter_sha256", "artifact_sha256", "base", "global_temperature", "image_route",
|
| 61 |
+
"image_route_config_sha256", "regularization", "route_type_counts",
|
| 62 |
+
"route_type_temperatures", "rules_sha256", "rules_version", "version",
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
class RequestError(ValueError):
|
| 67 |
+
"""A malformed or unsupported request (HTTP 400)."""
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _compact(value: Any) -> bytes:
|
| 71 |
+
return json.dumps(value, sort_keys=True, separators=(",", ":"), allow_nan=False).encode()
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def _temperature(value: Any, name: str) -> float:
|
| 75 |
+
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
| 76 |
+
raise ValueError(f"{name} must be a number")
|
| 77 |
+
value = float(value)
|
| 78 |
+
if not math.isfinite(value) or value <= 0:
|
| 79 |
+
raise ValueError(f"{name} must be finite and positive")
|
| 80 |
+
return value
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
@dataclass(frozen=True)
|
| 84 |
+
class Calibration:
|
| 85 |
+
"""The route-bound temperature map (``image-jev/calibration/v2``) as served."""
|
| 86 |
+
|
| 87 |
+
global_temperature: float
|
| 88 |
+
type_temperatures: Mapping[str, float]
|
| 89 |
+
artifact_sha256: str
|
| 90 |
+
identity_sha256: str
|
| 91 |
+
|
| 92 |
+
@classmethod
|
| 93 |
+
def from_artifact(cls, value: Mapping[str, Any], *,
|
| 94 |
+
checkpoint_sha256: str | None = "") -> "Calibration":
|
| 95 |
+
if not isinstance(value, Mapping) or set(value) != _CALIBRATION_FIELDS:
|
| 96 |
+
raise ValueError("calibration artifact fields differ from image-jev/calibration/v2")
|
| 97 |
+
body = {key: item for key, item in value.items() if key != "artifact_sha256"}
|
| 98 |
+
if value["artifact_sha256"] != hashlib.sha256(_compact(body)).hexdigest():
|
| 99 |
+
raise ValueError("calibration artifact hash differs")
|
| 100 |
+
if value["version"] != CALIBRATION_VERSION:
|
| 101 |
+
raise ValueError("calibration artifact version differs")
|
| 102 |
+
if value["image_route"] != IMAGE_ROUTE:
|
| 103 |
+
raise ValueError("calibration was not fitted on the native image route")
|
| 104 |
+
if value["image_route_config_sha256"] != vision.IMAGE_ROUTE_CONFIG_SHA256[IMAGE_ROUTE]:
|
| 105 |
+
raise ValueError("calibration image-route configuration differs")
|
| 106 |
+
if value["routing_config_sha256"] != vision.ROUTING_CONFIG_SHA256:
|
| 107 |
+
raise ValueError("calibration routing configuration differs")
|
| 108 |
+
checkpoint = value["checkpoint_sha256"]
|
| 109 |
+
if not isinstance(checkpoint, str) or (checkpoint and
|
| 110 |
+
(len(checkpoint) != 64 or any(
|
| 111 |
+
char not in "0123456789abcdef"
|
| 112 |
+
for char in checkpoint))):
|
| 113 |
+
raise ValueError("calibration checkpoint_sha256 must be empty or a lowercase SHA-256")
|
| 114 |
+
if checkpoint_sha256 is not None and checkpoint != checkpoint_sha256:
|
| 115 |
+
target = "frozen base" if checkpoint_sha256 == "" else "selected adapter"
|
| 116 |
+
raise ValueError(f"calibration is not bound to the {target}")
|
| 117 |
+
temperatures = value["type_temperatures"]
|
| 118 |
+
if not isinstance(temperatures, Mapping):
|
| 119 |
+
raise ValueError("type_temperatures must be an object")
|
| 120 |
+
return cls(
|
| 121 |
+
global_temperature=_temperature(value["global_temperature"], "global_temperature"),
|
| 122 |
+
type_temperatures={
|
| 123 |
+
str(key): _temperature(item, f"type_temperatures[{key!r}]")
|
| 124 |
+
for key, item in temperatures.items()
|
| 125 |
+
},
|
| 126 |
+
artifact_sha256=str(value["artifact_sha256"]),
|
| 127 |
+
identity_sha256=hashlib.sha256(_compact(dict(value))).hexdigest(),
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
@classmethod
|
| 131 |
+
def load(cls, path: str | Path, *, checkpoint_sha256: str | None = "") -> "Calibration":
|
| 132 |
+
try:
|
| 133 |
+
value = json.loads(Path(path).read_text(encoding="utf-8"))
|
| 134 |
+
except (OSError, json.JSONDecodeError) as error:
|
| 135 |
+
raise ValueError(f"cannot load calibration {path}: {error}") from error
|
| 136 |
+
return cls.from_artifact(value, checkpoint_sha256=checkpoint_sha256)
|
| 137 |
+
|
| 138 |
+
def temperature(self, n_options: int) -> float:
|
| 139 |
+
bucket = TYPE_BUCKET_BY_OPTION_COUNT.get(int(n_options))
|
| 140 |
+
if bucket is None:
|
| 141 |
+
return self.global_temperature
|
| 142 |
+
return self.type_temperatures.get(bucket, self.global_temperature)
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
@dataclass(frozen=True)
|
| 146 |
+
class RoutedCalibration:
|
| 147 |
+
"""Serving view of ``image-jev/router-calibration/v1``."""
|
| 148 |
+
|
| 149 |
+
global_temperature: float
|
| 150 |
+
route_type_temperatures: Mapping[str, Mapping[str, float]]
|
| 151 |
+
artifact_sha256: str
|
| 152 |
+
identity_sha256: str
|
| 153 |
+
adapter_sha256: str
|
| 154 |
+
base: Mapping[str, str]
|
| 155 |
+
rules_sha256: str
|
| 156 |
+
|
| 157 |
+
@classmethod
|
| 158 |
+
def from_artifact(
|
| 159 |
+
cls, value: Mapping[str, Any], *, adapter_sha256: str | None = None,
|
| 160 |
+
base: Mapping[str, str] | None = None,
|
| 161 |
+
) -> "RoutedCalibration":
|
| 162 |
+
if not isinstance(value, Mapping) or set(value) != _ROUTED_CALIBRATION_FIELDS:
|
| 163 |
+
raise ValueError("routed calibration artifact fields differ")
|
| 164 |
+
body = {key: item for key, item in value.items() if key != "artifact_sha256"}
|
| 165 |
+
if value["artifact_sha256"] != hashlib.sha256(_compact(body)).hexdigest():
|
| 166 |
+
raise ValueError("routed calibration artifact hash differs")
|
| 167 |
+
if value["version"] != "image-jev/router-calibration/v1":
|
| 168 |
+
raise ValueError("routed calibration artifact version differs")
|
| 169 |
+
if value["rules_version"] != router.RULES_VERSION \
|
| 170 |
+
or value["rules_sha256"] != router.RULES_SHA256:
|
| 171 |
+
raise ValueError("routed calibration rules binding differs")
|
| 172 |
+
if value["image_route"] != IMAGE_ROUTE \
|
| 173 |
+
or value["image_route_config_sha256"] != vision.IMAGE_ROUTE_CONFIG_SHA256[IMAGE_ROUTE]:
|
| 174 |
+
raise ValueError("routed calibration image-route binding differs")
|
| 175 |
+
if adapter_sha256 is not None and value["adapter_sha256"] != adapter_sha256:
|
| 176 |
+
raise ValueError("routed calibration adapter binding differs")
|
| 177 |
+
if base is not None and value["base"] != dict(base):
|
| 178 |
+
raise ValueError("routed calibration base binding differs")
|
| 179 |
+
if (
|
| 180 |
+
not isinstance(value["adapter_sha256"], str)
|
| 181 |
+
or len(value["adapter_sha256"]) != 64
|
| 182 |
+
or any(char not in "0123456789abcdef" for char in value["adapter_sha256"])
|
| 183 |
+
):
|
| 184 |
+
raise ValueError("routed calibration adapter hash is invalid")
|
| 185 |
+
if not isinstance(value["base"], Mapping) or set(value["base"]) != {"repo", "revision"}:
|
| 186 |
+
raise ValueError("routed calibration base pin differs")
|
| 187 |
+
route_temperatures = value["route_type_temperatures"]
|
| 188 |
+
route_counts = value["route_type_counts"]
|
| 189 |
+
if not isinstance(route_temperatures, Mapping) or set(route_temperatures) != set(router.ROUTES):
|
| 190 |
+
raise ValueError("routed calibration routes differ")
|
| 191 |
+
if not isinstance(route_counts, Mapping) or set(route_counts) != set(router.ROUTES):
|
| 192 |
+
raise ValueError("routed calibration count routes differ")
|
| 193 |
+
checked = {}
|
| 194 |
+
for route_name in router.ROUTES:
|
| 195 |
+
values = route_temperatures[route_name]
|
| 196 |
+
counts = route_counts[route_name]
|
| 197 |
+
if (
|
| 198 |
+
not isinstance(values, Mapping)
|
| 199 |
+
or set(values) != set(TYPE_BUCKET_BY_OPTION_COUNT.values())
|
| 200 |
+
or not isinstance(counts, Mapping)
|
| 201 |
+
or set(counts) != set(TYPE_BUCKET_BY_OPTION_COUNT.values())
|
| 202 |
+
):
|
| 203 |
+
raise ValueError("routed calibration temperatures must be objects")
|
| 204 |
+
if any(
|
| 205 |
+
isinstance(count, bool) or not isinstance(count, int) or count < 0
|
| 206 |
+
for count in counts.values()
|
| 207 |
+
):
|
| 208 |
+
raise ValueError("routed calibration counts are invalid")
|
| 209 |
+
checked[route_name] = {
|
| 210 |
+
str(kind): _temperature(item, f"route_type_temperatures[{route_name}][{kind}]")
|
| 211 |
+
for kind, item in values.items()
|
| 212 |
+
}
|
| 213 |
+
return cls(
|
| 214 |
+
global_temperature=_temperature(value["global_temperature"], "global_temperature"),
|
| 215 |
+
route_type_temperatures=checked,
|
| 216 |
+
artifact_sha256=str(value["artifact_sha256"]),
|
| 217 |
+
identity_sha256=hashlib.sha256(_compact(dict(value))).hexdigest(),
|
| 218 |
+
adapter_sha256=str(value["adapter_sha256"]), base=dict(value["base"]),
|
| 219 |
+
rules_sha256=str(value["rules_sha256"]),
|
| 220 |
+
)
|
| 221 |
+
|
| 222 |
+
def temperature(self, n_options: int, route_name: str) -> float:
|
| 223 |
+
if route_name not in router.ROUTES:
|
| 224 |
+
raise ValueError("unknown routed calibration route")
|
| 225 |
+
bucket = TYPE_BUCKET_BY_OPTION_COUNT.get(int(n_options))
|
| 226 |
+
if bucket is None:
|
| 227 |
+
return self.global_temperature
|
| 228 |
+
return self.route_type_temperatures[route_name].get(bucket, self.global_temperature)
|
| 229 |
+
|
| 230 |
+
|
| 231 |
+
def serving_calibration_from_artifact(
|
| 232 |
+
value: Mapping[str, Any], *, checkpoint_sha256: str | None = "",
|
| 233 |
+
adapter_sha256: str | None = None, base: Mapping[str, str] | None = None,
|
| 234 |
+
) -> Calibration | RoutedCalibration:
|
| 235 |
+
if not isinstance(value, Mapping):
|
| 236 |
+
raise ValueError("calibration artifact must be an object")
|
| 237 |
+
if value.get("version") == "image-jev/router-calibration/v1":
|
| 238 |
+
return RoutedCalibration.from_artifact(
|
| 239 |
+
value, adapter_sha256=adapter_sha256, base=base
|
| 240 |
+
)
|
| 241 |
+
return Calibration.from_artifact(value, checkpoint_sha256=checkpoint_sha256)
|
| 242 |
+
|
| 243 |
+
|
| 244 |
+
@dataclass(frozen=True)
|
| 245 |
+
class Question:
|
| 246 |
+
qid: str
|
| 247 |
+
kind: str
|
| 248 |
+
instructions: str
|
| 249 |
+
options: tuple[dict[str, str], ...]
|
| 250 |
+
route_instructions: str | None = None
|
| 251 |
+
|
| 252 |
+
|
| 253 |
+
def _image_source(value: Any, index: int):
|
| 254 |
+
if not isinstance(value, str) or not value:
|
| 255 |
+
raise RequestError(f"image {index} must be a non-empty data URI or base64 string")
|
| 256 |
+
if value.startswith("data:"):
|
| 257 |
+
return value
|
| 258 |
+
if (len(value) // 4) * 3 > vision.MAX_IMAGE_BYTES + 2:
|
| 259 |
+
raise RequestError(f"image {index} exceeds {vision.MAX_IMAGE_BYTES} bytes")
|
| 260 |
+
try:
|
| 261 |
+
return base64.b64decode(value, validate=True)
|
| 262 |
+
except (binascii.Error, ValueError):
|
| 263 |
+
raise RequestError(f"image {index} is neither a data URI nor valid base64") from None
|
| 264 |
+
|
| 265 |
+
|
| 266 |
+
def _images(body: Mapping[str, Any]) -> list:
|
| 267 |
+
raw = body.get("images")
|
| 268 |
+
state = body.get("state")
|
| 269 |
+
if raw is None and isinstance(state, Mapping):
|
| 270 |
+
raw = state.get("images", state.get("image"))
|
| 271 |
+
if isinstance(raw, str):
|
| 272 |
+
raw = [raw]
|
| 273 |
+
if not isinstance(raw, list) or not raw:
|
| 274 |
+
raise RequestError("request needs at least one image in 'images'")
|
| 275 |
+
if len(raw) > MAX_IMAGES:
|
| 276 |
+
raise RequestError(f"at most {MAX_IMAGES} images per request")
|
| 277 |
+
return [_image_source(value, index) for index, value in enumerate(raw)]
|
| 278 |
+
|
| 279 |
+
|
| 280 |
+
def _context(state: Any) -> str | None:
|
| 281 |
+
if state is None:
|
| 282 |
+
return None
|
| 283 |
+
if isinstance(state, str):
|
| 284 |
+
return state.strip() or None
|
| 285 |
+
if isinstance(state, Mapping):
|
| 286 |
+
rest = {key: value for key, value in state.items() if key not in {"image", "images"}}
|
| 287 |
+
return json.dumps(rest, ensure_ascii=False, sort_keys=True) if rest else None
|
| 288 |
+
raise RequestError("state must be a string or an object")
|
| 289 |
+
|
| 290 |
+
|
| 291 |
+
def _options(criteria: Any) -> tuple[dict[str, str], ...]:
|
| 292 |
+
if isinstance(criteria, Mapping):
|
| 293 |
+
values = [{"name": str(name), "description": str(text)} for name, text in criteria.items()]
|
| 294 |
+
elif isinstance(criteria, list):
|
| 295 |
+
values = []
|
| 296 |
+
for index, option in enumerate(criteria):
|
| 297 |
+
if isinstance(option, str):
|
| 298 |
+
values.append({"name": option, "description": option})
|
| 299 |
+
elif isinstance(option, Mapping) and "name" in option:
|
| 300 |
+
values.append({"name": str(option["name"]),
|
| 301 |
+
"description": str(option.get("description", option["name"]))})
|
| 302 |
+
else:
|
| 303 |
+
raise RequestError(f"criteria[{index}] must be text or an object with a name")
|
| 304 |
+
else:
|
| 305 |
+
raise RequestError("choice criteria must be an object or a list")
|
| 306 |
+
names = [value["name"] for value in values]
|
| 307 |
+
if not 2 <= len(values) <= len(vision.hopper_prompt.LETTERS):
|
| 308 |
+
raise RequestError("choice questions need 2 to 26 options")
|
| 309 |
+
if len(set(names)) != len(names) or any(not name for name in names):
|
| 310 |
+
raise RequestError("option names must be unique and non-empty")
|
| 311 |
+
return tuple(values)
|
| 312 |
+
|
| 313 |
+
|
| 314 |
+
def parse_request(body: Any, *, context_policy: str = "prepend") -> tuple[list, list[Question]]:
|
| 315 |
+
"""Validate a ``/v1/systemone`` body into images and questions (no model work)."""
|
| 316 |
+
if context_policy not in CONTEXT_POLICIES:
|
| 317 |
+
raise ValueError(f"context_policy must be one of {CONTEXT_POLICIES}")
|
| 318 |
+
if not isinstance(body, Mapping):
|
| 319 |
+
raise RequestError("request body must be a JSON object")
|
| 320 |
+
questions = body.get("questions")
|
| 321 |
+
if not isinstance(questions, Mapping) or not questions:
|
| 322 |
+
raise RequestError("request needs a non-empty 'questions' object")
|
| 323 |
+
images = _images(body)
|
| 324 |
+
context = _context(body.get("state")) if context_policy == "prepend" else None
|
| 325 |
+
parsed = []
|
| 326 |
+
for qid, spec in questions.items():
|
| 327 |
+
if not isinstance(spec, Mapping):
|
| 328 |
+
raise RequestError(f"question {qid!r} must be an object")
|
| 329 |
+
kind = spec.get("type", "choice")
|
| 330 |
+
instructions = spec.get("instructions")
|
| 331 |
+
if not isinstance(instructions, str) or not instructions.strip():
|
| 332 |
+
raise RequestError(f"question {qid!r} needs non-empty instructions")
|
| 333 |
+
if context:
|
| 334 |
+
instructions = f"{context}\n\n{instructions}"
|
| 335 |
+
if kind == "choice":
|
| 336 |
+
options = _options(spec.get("criteria"))
|
| 337 |
+
elif kind == "noul":
|
| 338 |
+
options = NOUL_OPTIONS
|
| 339 |
+
else:
|
| 340 |
+
raise RequestError(f"question {qid!r}: unsupported type {kind!r} (choice or noul)")
|
| 341 |
+
parsed.append(Question(str(qid), kind, instructions, options, str(spec["instructions"])))
|
| 342 |
+
return images, parsed
|
| 343 |
+
|
| 344 |
+
|
| 345 |
+
def _length(value: Any) -> int:
|
| 346 |
+
shape = getattr(value, "shape", None)
|
| 347 |
+
if shape is not None:
|
| 348 |
+
return int(shape[-1])
|
| 349 |
+
first = value[0] if value and isinstance(value[0], (list, tuple)) else value
|
| 350 |
+
return len(first)
|
| 351 |
+
|
| 352 |
+
|
| 353 |
+
class Predictor:
|
| 354 |
+
"""One prefill per question; calibrated option probabilities in the board's shape."""
|
| 355 |
+
|
| 356 |
+
def __init__(self, model, processor, calibration: Calibration | RoutedCalibration, *, model_name: str,
|
| 357 |
+
context_policy: str = "prepend", max_input_tokens: int | None = None,
|
| 358 |
+
inference_context: Callable[[], Any] | None = None,
|
| 359 |
+
adapter_context: Callable[[str], Any] | None = None,
|
| 360 |
+
synchronize: Callable[[], Any] | None = None, forward_call=None) -> None:
|
| 361 |
+
if context_policy not in CONTEXT_POLICIES:
|
| 362 |
+
raise ValueError(f"context_policy must be one of {CONTEXT_POLICIES}")
|
| 363 |
+
self.model, self.processor, self.calibration = model, processor, calibration
|
| 364 |
+
self.model_name = model_name
|
| 365 |
+
self.context_policy = context_policy
|
| 366 |
+
self.max_input_tokens = max_input_tokens
|
| 367 |
+
self._inference_context = inference_context
|
| 368 |
+
self._adapter_context = adapter_context
|
| 369 |
+
self._synchronize = synchronize
|
| 370 |
+
self._forward_call = forward_call
|
| 371 |
+
self._cache = NativeCache(processor)
|
| 372 |
+
try:
|
| 373 |
+
self.device = next(model.parameters()).device
|
| 374 |
+
except (StopIteration, AttributeError):
|
| 375 |
+
self.device = None
|
| 376 |
+
|
| 377 |
+
def _read(self, images: Sequence, question: Question, *, loaded_images=None, timer=None) -> tuple[list[float], int]:
|
| 378 |
+
example = {"question": question.instructions, "options": list(question.options),
|
| 379 |
+
"images": list(images)}
|
| 380 |
+
try:
|
| 381 |
+
prepared = vision.prepare_native(example, self.processor, loaded_images=loaded_images,
|
| 382 |
+
cache=self._cache, timer=timer, device=self.device)
|
| 383 |
+
except (TypeError, ValueError) as error:
|
| 384 |
+
raise RequestError(str(error)) from error
|
| 385 |
+
tokens = _length(prepared.input_ids)
|
| 386 |
+
if self.max_input_tokens is not None and tokens > self.max_input_tokens:
|
| 387 |
+
raise RequestError(
|
| 388 |
+
f"processed request has {tokens} tokens, above the configured "
|
| 389 |
+
f"{self.max_input_tokens}-token limit"
|
| 390 |
+
)
|
| 391 |
+
with timer.phase("route_adapter_switch"):
|
| 392 |
+
route_name = None
|
| 393 |
+
if isinstance(self.calibration, RoutedCalibration):
|
| 394 |
+
route_name = router.route(
|
| 395 |
+
question.route_instructions or question.instructions, question.options
|
| 396 |
+
)
|
| 397 |
+
temperature = self.calibration.temperature(len(question.options), route_name)
|
| 398 |
+
else:
|
| 399 |
+
temperature = self.calibration.temperature(len(question.options))
|
| 400 |
+
route_context = (
|
| 401 |
+
self._adapter_context(route_name) if self._adapter_context is not None
|
| 402 |
+
else nullcontext()
|
| 403 |
+
)
|
| 404 |
+
inference_context = (
|
| 405 |
+
self._inference_context() if self._inference_context is not None else nullcontext()
|
| 406 |
+
)
|
| 407 |
+
with route_context:
|
| 408 |
+
with inference_context:
|
| 409 |
+
logits = vision.last_position_logits(
|
| 410 |
+
self.model, prepared, timer=timer, forward_call=self._forward_call)
|
| 411 |
+
with timer.phase("readout_calibration"):
|
| 412 |
+
probabilities = vision.option_probs(
|
| 413 |
+
logits, prepared.letter_token_ids, temperature=temperature)
|
| 414 |
+
with timer.phase("readout_calibration"):
|
| 415 |
+
total = math.fsum(probabilities)
|
| 416 |
+
return [value / total for value in probabilities], tokens
|
| 417 |
+
|
| 418 |
+
def __call__(self, body: Any, *, phase_seconds=None) -> dict[str, Any]:
|
| 419 |
+
timer = PhaseTimer(phase_seconds, self._synchronize)
|
| 420 |
+
with timer.phase("decode_load"):
|
| 421 |
+
images, questions = parse_request(body, context_policy=self.context_policy)
|
| 422 |
+
return self.predict_questions(images, questions, timer=timer)
|
| 423 |
+
|
| 424 |
+
def predict_questions(self, images, questions, *, timer=None) -> dict[str, Any]:
|
| 425 |
+
timer = timer or PhaseTimer()
|
| 426 |
+
with timer.phase("decode_load"):
|
| 427 |
+
try:
|
| 428 |
+
loaded = [vision._load_image(source, exif_transpose=False) for source in images]
|
| 429 |
+
loaded = [image if image.mode == "RGB" else image.convert("RGB") for image in loaded]
|
| 430 |
+
except (TypeError, ValueError) as error:
|
| 431 |
+
raise RequestError(str(error)) from error
|
| 432 |
+
answers, input_tokens = {}, 0
|
| 433 |
+
for question in questions:
|
| 434 |
+
probabilities, tokens = self._read(images, question, loaded_images=loaded, timer=timer)
|
| 435 |
+
with timer.phase("response_build"):
|
| 436 |
+
input_tokens += tokens
|
| 437 |
+
names = [option["name"] for option in question.options]
|
| 438 |
+
# Exact ties go to the first option, as the harness's scorer breaks them.
|
| 439 |
+
top = max(range(len(names)), key=lambda index: (probabilities[index], -index))
|
| 440 |
+
if question.kind == "noul":
|
| 441 |
+
answers[question.qid] = {"type": "noul", "noul": probabilities[0]}
|
| 442 |
+
else:
|
| 443 |
+
answers[question.qid] = {
|
| 444 |
+
"type": "choice", "choice": names[top],
|
| 445 |
+
"probabilities": dict(zip(names, probabilities, strict=True)),
|
| 446 |
+
}
|
| 447 |
+
with timer.phase("response_build"):
|
| 448 |
+
return {"model": self.model_name,
|
| 449 |
+
"usage": {"input_tokens": input_tokens, "output_tokens": 0},
|
| 450 |
+
"answers": answers}
|
| 451 |
+
|
| 452 |
+
|
| 453 |
+
def file_sha256(path: str | Path) -> str:
|
| 454 |
+
digest = hashlib.sha256()
|
| 455 |
+
with Path(path).open("rb") as stream:
|
| 456 |
+
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
| 457 |
+
digest.update(chunk)
|
| 458 |
+
return digest.hexdigest()
|
| 459 |
+
|
| 460 |
+
|
| 461 |
+
def tree_sha256(path: str | Path) -> str:
|
| 462 |
+
"""Hash a local adapter as sorted relative paths and file-byte hashes."""
|
| 463 |
+
root = Path(path)
|
| 464 |
+
rows = [
|
| 465 |
+
{"path": item.relative_to(root).as_posix(), "sha256": file_sha256(item)}
|
| 466 |
+
for item in sorted(root.rglob("*")) if item.is_file()
|
| 467 |
+
]
|
| 468 |
+
if not rows:
|
| 469 |
+
raise ValueError("adapter directory is empty")
|
| 470 |
+
return hashlib.sha256(_compact(rows)).hexdigest()
|
| 471 |
+
|
| 472 |
+
|
| 473 |
+
def verify_snapshot(snapshot: str | Path, expected: Mapping[str, str]) -> dict[str, str]:
|
| 474 |
+
"""Hash every file listed for the pinned base revision; refuse any difference."""
|
| 475 |
+
root = Path(snapshot)
|
| 476 |
+
failures = []
|
| 477 |
+
for name, digest in sorted(expected.items()):
|
| 478 |
+
path = root / name
|
| 479 |
+
if not path.is_file():
|
| 480 |
+
failures.append(f"missing {name}")
|
| 481 |
+
elif file_sha256(path) != digest:
|
| 482 |
+
failures.append(f"hash differs: {name}")
|
| 483 |
+
if failures:
|
| 484 |
+
raise RuntimeError("base model snapshot differs from the pinned revision: "
|
| 485 |
+
+ "; ".join(failures))
|
| 486 |
+
return dict(expected)
|
| 487 |
+
|
| 488 |
+
|
| 489 |
+
def check_fast_path() -> None:
|
| 490 |
+
"""Qwen3.5's linear-attention layers need these kernels; the fallback is far slower."""
|
| 491 |
+
missing = []
|
| 492 |
+
for module in ("fla", "causal_conv1d"):
|
| 493 |
+
try:
|
| 494 |
+
__import__(module)
|
| 495 |
+
except ImportError:
|
| 496 |
+
missing.append(module)
|
| 497 |
+
if missing:
|
| 498 |
+
raise RuntimeError(
|
| 499 |
+
"fast linear-attention kernels unavailable (" + ", ".join(missing) + "); install "
|
| 500 |
+
"flash-linear-attention and causal-conv1d from requirements.txt, or pass "
|
| 501 |
+
"--allow-slow-path"
|
| 502 |
+
)
|
| 503 |
+
|
| 504 |
+
|
| 505 |
+
def load_predictor(*, calibration_path: str | Path, model_name: str,
|
| 506 |
+
model_cache: str | None = None, offline: bool = False,
|
| 507 |
+
adapter_path: str | Path | None = None,
|
| 508 |
+
base_hashes_path: str | Path | None = None, device: str = "cuda",
|
| 509 |
+
context_policy: str = "prepend", max_input_tokens: int | None = None,
|
| 510 |
+
allow_slow_path: bool = False) -> tuple[Predictor, dict[str, Any]]:
|
| 511 |
+
"""Load the pinned frozen base in bf16 and return the predictor and its provenance."""
|
| 512 |
+
import platform
|
| 513 |
+
|
| 514 |
+
import torch
|
| 515 |
+
import transformers
|
| 516 |
+
from huggingface_hub import snapshot_download
|
| 517 |
+
from transformers import AutoProcessor
|
| 518 |
+
|
| 519 |
+
if not allow_slow_path:
|
| 520 |
+
check_fast_path()
|
| 521 |
+
adapter = None if adapter_path is None else Path(adapter_path)
|
| 522 |
+
adapter_sha256 = "" if adapter is None else tree_sha256(adapter)
|
| 523 |
+
try:
|
| 524 |
+
calibration_value = json.loads(Path(calibration_path).read_text(encoding="utf-8"))
|
| 525 |
+
except (OSError, json.JSONDecodeError) as error:
|
| 526 |
+
raise ValueError(f"cannot load calibration {calibration_path}: {error}") from error
|
| 527 |
+
base_pin = {"repo": BASE_REPO, "revision": BASE_REVISION}
|
| 528 |
+
calibration = serving_calibration_from_artifact(
|
| 529 |
+
calibration_value, checkpoint_sha256=adapter_sha256,
|
| 530 |
+
adapter_sha256=adapter_sha256 or None, base=base_pin,
|
| 531 |
+
)
|
| 532 |
+
routed = isinstance(calibration, RoutedCalibration)
|
| 533 |
+
if routed and adapter is None:
|
| 534 |
+
raise ValueError("routed calibration requires the screen_geometry adapter")
|
| 535 |
+
snapshot = Path(snapshot_download(BASE_REPO, revision=BASE_REVISION, cache_dir=model_cache,
|
| 536 |
+
local_files_only=offline))
|
| 537 |
+
verified = None
|
| 538 |
+
if base_hashes_path is not None:
|
| 539 |
+
listing = json.loads(Path(base_hashes_path).read_text(encoding="utf-8"))
|
| 540 |
+
if listing.get("model_id") != BASE_REPO or listing.get("revision") != BASE_REVISION:
|
| 541 |
+
raise RuntimeError("base-model hash listing is for a different model or revision")
|
| 542 |
+
verified = len(verify_snapshot(snapshot, listing["files"]))
|
| 543 |
+
processor = AutoProcessor.from_pretrained(str(snapshot), trust_remote_code=False, use_fast=True)
|
| 544 |
+
auto_model = getattr(transformers, "AutoModelForMultimodalLM", None) or getattr(
|
| 545 |
+
transformers, "AutoModelForImageTextToText")
|
| 546 |
+
model = auto_model.from_pretrained(str(snapshot), torch_dtype=torch.bfloat16,
|
| 547 |
+
device_map=device, trust_remote_code=False, attn_implementation="sdpa")
|
| 548 |
+
if adapter is not None:
|
| 549 |
+
from peft import PeftModel
|
| 550 |
+
|
| 551 |
+
loaded = PeftModel.from_pretrained(
|
| 552 |
+
model, str(adapter), **({"adapter_name": router.SCREEN_GEOMETRY} if routed else {})
|
| 553 |
+
)
|
| 554 |
+
model = loaded if routed else loaded.merge_and_unload()
|
| 555 |
+
model = model.eval()
|
| 556 |
+
adapter_context = AdapterSwitch(model) if routed else None
|
| 557 |
+
predictor = Predictor(model, processor, calibration, model_name=model_name,
|
| 558 |
+
context_policy=context_policy, max_input_tokens=max_input_tokens,
|
| 559 |
+
inference_context=torch.inference_mode,
|
| 560 |
+
adapter_context=adapter_context,
|
| 561 |
+
synchronize=(torch.cuda.synchronize if str(device).startswith("cuda") else None))
|
| 562 |
+
provenance = {
|
| 563 |
+
"base_repo": BASE_REPO, "base_revision": BASE_REVISION,
|
| 564 |
+
"base_files_verified": verified,
|
| 565 |
+
"adapter": None if adapter is None else {"path": str(adapter), "sha256": adapter_sha256},
|
| 566 |
+
"image_route": IMAGE_ROUTE,
|
| 567 |
+
"image_route_config_sha256": vision.IMAGE_ROUTE_CONFIG_SHA256[IMAGE_ROUTE],
|
| 568 |
+
"calibration_artifact_sha256": calibration.artifact_sha256,
|
| 569 |
+
"calibration_identity_sha256": calibration.identity_sha256,
|
| 570 |
+
"system_kind": "routed" if routed else "single-route",
|
| 571 |
+
"router_rules_version": router.RULES_VERSION if routed else None,
|
| 572 |
+
"router_rules_sha256": router.RULES_SHA256 if routed else None,
|
| 573 |
+
"adapter_switching": "PEFT layer enable/disable on route transition" if routed else None,
|
| 574 |
+
"precision": "bfloat16", "attention": "sdpa", "processor_use_fast": True, "scorer": "one-prefill-option-letter",
|
| 575 |
+
"context_policy": context_policy, "max_input_tokens": max_input_tokens,
|
| 576 |
+
"torch_version": torch.__version__, "transformers_version": transformers.__version__,
|
| 577 |
+
"python_version": platform.python_version(),
|
| 578 |
+
}
|
| 579 |
+
return predictor, provenance
|
| 580 |
+
|
| 581 |
+
|
| 582 |
+
__all__ = [
|
| 583 |
+
"BASE_REPO", "BASE_REVISION", "Calibration", "RoutedCalibration", "Predictor", "Question", "RequestError",
|
| 584 |
+
"TYPE_BUCKET_BY_OPTION_COUNT", "check_fast_path", "load_predictor", "parse_request",
|
| 585 |
+
"serving_calibration_from_artifact", "tree_sha256", "verify_snapshot",
|
| 586 |
+
]
|
open_decisions/image_jev/release/server.py
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""HTTP server for the frozen-base image decision release.
|
| 2 |
+
|
| 3 |
+
POST /v1/systemone TypeSafe-style request (see ``predictor``), JSON response
|
| 4 |
+
GET /health model, pinned base revision and calibration identity
|
| 5 |
+
|
| 6 |
+
Single-threaded on purpose: one GPU, one request at a time, as published image
|
| 7 |
+
submissions are served. Any ``Authorization`` header is ignored.
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import argparse
|
| 13 |
+
import json
|
| 14 |
+
import sys
|
| 15 |
+
from http.server import BaseHTTPRequestHandler, HTTPServer
|
| 16 |
+
from typing import Any, Callable, Mapping
|
| 17 |
+
|
| 18 |
+
from open_decisions.image_jev.release.predictor import CONTEXT_POLICIES, RequestError
|
| 19 |
+
|
| 20 |
+
MAX_BODY_BYTES = 128 * 1024 * 1024
|
| 21 |
+
DEFAULT_NAME = "image-hopper"
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def handler(predict: Callable[[Any], dict], health: Mapping[str, Any], *, phase_timing=False):
|
| 25 |
+
class Handler(BaseHTTPRequestHandler):
|
| 26 |
+
def log_message(self, *args): # quiet; the harness measures latency itself
|
| 27 |
+
pass
|
| 28 |
+
|
| 29 |
+
def _send(self, status: int, value: Mapping[str, Any]) -> None:
|
| 30 |
+
payload = json.dumps(value, ensure_ascii=False).encode("utf-8")
|
| 31 |
+
self.send_response(status)
|
| 32 |
+
self.send_header("Content-Type", "application/json")
|
| 33 |
+
self.send_header("Content-Length", str(len(payload)))
|
| 34 |
+
self.end_headers()
|
| 35 |
+
self.wfile.write(payload)
|
| 36 |
+
|
| 37 |
+
def do_GET(self):
|
| 38 |
+
if self.path == "/health":
|
| 39 |
+
self._send(200, dict(health))
|
| 40 |
+
else:
|
| 41 |
+
self._send(404, {"error": "not found"})
|
| 42 |
+
|
| 43 |
+
def do_POST(self):
|
| 44 |
+
if self.path != "/v1/systemone":
|
| 45 |
+
self._send(404, {"error": "not found"})
|
| 46 |
+
return
|
| 47 |
+
try:
|
| 48 |
+
length = int(self.headers.get("Content-Length", "0"))
|
| 49 |
+
except ValueError:
|
| 50 |
+
length = -1
|
| 51 |
+
if not 0 < length <= MAX_BODY_BYTES:
|
| 52 |
+
self._send(400, {"error": "missing or oversized request body"})
|
| 53 |
+
return
|
| 54 |
+
try:
|
| 55 |
+
body = json.loads(self.rfile.read(length))
|
| 56 |
+
if phase_timing:
|
| 57 |
+
seconds = {}
|
| 58 |
+
result = predict(body, phase_seconds=seconds)
|
| 59 |
+
result["phase_seconds"] = seconds
|
| 60 |
+
else:
|
| 61 |
+
result = predict(body)
|
| 62 |
+
self._send(200, result)
|
| 63 |
+
except (json.JSONDecodeError, RequestError) as error:
|
| 64 |
+
self._send(400, {"error": str(error)})
|
| 65 |
+
except Exception as error: # noqa: BLE001 - report, keep serving
|
| 66 |
+
self._send(500, {"error": f"{type(error).__name__}: {error}"})
|
| 67 |
+
|
| 68 |
+
return Handler
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def build_parser() -> argparse.ArgumentParser:
|
| 72 |
+
parser = argparse.ArgumentParser(description=__doc__,
|
| 73 |
+
formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 74 |
+
parser.add_argument("--calibration", default="calibration.json")
|
| 75 |
+
parser.add_argument("--adapter", default=None,
|
| 76 |
+
help="optional local adapter directory; calibration must bind its tree hash")
|
| 77 |
+
parser.add_argument("--base-hashes", default="base-model-files.json",
|
| 78 |
+
help="hash listing of the pinned base snapshot; '' skips the check")
|
| 79 |
+
parser.add_argument("--model-cache", default=None, help="Hugging Face cache directory")
|
| 80 |
+
parser.add_argument("--offline", action="store_true", help="never download; use the cache")
|
| 81 |
+
parser.add_argument("--name", default=DEFAULT_NAME)
|
| 82 |
+
parser.add_argument("--host", default="127.0.0.1")
|
| 83 |
+
parser.add_argument("--port", type=int, default=8080)
|
| 84 |
+
parser.add_argument("--device", default="cuda")
|
| 85 |
+
parser.add_argument("--context-policy", choices=CONTEXT_POLICIES, default="prepend",
|
| 86 |
+
help="how request 'state' text reaches the prompt")
|
| 87 |
+
parser.add_argument("--max-input-tokens", type=int, default=None,
|
| 88 |
+
help="refuse (HTTP 400) requests above this many processed tokens")
|
| 89 |
+
parser.add_argument("--allow-slow-path", action="store_true")
|
| 90 |
+
parser.add_argument("--phase-timing", action="store_true",
|
| 91 |
+
help="diagnostic synchronized phase_seconds in responses; default off")
|
| 92 |
+
return parser
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def main(argv=None) -> int:
|
| 96 |
+
from open_decisions.image_jev.release.predictor import load_predictor
|
| 97 |
+
|
| 98 |
+
args = build_parser().parse_args(argv)
|
| 99 |
+
predictor, provenance = load_predictor(
|
| 100 |
+
calibration_path=args.calibration, model_name=args.name, model_cache=args.model_cache,
|
| 101 |
+
adapter_path=args.adapter,
|
| 102 |
+
offline=args.offline, base_hashes_path=args.base_hashes or None, device=args.device,
|
| 103 |
+
context_policy=args.context_policy, max_input_tokens=args.max_input_tokens,
|
| 104 |
+
allow_slow_path=args.allow_slow_path,
|
| 105 |
+
)
|
| 106 |
+
health = {"model": args.name, **provenance}
|
| 107 |
+
print(json.dumps(health, indent=2, sort_keys=True), file=sys.stderr)
|
| 108 |
+
server = HTTPServer((args.host, args.port), handler(predictor, health, phase_timing=args.phase_timing))
|
| 109 |
+
print(f"serving {args.name} on http://{args.host}:{args.port}/v1/systemone", file=sys.stderr)
|
| 110 |
+
try:
|
| 111 |
+
server.serve_forever()
|
| 112 |
+
except KeyboardInterrupt:
|
| 113 |
+
pass
|
| 114 |
+
return 0
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
if __name__ == "__main__":
|
| 118 |
+
raise SystemExit(main())
|
open_decisions/image_jev/release/smoke.py
ADDED
|
@@ -0,0 +1,223 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Local smoke test with a fake processor and model (CPU, no weights, no network).
|
| 2 |
+
|
| 3 |
+
Checks the request contract end to end through the real HTTP handler: native image
|
| 4 |
+
preparation, the one-prefill readout, the per-answer-type temperature, the response
|
| 5 |
+
shape, and refusal of malformed requests. Run from the bundle root::
|
| 6 |
+
|
| 7 |
+
python -m open_decisions.image_jev.release.smoke --calibration calibration.json
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import argparse
|
| 13 |
+
import base64
|
| 14 |
+
import io
|
| 15 |
+
import json
|
| 16 |
+
import math
|
| 17 |
+
from pathlib import Path
|
| 18 |
+
|
| 19 |
+
LETTER_BASE = 1000
|
| 20 |
+
VOCABULARY = 1100
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
class FakeTokenizer:
|
| 24 |
+
def encode(self, text, add_special_tokens=False):
|
| 25 |
+
return [LETTER_BASE + ord(text) - ord("A")]
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
class FakeProcessor:
|
| 29 |
+
"""Shapes like the real processor's output; one token per prompt character."""
|
| 30 |
+
|
| 31 |
+
def __init__(self):
|
| 32 |
+
self.tokenizer = FakeTokenizer()
|
| 33 |
+
|
| 34 |
+
def apply_chat_template(self, messages, **kwargs):
|
| 35 |
+
assert kwargs.get("enable_thinking") is False
|
| 36 |
+
return "\n".join(message["content"] for message in messages)
|
| 37 |
+
|
| 38 |
+
def __call__(self, *, text, images, padding, return_tensors):
|
| 39 |
+
import torch
|
| 40 |
+
|
| 41 |
+
ids = torch.tensor([[ord(char) % 256 for char in text[0]]], dtype=torch.long)
|
| 42 |
+
pixels = torch.stack([
|
| 43 |
+
torch.tensor(list(image.convert("L").resize((4, 4)).tobytes()), dtype=torch.float32)
|
| 44 |
+
for image in images
|
| 45 |
+
])
|
| 46 |
+
grid = torch.tensor([[1, 2, 2]] * len(images), dtype=torch.long)
|
| 47 |
+
return {"input_ids": ids, "pixel_values": pixels, "image_grid_thw": grid}
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _fake_model():
|
| 51 |
+
import torch
|
| 52 |
+
|
| 53 |
+
class FakeModel(torch.nn.Module):
|
| 54 |
+
"""Letter k gets logit -0.5 k: option A always leads, by a fixed margin."""
|
| 55 |
+
|
| 56 |
+
def __init__(self):
|
| 57 |
+
super().__init__()
|
| 58 |
+
self.anchor = torch.nn.Parameter(torch.zeros(1))
|
| 59 |
+
self.calls = 0
|
| 60 |
+
|
| 61 |
+
def forward(self, input_ids, pixel_values, image_grid_thw, use_cache=False,
|
| 62 |
+
return_dict=True, logits_to_keep=None, **_kwargs):
|
| 63 |
+
assert logits_to_keep == 1 and use_cache is False
|
| 64 |
+
self.calls += 1
|
| 65 |
+
logits = torch.full((input_ids.shape[0], 1, VOCABULARY), -50.0)
|
| 66 |
+
for index in range(26):
|
| 67 |
+
logits[:, 0, LETTER_BASE + index] = -0.5 * index
|
| 68 |
+
return _Output(logits=logits + self.anchor)
|
| 69 |
+
|
| 70 |
+
return FakeModel().eval()
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
class _Output(dict):
|
| 74 |
+
"""Model output readable both as a mapping and by attribute, like transformers' outputs."""
|
| 75 |
+
|
| 76 |
+
def __getattr__(self, name):
|
| 77 |
+
try:
|
| 78 |
+
return self[name]
|
| 79 |
+
except KeyError as error:
|
| 80 |
+
raise AttributeError(name) from error
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def _png() -> str:
|
| 84 |
+
from PIL import Image
|
| 85 |
+
|
| 86 |
+
buffer = io.BytesIO()
|
| 87 |
+
Image.new("RGB", (64, 48), (90, 140, 200)).save(buffer, format="PNG")
|
| 88 |
+
return "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode("ascii")
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def _expected(n_options: int, temperature: float) -> list[float]:
|
| 92 |
+
weights = [math.exp(-0.5 * index / temperature) for index in range(n_options)]
|
| 93 |
+
total = sum(weights)
|
| 94 |
+
return [weight / total for weight in weights]
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
class _MemoryConnection:
|
| 98 |
+
"""Minimal socket facade for one BaseHTTPRequestHandler exchange."""
|
| 99 |
+
|
| 100 |
+
def __init__(self, request: bytes):
|
| 101 |
+
self.request = io.BytesIO(request)
|
| 102 |
+
self.response = io.BytesIO()
|
| 103 |
+
|
| 104 |
+
def makefile(self, mode, _buffering=None):
|
| 105 |
+
return self.request if "r" in mode else self.response
|
| 106 |
+
|
| 107 |
+
def sendall(self, value):
|
| 108 |
+
self.response.write(value)
|
| 109 |
+
|
| 110 |
+
def close(self):
|
| 111 |
+
pass
|
| 112 |
+
|
| 113 |
+
|
| 114 |
+
def _exchange(handler_type, method: str, path: str, body=None) -> tuple[int, dict]:
|
| 115 |
+
payload = b"" if body is None else json.dumps(body).encode("utf-8")
|
| 116 |
+
headers = [f"{method} {path} HTTP/1.1", "Host: smoke"]
|
| 117 |
+
if body is not None:
|
| 118 |
+
headers += ["Content-Type: application/json", f"Content-Length: {len(payload)}"]
|
| 119 |
+
request = ("\r\n".join(headers) + "\r\n\r\n").encode("ascii") + payload
|
| 120 |
+
connection = _MemoryConnection(request)
|
| 121 |
+
handler_type(connection, ("127.0.0.1", 1), object())
|
| 122 |
+
head, raw_body = connection.response.getvalue().split(b"\r\n\r\n", 1)
|
| 123 |
+
status = int(head.splitlines()[0].split()[1])
|
| 124 |
+
return status, json.loads(raw_body)
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
def run(calibration_path: str | Path) -> dict:
|
| 128 |
+
from open_decisions.image_jev import router
|
| 129 |
+
from open_decisions.image_jev.release.predictor import (
|
| 130 |
+
Predictor, RoutedCalibration, serving_calibration_from_artifact,
|
| 131 |
+
)
|
| 132 |
+
from open_decisions.image_jev.release.server import handler
|
| 133 |
+
|
| 134 |
+
# The fake model validates the readout for either frozen-base or adapter bundles. The
|
| 135 |
+
# bundle builder/server perform the real checkpoint-binding check.
|
| 136 |
+
artifact = json.loads(Path(calibration_path).read_text(encoding="utf-8"))
|
| 137 |
+
calibration = serving_calibration_from_artifact(
|
| 138 |
+
artifact, checkpoint_sha256=None, adapter_sha256=None, base=None
|
| 139 |
+
)
|
| 140 |
+
model = _fake_model()
|
| 141 |
+
predictor = Predictor(model, FakeProcessor(), calibration, model_name="smoke")
|
| 142 |
+
handler_type = handler(predictor, {"model": "smoke"})
|
| 143 |
+
|
| 144 |
+
def post(body) -> tuple[int, dict]:
|
| 145 |
+
return _exchange(handler_type, "POST", "/v1/systemone", body)
|
| 146 |
+
|
| 147 |
+
image = _png()
|
| 148 |
+
checks = {}
|
| 149 |
+
try:
|
| 150 |
+
status, health = _exchange(handler_type, "GET", "/health")
|
| 151 |
+
assert status == 200 and health == {"model": "smoke"}
|
| 152 |
+
checks["health_shape"] = True
|
| 153 |
+
|
| 154 |
+
status, body = post({
|
| 155 |
+
"state": "", "images": [image],
|
| 156 |
+
"questions": {
|
| 157 |
+
"four": {"type": "choice", "instructions": "Which shelf is fullest?",
|
| 158 |
+
"criteria": {"top": "Top shelf", "middle": "Middle shelf",
|
| 159 |
+
"bottom": "Bottom shelf", "none": "All equal"}},
|
| 160 |
+
"three": {"type": "choice", "instructions": "Pick one.",
|
| 161 |
+
"criteria": ["red", "green", "blue"]},
|
| 162 |
+
"binary": {"type": "noul", "instructions": "Is the item damaged?"},
|
| 163 |
+
},
|
| 164 |
+
})
|
| 165 |
+
assert status == 200, body
|
| 166 |
+
assert set(body) == {"model", "usage", "answers"} and body["model"] == "smoke"
|
| 167 |
+
assert set(body["usage"]) == {"input_tokens", "output_tokens"}
|
| 168 |
+
answers = body["answers"]
|
| 169 |
+
assert set(answers) == {"four", "three", "binary"}
|
| 170 |
+
assert set(answers["four"]) == {"type", "choice", "probabilities"}
|
| 171 |
+
assert set(answers["binary"]) == {"type", "noul"}
|
| 172 |
+
four = answers["four"]["probabilities"]
|
| 173 |
+
assert list(four) == ["top", "middle", "bottom", "none"]
|
| 174 |
+
assert answers["four"]["choice"] == "top"
|
| 175 |
+
four_temperature = (
|
| 176 |
+
calibration.temperature(4, router.DEFAULT)
|
| 177 |
+
if isinstance(calibration, RoutedCalibration) else calibration.temperature(4)
|
| 178 |
+
)
|
| 179 |
+
for name, got, want in zip(four, four.values(), _expected(4, four_temperature)):
|
| 180 |
+
assert abs(got - want) < 1e-6, (name, got, want)
|
| 181 |
+
three = answers["three"]["probabilities"]
|
| 182 |
+
for got, want in zip(three.values(), _expected(3, calibration.global_temperature)):
|
| 183 |
+
assert abs(got - want) < 1e-6
|
| 184 |
+
binary_temperature = (
|
| 185 |
+
calibration.temperature(2, router.DEFAULT)
|
| 186 |
+
if isinstance(calibration, RoutedCalibration) else calibration.temperature(2)
|
| 187 |
+
)
|
| 188 |
+
want_yes = _expected(2, binary_temperature)[0]
|
| 189 |
+
assert abs(answers["binary"]["noul"] - want_yes) < 1e-6
|
| 190 |
+
assert body["usage"]["output_tokens"] == 0 and body["usage"]["input_tokens"] > 0
|
| 191 |
+
assert model.calls == 3
|
| 192 |
+
checks["calibrated_probabilities"] = True
|
| 193 |
+
checks["systemone_response_shape"] = True
|
| 194 |
+
|
| 195 |
+
bare = image.split(",", 1)[1]
|
| 196 |
+
status, body = post({"images": [bare], "questions": {"d": {
|
| 197 |
+
"instructions": "Pick.", "criteria": {"a": "A", "b": "B"}}}})
|
| 198 |
+
assert status == 200 and set(body["answers"]["d"]["probabilities"]) == {"a", "b"}
|
| 199 |
+
checks["bare_base64_image"] = True
|
| 200 |
+
|
| 201 |
+
for bad in ({"questions": {"d": {"instructions": "x", "criteria": ["a", "b"]}}},
|
| 202 |
+
{"images": [image], "questions": {"d": {"type": "score", "instructions": "x"}}},
|
| 203 |
+
{"images": [image], "questions": {"d": {"instructions": "x", "criteria": ["a"]}}}):
|
| 204 |
+
status, body = post(bad)
|
| 205 |
+
assert status == 400 and "error" in body, (bad, status, body)
|
| 206 |
+
checks["malformed_requests_refused"] = True
|
| 207 |
+
finally:
|
| 208 |
+
pass
|
| 209 |
+
return {"ok": True, "checks": checks,
|
| 210 |
+
"calibration_identity_sha256": calibration.identity_sha256}
|
| 211 |
+
|
| 212 |
+
|
| 213 |
+
def main(argv=None) -> int:
|
| 214 |
+
parser = argparse.ArgumentParser(description=__doc__,
|
| 215 |
+
formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 216 |
+
parser.add_argument("--calibration", default="calibration.json")
|
| 217 |
+
args = parser.parse_args(argv)
|
| 218 |
+
print(json.dumps(run(args.calibration), indent=2, sort_keys=True))
|
| 219 |
+
return 0
|
| 220 |
+
|
| 221 |
+
|
| 222 |
+
if __name__ == "__main__":
|
| 223 |
+
raise SystemExit(main())
|
open_decisions/image_jev/router.py
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Deterministic text-only router registered as ``router-rules/v1``.
|
| 2 |
+
|
| 3 |
+
Only the question and visible option text are inspected. Source metadata,
|
| 4 |
+
board categories, images, and conversation state are intentionally absent
|
| 5 |
+
from this API.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import hashlib
|
| 11 |
+
import json
|
| 12 |
+
import re
|
| 13 |
+
from collections.abc import Mapping, Sequence
|
| 14 |
+
from typing import Any
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
RULES_VERSION = "router-rules/v1"
|
| 18 |
+
SCREEN_GEOMETRY = "screen_geometry"
|
| 19 |
+
DEFAULT = "default"
|
| 20 |
+
ROUTES = (SCREEN_GEOMETRY, DEFAULT)
|
| 21 |
+
|
| 22 |
+
GUI_TARGET_CUES = (
|
| 23 |
+
"marker", "click", "tap", "button", "icon", "menu", "toolbar", "tab",
|
| 24 |
+
"window", "dialog", "element", "on screen", "screenshot", "cursor", "link",
|
| 25 |
+
"field",
|
| 26 |
+
)
|
| 27 |
+
GEOMETRY_CUES = (
|
| 28 |
+
"angle", "triangle", "circle", "radius", "diameter", "perimeter", "area of",
|
| 29 |
+
"length of", "parallel", "perpendicular", "polygon", "quadrilateral", "chord",
|
| 30 |
+
"tangent", "arc", "degrees", "°", "find x", "find y", r"\triangle",
|
| 31 |
+
)
|
| 32 |
+
LITERAL_CUES = ("°", r"\triangle")
|
| 33 |
+
|
| 34 |
+
RULE_DEFINITION = {
|
| 35 |
+
"version": RULES_VERSION,
|
| 36 |
+
"input": "question+option-text-only",
|
| 37 |
+
"case_sensitive": False,
|
| 38 |
+
"matching": {
|
| 39 |
+
"word_and_phrase_cues": "unicode-word-boundaries",
|
| 40 |
+
"literal_cues": list(LITERAL_CUES),
|
| 41 |
+
},
|
| 42 |
+
"screen_geometry": {
|
| 43 |
+
"gui_target_cues": list(GUI_TARGET_CUES),
|
| 44 |
+
"geometry_cues": list(GEOMETRY_CUES),
|
| 45 |
+
},
|
| 46 |
+
"fallback": DEFAULT,
|
| 47 |
+
"ambiguous": DEFAULT,
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _canonical_bytes(value: Any) -> bytes:
|
| 52 |
+
return json.dumps(
|
| 53 |
+
value, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
|
| 54 |
+
allow_nan=False,
|
| 55 |
+
).encode("utf-8")
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
RULES_SHA256 = hashlib.sha256(_canonical_bytes(RULE_DEFINITION)).hexdigest()
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
def _pattern(cue: str) -> re.Pattern[str]:
|
| 62 |
+
# Every non-literal registered cue consists of words separated by spaces.
|
| 63 |
+
# Lookarounds make phrases boundary-aware at both ends without ASCII-only
|
| 64 |
+
# ``\b`` behaviour ("tablet" must not match "tab", for example).
|
| 65 |
+
return re.compile(r"(?<!\w)" + re.escape(cue) + r"(?!\w)", re.IGNORECASE)
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
_WORD_PATTERNS = tuple(
|
| 69 |
+
_pattern(cue)
|
| 70 |
+
for cue in (*GUI_TARGET_CUES, *GEOMETRY_CUES)
|
| 71 |
+
if cue not in LITERAL_CUES
|
| 72 |
+
)
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
def _option_text(option: Any) -> str | None:
|
| 76 |
+
if isinstance(option, str):
|
| 77 |
+
return option
|
| 78 |
+
if isinstance(option, Mapping):
|
| 79 |
+
# These are the two visible fields accepted by the benchmark and
|
| 80 |
+
# /v1/systemone contracts. Metadata fields are deliberately ignored.
|
| 81 |
+
values = []
|
| 82 |
+
for key in ("name", "description", "text"):
|
| 83 |
+
value = option.get(key)
|
| 84 |
+
if value is not None:
|
| 85 |
+
if not isinstance(value, str):
|
| 86 |
+
return None
|
| 87 |
+
values.append(value)
|
| 88 |
+
return " ".join(values) if values else None
|
| 89 |
+
return None
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
def route(question: str, options: Sequence[Any]) -> str:
|
| 93 |
+
"""Return the registered route, failing ambiguous/malformed inputs closed.
|
| 94 |
+
|
| 95 |
+
Matching is case-insensitive. Word cues and multi-word phrases require
|
| 96 |
+
word boundaries; ``°`` and ``\\triangle`` are matched literally.
|
| 97 |
+
"""
|
| 98 |
+
if not isinstance(question, str) or isinstance(options, (str, bytes)):
|
| 99 |
+
return DEFAULT
|
| 100 |
+
try:
|
| 101 |
+
option_values = list(options)
|
| 102 |
+
except TypeError:
|
| 103 |
+
return DEFAULT
|
| 104 |
+
visible = []
|
| 105 |
+
for option in option_values:
|
| 106 |
+
value = _option_text(option)
|
| 107 |
+
if value is None:
|
| 108 |
+
return DEFAULT
|
| 109 |
+
visible.append(value)
|
| 110 |
+
text = "\n".join((question, *visible)).casefold()
|
| 111 |
+
if any(cue.casefold() in text for cue in LITERAL_CUES):
|
| 112 |
+
return SCREEN_GEOMETRY
|
| 113 |
+
if any(pattern.search(text) for pattern in _WORD_PATTERNS):
|
| 114 |
+
return SCREEN_GEOMETRY
|
| 115 |
+
return DEFAULT
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
__all__ = [
|
| 119 |
+
"DEFAULT", "GEOMETRY_CUES", "GUI_TARGET_CUES", "LITERAL_CUES", "ROUTES",
|
| 120 |
+
"RULES_SHA256", "RULES_VERSION", "RULE_DEFINITION", "SCREEN_GEOMETRY", "route",
|
| 121 |
+
]
|
open_decisions/image_jev/speed.py
ADDED
|
@@ -0,0 +1,164 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Serial inference helpers; no model imports or GPU synchronization when untimed."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
from contextlib import contextmanager, nullcontext
|
| 5 |
+
from time import perf_counter
|
| 6 |
+
|
| 7 |
+
PHASES = ("decode_load", "processor_preprocess", "tokenise_chat_template",
|
| 8 |
+
"route_adapter_switch", "h2d_copy", "forward", "readout_calibration", "response_build")
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class PhaseTimer:
|
| 12 |
+
def __init__(self, seconds=None, synchronize=None):
|
| 13 |
+
self.seconds, self.synchronize = seconds, synchronize
|
| 14 |
+
if seconds is not None:
|
| 15 |
+
seconds.update(dict.fromkeys(PHASES, 0.0))
|
| 16 |
+
|
| 17 |
+
def phase(self, name):
|
| 18 |
+
return self._phase(name) if self.seconds is not None else nullcontext()
|
| 19 |
+
|
| 20 |
+
@contextmanager
|
| 21 |
+
def _phase(self, name):
|
| 22 |
+
if self.synchronize is not None:
|
| 23 |
+
self.synchronize()
|
| 24 |
+
started = perf_counter()
|
| 25 |
+
try:
|
| 26 |
+
yield
|
| 27 |
+
finally:
|
| 28 |
+
if self.synchronize is not None:
|
| 29 |
+
self.synchronize()
|
| 30 |
+
self.seconds[name] += perf_counter() - started
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
class AdapterSwitch:
|
| 34 |
+
"""Keep PEFT's selected adapter fixed; toggle layers only on route transitions.
|
| 35 |
+
|
| 36 |
+
Owns adapter state for a serial reader. Do not mutate adapters externally.
|
| 37 |
+
"""
|
| 38 |
+
def __init__(self, model):
|
| 39 |
+
from open_decisions.image_jev import router
|
| 40 |
+
self.model = model
|
| 41 |
+
self.layers = model.base_model
|
| 42 |
+
model.set_adapter(router.SCREEN_GEOMETRY)
|
| 43 |
+
self.route = router.SCREEN_GEOMETRY
|
| 44 |
+
|
| 45 |
+
def __call__(self, route):
|
| 46 |
+
from open_decisions.image_jev import router
|
| 47 |
+
if route not in router.ROUTES:
|
| 48 |
+
raise ValueError("unknown adapter route")
|
| 49 |
+
if route != self.route:
|
| 50 |
+
if route == router.SCREEN_GEOMETRY:
|
| 51 |
+
self.layers.enable_adapter_layers()
|
| 52 |
+
else:
|
| 53 |
+
self.layers.disable_adapter_layers()
|
| 54 |
+
self.route = route
|
| 55 |
+
return nullcontext()
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def move_to_device(value, device):
|
| 59 |
+
if device is None or not hasattr(value, "to"):
|
| 60 |
+
return value
|
| 61 |
+
if getattr(value, "device", None) == device:
|
| 62 |
+
return value
|
| 63 |
+
# Pin only CUDA-bound host tensors. Already-device pixels need no copy.
|
| 64 |
+
if getattr(device, "type", str(device).split(":")[0]) == "cuda":
|
| 65 |
+
if getattr(getattr(value, "device", None), "type", None) == "cpu":
|
| 66 |
+
value = value.pin_memory()
|
| 67 |
+
return value.to(device, non_blocking=True)
|
| 68 |
+
return value.to(device)
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
class NativeCache:
|
| 72 |
+
"""Per-processor caches for the immutable letter ids and chat shell."""
|
| 73 |
+
def __init__(self, processor):
|
| 74 |
+
self.processor = processor
|
| 75 |
+
self.tokenizer = CachedTokenizer(processor.tokenizer)
|
| 76 |
+
processor.tokenizer = self.tokenizer
|
| 77 |
+
self.letters = None
|
| 78 |
+
self.shell = None
|
| 79 |
+
|
| 80 |
+
def prompt(self, user):
|
| 81 |
+
from open_decisions.image_jev.vision import IMAGE_SYSTEM
|
| 82 |
+
processor = self.processor
|
| 83 |
+
if not hasattr(processor, "apply_chat_template"):
|
| 84 |
+
return user
|
| 85 |
+
def render(text):
|
| 86 |
+
return processor.apply_chat_template(
|
| 87 |
+
[{"role": "system", "content": IMAGE_SYSTEM}, {"role": "user", "content": text}],
|
| 88 |
+
tokenize=False, add_generation_prompt=True, enable_thinking=False)
|
| 89 |
+
if self.shell is None:
|
| 90 |
+
marker = "IMAGE_HOPPER_STATIC_SHELL_56"
|
| 91 |
+
sample = render(marker)
|
| 92 |
+
parts = sample.split(marker)
|
| 93 |
+
# Verify the shell against the actual first request (templates may inspect content).
|
| 94 |
+
self.shell = tuple(parts) if len(parts) == 2 and "".join((parts[0], user, parts[1])) == render(user) else False
|
| 95 |
+
if self.shell is False:
|
| 96 |
+
return render(user)
|
| 97 |
+
self.tokenizer.configure(*self.shell)
|
| 98 |
+
return self.shell[0] + user + self.shell[1]
|
| 99 |
+
|
| 100 |
+
def letter_ids(self, count):
|
| 101 |
+
from open_decisions.scoring.prompt import letter_token_ids
|
| 102 |
+
if self.letters is None:
|
| 103 |
+
self.letters = letter_token_ids(self.processor.tokenizer)
|
| 104 |
+
return self.letters[:count]
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
class CachedTokenizer:
|
| 108 |
+
"""Cache Qwen chat tokens only at isolated added-special-token boundaries.
|
| 109 |
+
|
| 110 |
+
Unknown kwargs, tokenizer families and shapes use the original tokenizer.
|
| 111 |
+
The first full encoding also verifies exact token-id equality.
|
| 112 |
+
"""
|
| 113 |
+
def __init__(self, tokenizer):
|
| 114 |
+
self.original = tokenizer
|
| 115 |
+
self.parts = None
|
| 116 |
+
self.timer = PhaseTimer()
|
| 117 |
+
|
| 118 |
+
def __getattr__(self, name):
|
| 119 |
+
return getattr(self.original, name)
|
| 120 |
+
|
| 121 |
+
def configure(self, prefix, suffix):
|
| 122 |
+
if self.parts is not None or not getattr(self.original, "is_fast", False):
|
| 123 |
+
return
|
| 124 |
+
if "Qwen" not in type(self.original).__name__ or "<|im_start|>" not in prefix:
|
| 125 |
+
return
|
| 126 |
+
boundary = prefix.rfind("<|im_start|>") + len("<|im_start|>")
|
| 127 |
+
prefix = prefix[:boundary]
|
| 128 |
+
if not suffix.startswith("<|im_end|>"):
|
| 129 |
+
return
|
| 130 |
+
tokens = {str(token): token for token in self.original.added_tokens_decoder.values()}
|
| 131 |
+
for key in ("<|im_start|>", "<|im_end|>"):
|
| 132 |
+
token = tokens.get(key)
|
| 133 |
+
if token is None or not token.special or token.lstrip or token.rstrip:
|
| 134 |
+
return
|
| 135 |
+
if self.original.num_special_tokens_to_add(pair=False):
|
| 136 |
+
return
|
| 137 |
+
self.parts = (prefix, suffix, self.original.encode(prefix, add_special_tokens=False),
|
| 138 |
+
self.original.encode(suffix, add_special_tokens=False), False)
|
| 139 |
+
|
| 140 |
+
def __call__(self, text, **kwargs):
|
| 141 |
+
with self.timer.phase("tokenise_chat_template"):
|
| 142 |
+
allowed = {"padding", "return_attention_mask", "return_token_type_ids", "add_special_tokens", "return_tensors"}
|
| 143 |
+
if (not self.parts or set(kwargs) - allowed or kwargs.get("return_token_type_ids") or
|
| 144 |
+
not isinstance(text, list) or len(text) != 1 or not isinstance(text[0], str)):
|
| 145 |
+
return self.original(text, **kwargs)
|
| 146 |
+
prefix, suffix, before, after, verified = self.parts
|
| 147 |
+
value = text[0]
|
| 148 |
+
if not value.startswith(prefix) or not value.endswith(suffix):
|
| 149 |
+
return self.original(text, **kwargs)
|
| 150 |
+
middle = value[len(prefix):-len(suffix)]
|
| 151 |
+
ids = before + self.original.encode(middle, add_special_tokens=False) + after
|
| 152 |
+
if not verified:
|
| 153 |
+
reference = self.original(text, **kwargs)
|
| 154 |
+
expected = reference["input_ids"]
|
| 155 |
+
if hasattr(expected, "tolist"):
|
| 156 |
+
expected = expected.tolist()
|
| 157 |
+
if expected != [ids]:
|
| 158 |
+
self.parts = False
|
| 159 |
+
return reference
|
| 160 |
+
self.parts = (prefix, suffix, before, after, True)
|
| 161 |
+
return reference
|
| 162 |
+
return self.original.pad({"input_ids": [ids]}, padding=kwargs.get("padding", False),
|
| 163 |
+
return_attention_mask=kwargs.get("return_attention_mask"),
|
| 164 |
+
return_tensors=kwargs.get("return_tensors"))
|
open_decisions/image_jev/vision.py
ADDED
|
@@ -0,0 +1,521 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Shared image preparation and option readout for Image JevBench.
|
| 2 |
+
|
| 3 |
+
This module is deliberately the only path from source images to processor inputs. Training and
|
| 4 |
+
serving should both call :func:`prepare_for_route`; keeping resize, prompt and readout decisions
|
| 5 |
+
here makes a route or budget change visible and versioned instead of an accidental processor
|
| 6 |
+
configuration change.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import base64
|
| 12 |
+
import binascii
|
| 13 |
+
import io
|
| 14 |
+
import json
|
| 15 |
+
import hashlib
|
| 16 |
+
import inspect
|
| 17 |
+
import math
|
| 18 |
+
import os
|
| 19 |
+
from dataclasses import dataclass
|
| 20 |
+
from pathlib import Path
|
| 21 |
+
from types import MappingProxyType
|
| 22 |
+
from typing import Any
|
| 23 |
+
|
| 24 |
+
from PIL import Image, ImageOps, UnidentifiedImageError
|
| 25 |
+
|
| 26 |
+
from open_decisions.image_jev.speed import PhaseTimer, move_to_device
|
| 27 |
+
|
| 28 |
+
# One source for option rendering in training and serving (no fallback path that could drift);
|
| 29 |
+
# tests/test_image_jev_vision.py checks it against the public hopper_decisions copy when that is importable.
|
| 30 |
+
from open_decisions.scoring import prompt as hopper_prompt
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
FACTOR = 32
|
| 34 |
+
MIN_PIXELS = 65_536
|
| 35 |
+
MAX_PIXELS = 16_777_216
|
| 36 |
+
MAX_IMAGE_BYTES = 20 * 1024 * 1024
|
| 37 |
+
MAX_IMAGE_PIXELS = 20_000_000
|
| 38 |
+
BUDGET_VERSION = "b1"
|
| 39 |
+
BUDGETS = MappingProxyType({"photo": 160, "scene": 224, "document": 384, "dense": 768})
|
| 40 |
+
ROUTING_VERSION = "image-jev/content-routing/v1"
|
| 41 |
+
ROUTING_CONFIG = MappingProxyType({
|
| 42 |
+
"version": ROUTING_VERSION,
|
| 43 |
+
"inputs": "decoded-oriented-rgb-pixels-only",
|
| 44 |
+
"policy": "conservative-document",
|
| 45 |
+
"content_class": "document",
|
| 46 |
+
})
|
| 47 |
+
ROUTING_CONFIG_SHA256 = hashlib.sha256(
|
| 48 |
+
json.dumps(dict(ROUTING_CONFIG), sort_keys=True, separators=(",", ":")).encode("utf-8")
|
| 49 |
+
).hexdigest()
|
| 50 |
+
IMAGE_ROUTE_VERSION = "image-jev/image-route/v1"
|
| 51 |
+
IMAGE_ROUTES = ("native", "capped")
|
| 52 |
+
IMAGE_ROUTE_CONFIG_SHA256 = MappingProxyType({
|
| 53 |
+
route: hashlib.sha256(json.dumps(
|
| 54 |
+
{"version": IMAGE_ROUTE_VERSION, "image_route": route},
|
| 55 |
+
sort_keys=True, separators=(",", ":"),
|
| 56 |
+
).encode("utf-8")).hexdigest()
|
| 57 |
+
for route in IMAGE_ROUTES
|
| 58 |
+
})
|
| 59 |
+
|
| 60 |
+
IMAGE_SYSTEM = ("You make decisions about supplied images. Inspect every image and reply with the letter of the "
|
| 61 |
+
"correct option and nothing else. Do not produce reasoning.")
|
| 62 |
+
VISION_TOKEN = "<|vision_start|><|image_pad|><|vision_end|>"
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
@dataclass(frozen=True)
|
| 66 |
+
class PreparedInput:
|
| 67 |
+
"""The complete model input produced identically for training and serving."""
|
| 68 |
+
|
| 69 |
+
input_ids: Any
|
| 70 |
+
pixel_values: Any
|
| 71 |
+
image_grid_thw: Any
|
| 72 |
+
resize_records: list[dict[str, Any]]
|
| 73 |
+
letter_token_ids: list[int]
|
| 74 |
+
prompt: str
|
| 75 |
+
routing_config_sha256: str = ROUTING_CONFIG_SHA256
|
| 76 |
+
# transformers >= 5.x Qwen-VL needs this for multimodal RoPE; returned by the processor beside input_ids.
|
| 77 |
+
mm_token_type_ids: Any = None
|
| 78 |
+
image_route: str = "capped"
|
| 79 |
+
|
| 80 |
+
@property
|
| 81 |
+
def ids(self):
|
| 82 |
+
"""Short alias used by call sites that name language inputs ``ids``."""
|
| 83 |
+
return self.input_ids
|
| 84 |
+
|
| 85 |
+
@property
|
| 86 |
+
def pixels(self):
|
| 87 |
+
"""Short alias for the processor's image tensor."""
|
| 88 |
+
return self.pixel_values
|
| 89 |
+
|
| 90 |
+
@property
|
| 91 |
+
def records(self):
|
| 92 |
+
"""Short alias for the resize audit records."""
|
| 93 |
+
return self.resize_records
|
| 94 |
+
|
| 95 |
+
@property
|
| 96 |
+
def image_route_config_sha256(self):
|
| 97 |
+
"""Registered identity of the native/capped preparation choice."""
|
| 98 |
+
return IMAGE_ROUTE_CONFIG_SHA256[self.image_route]
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def smart_resize(height: int, width: int, factor: int = FACTOR, min_pixels: int = MIN_PIXELS,
|
| 102 |
+
max_pixels: int = MAX_PIXELS) -> tuple[int, int]:
|
| 103 |
+
"""Qwen2-VL/Qwen3-VL's ``smart_resize``, with Qwen3.5-4B image defaults.
|
| 104 |
+
|
| 105 |
+
The arithmetic and its boundary behavior intentionally track Transformers exactly, including
|
| 106 |
+
its rejection only when the absolute aspect ratio is *greater than* 200.
|
| 107 |
+
"""
|
| 108 |
+
if max(height, width) / min(height, width) > 200:
|
| 109 |
+
raise ValueError(
|
| 110 |
+
f"absolute aspect ratio must be smaller than 200, got {max(height, width) / min(height, width)}"
|
| 111 |
+
)
|
| 112 |
+
h_bar = round(height / factor) * factor
|
| 113 |
+
w_bar = round(width / factor) * factor
|
| 114 |
+
if h_bar * w_bar > max_pixels:
|
| 115 |
+
beta = math.sqrt((height * width) / max_pixels)
|
| 116 |
+
h_bar = max(factor, math.floor(height / beta / factor) * factor)
|
| 117 |
+
w_bar = max(factor, math.floor(width / beta / factor) * factor)
|
| 118 |
+
elif h_bar * w_bar < min_pixels:
|
| 119 |
+
beta = math.sqrt(min_pixels / (height * width))
|
| 120 |
+
h_bar = math.ceil(height * beta / factor) * factor
|
| 121 |
+
w_bar = math.ceil(width * beta / factor) * factor
|
| 122 |
+
return h_bar, w_bar
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
def image_tokens(height: int, width: int, factor: int = FACTOR) -> int:
|
| 126 |
+
"""Merged image-token count after the model processor's normal smart resize."""
|
| 127 |
+
resized_h, resized_w = smart_resize(height, width, factor=factor)
|
| 128 |
+
return (resized_h // factor) * (resized_w // factor)
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def budget_pixels(content_class: str) -> int:
|
| 132 |
+
"""Return the explicit pixel-area budget for a named content class."""
|
| 133 |
+
try:
|
| 134 |
+
return BUDGETS[content_class] * FACTOR * FACTOR
|
| 135 |
+
except (KeyError, TypeError):
|
| 136 |
+
raise ValueError(f"unknown image content class {content_class!r}; expected one of {tuple(BUDGETS)}") from None
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
def route_content(img: Image.Image) -> str:
|
| 140 |
+
"""Conservatively route every image to ``document`` for now.
|
| 141 |
+
|
| 142 |
+
Content routing is a later measured experiment. Any future classifier must use only observable
|
| 143 |
+
image content, never benchmark ids, source names, family names or other hidden metadata.
|
| 144 |
+
"""
|
| 145 |
+
if not isinstance(img, Image.Image):
|
| 146 |
+
raise TypeError("classify_content expects a PIL.Image.Image")
|
| 147 |
+
return "document"
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
# Compatibility name for callers that only need the returned class. All
|
| 151 |
+
# consumers route through ``route_content`` and bind ROUTING_CONFIG_SHA256.
|
| 152 |
+
classify_content = route_content
|
| 153 |
+
|
| 154 |
+
|
| 155 |
+
def _scaled_size(width: int, height: int, long_edge: int) -> tuple[int, int]:
|
| 156 |
+
"""Largest-edge parameterisation of a whole-image, aspect-preserving integer size."""
|
| 157 |
+
if width >= height:
|
| 158 |
+
return long_edge, max(1, min(height, round(height * long_edge / width)))
|
| 159 |
+
return max(1, min(width, round(width * long_edge / height))), long_edge
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
def _budgeted_size(width: int, height: int, token_budget: int) -> tuple[int, int]:
|
| 163 |
+
"""A whole-image size whose default processor result meets ``token_budget``."""
|
| 164 |
+
if image_tokens(height, width) <= token_budget:
|
| 165 |
+
return width, height
|
| 166 |
+
|
| 167 |
+
# Prefer the processor's own factor-aligned target. This makes its subsequent smart resize a
|
| 168 |
+
# no-op and ensures LANCZOS, rather than a second processor interpolation, sets the final pixels.
|
| 169 |
+
target_h, target_w = smart_resize(height, width, max_pixels=token_budget * FACTOR * FACTOR)
|
| 170 |
+
while (target_h // FACTOR) * (target_w // FACTOR) > token_budget:
|
| 171 |
+
if target_w >= target_h and target_w > FACTOR:
|
| 172 |
+
target_w -= FACTOR
|
| 173 |
+
elif target_h > FACTOR:
|
| 174 |
+
target_h -= FACTOR
|
| 175 |
+
else:
|
| 176 |
+
break
|
| 177 |
+
try:
|
| 178 |
+
aligned_tokens = image_tokens(target_h, target_w)
|
| 179 |
+
except ValueError:
|
| 180 |
+
aligned_tokens = token_budget + 1
|
| 181 |
+
if target_w <= width and target_h <= height and aligned_tokens <= token_budget:
|
| 182 |
+
return target_w, target_h
|
| 183 |
+
|
| 184 |
+
# A source edge shorter than one factor can make the aligned target an upscale. Search actual
|
| 185 |
+
# image sizes in that edge case; the saved pixels still enforce the budget even if a later
|
| 186 |
+
# processor is constructed with its native 16M maximum.
|
| 187 |
+
low, high = 1, max(width, height)
|
| 188 |
+
best = None
|
| 189 |
+
while low <= high:
|
| 190 |
+
middle = (low + high) // 2
|
| 191 |
+
candidate = _scaled_size(width, height, middle)
|
| 192 |
+
try:
|
| 193 |
+
fits = image_tokens(candidate[1], candidate[0]) <= token_budget
|
| 194 |
+
except ValueError:
|
| 195 |
+
fits = False
|
| 196 |
+
if fits:
|
| 197 |
+
best = candidate
|
| 198 |
+
low = middle + 1
|
| 199 |
+
else:
|
| 200 |
+
high = middle - 1
|
| 201 |
+
if best is None: # Unreachable for accepted (<=200:1) inputs and the smallest 160-token budget.
|
| 202 |
+
raise ValueError(f"image aspect ratio cannot fit the {token_budget}-token budget")
|
| 203 |
+
return best
|
| 204 |
+
|
| 205 |
+
|
| 206 |
+
def resize_for_budget(img: Image.Image, content_class: str) -> tuple[Image.Image, dict[str, Any]]:
|
| 207 |
+
"""Convert to RGB and, if necessary, LANCZOS-downscale the whole image to its named budget."""
|
| 208 |
+
if not isinstance(img, Image.Image):
|
| 209 |
+
raise TypeError("resize_for_budget expects a PIL.Image.Image")
|
| 210 |
+
token_budget = budget_pixels(content_class) // (FACTOR * FACTOR)
|
| 211 |
+
orig_size = img.size
|
| 212 |
+
new_size = _budgeted_size(*orig_size, token_budget)
|
| 213 |
+
converted = img.convert("RGB")
|
| 214 |
+
resized = converted if new_size == orig_size else converted.resize(new_size, Image.Resampling.LANCZOS)
|
| 215 |
+
tokens = image_tokens(resized.height, resized.width)
|
| 216 |
+
if tokens > token_budget: # Keep a hard invariant next to the operation that enforces it.
|
| 217 |
+
raise RuntimeError(f"resize produced {tokens} image tokens for a {token_budget}-token budget")
|
| 218 |
+
record = {"content_class": content_class, "budget_version": BUDGET_VERSION,
|
| 219 |
+
"orig_size": orig_size, "new_size": resized.size, "image_tokens": tokens}
|
| 220 |
+
return resized, record
|
| 221 |
+
|
| 222 |
+
|
| 223 |
+
def _normalise_options(options) -> list[dict[str, str]]:
|
| 224 |
+
normalised = []
|
| 225 |
+
for index, option in enumerate(options):
|
| 226 |
+
if isinstance(option, str):
|
| 227 |
+
normalised.append({"name": option, "description": option})
|
| 228 |
+
elif isinstance(option, dict) and "name" in option and "description" in option:
|
| 229 |
+
normalised.append({"name": str(option["name"]), "description": str(option["description"])})
|
| 230 |
+
elif isinstance(option, (tuple, list)) and len(option) == 2:
|
| 231 |
+
normalised.append({"name": str(option[0]), "description": str(option[1])})
|
| 232 |
+
else:
|
| 233 |
+
raise ValueError(f"option {index} must be text, a (name, description) pair, or a mapping with those keys")
|
| 234 |
+
if not 2 <= len(normalised) <= len(hopper_prompt.LETTERS):
|
| 235 |
+
raise ValueError("image decisions require 2 to 26 options")
|
| 236 |
+
return normalised
|
| 237 |
+
|
| 238 |
+
|
| 239 |
+
def render_prompt(question: str, options, n_images: int) -> str:
|
| 240 |
+
"""Render the image decision user turn with explicit letter-description pairs and thinking off."""
|
| 241 |
+
if not isinstance(question, str) or not question:
|
| 242 |
+
raise ValueError("question must be non-empty text")
|
| 243 |
+
if not isinstance(n_images, int) or isinstance(n_images, bool) or n_images < 1:
|
| 244 |
+
raise ValueError("n_images must be a positive integer")
|
| 245 |
+
normalised = _normalise_options(options)
|
| 246 |
+
shown = hopper_prompt.option_lines({"kind": "choice", "options": normalised})
|
| 247 |
+
payload = {"question": question,
|
| 248 |
+
"options": [{"letter": hopper_prompt.LETTERS[i], "description": text}
|
| 249 |
+
for i, (_, text) in enumerate(shown)]}
|
| 250 |
+
images = "\n".join(f"IMAGE {i + 1}\n{VISION_TOKEN}" for i in range(n_images))
|
| 251 |
+
return (f"{images}\n\n{json.dumps(payload, ensure_ascii=False)}\n\n"
|
| 252 |
+
"Reply with one option letter only; do not produce reasoning.")
|
| 253 |
+
|
| 254 |
+
|
| 255 |
+
def _decode_data_uri(uri: str) -> bytes:
|
| 256 |
+
try:
|
| 257 |
+
header, encoded = uri.split(",", 1)
|
| 258 |
+
except ValueError:
|
| 259 |
+
raise ValueError("bad image data URI: missing comma") from None
|
| 260 |
+
if not header.startswith("data:image/") or not header.endswith(";base64"):
|
| 261 |
+
raise ValueError("bad image data URI: expected data:image/...;base64,...")
|
| 262 |
+
# Reject an oversized payload before allocating its decoded representation. Padding means the
|
| 263 |
+
# estimate can exceed the true size by at most two bytes, so an exact check follows decoding.
|
| 264 |
+
if (len(encoded) // 4) * 3 > MAX_IMAGE_BYTES + 2:
|
| 265 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_BYTES} bytes")
|
| 266 |
+
try:
|
| 267 |
+
raw = base64.b64decode(encoded, validate=True)
|
| 268 |
+
except (binascii.Error, ValueError):
|
| 269 |
+
raise ValueError("bad image data URI: invalid base64 payload") from None
|
| 270 |
+
if len(raw) > MAX_IMAGE_BYTES:
|
| 271 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_BYTES} bytes")
|
| 272 |
+
return raw
|
| 273 |
+
|
| 274 |
+
|
| 275 |
+
def _image_bytes(source) -> bytes:
|
| 276 |
+
if isinstance(source, (bytes, bytearray, memoryview)):
|
| 277 |
+
raw = bytes(source)
|
| 278 |
+
if len(raw) > MAX_IMAGE_BYTES:
|
| 279 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_BYTES} bytes")
|
| 280 |
+
return raw
|
| 281 |
+
if isinstance(source, str) and source.startswith("data:"):
|
| 282 |
+
return _decode_data_uri(source)
|
| 283 |
+
if isinstance(source, (str, os.PathLike)):
|
| 284 |
+
path = Path(source)
|
| 285 |
+
try:
|
| 286 |
+
size = path.stat().st_size
|
| 287 |
+
except OSError as error:
|
| 288 |
+
raise ValueError(f"cannot read image path {path}: {error}") from error
|
| 289 |
+
if size > MAX_IMAGE_BYTES:
|
| 290 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_BYTES} bytes: {path}")
|
| 291 |
+
try:
|
| 292 |
+
raw = path.read_bytes()
|
| 293 |
+
except OSError as error:
|
| 294 |
+
raise ValueError(f"cannot read image path {path}: {error}") from error
|
| 295 |
+
if len(raw) > MAX_IMAGE_BYTES: # The file may have changed between stat and read.
|
| 296 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_BYTES} bytes: {path}")
|
| 297 |
+
return raw
|
| 298 |
+
raise TypeError("each image must be bytes, a filesystem path, or a base64 image data URI")
|
| 299 |
+
|
| 300 |
+
|
| 301 |
+
def _load_image(source, *, exif_transpose: bool = True) -> Image.Image:
|
| 302 |
+
raw = _image_bytes(source)
|
| 303 |
+
try:
|
| 304 |
+
with Image.open(io.BytesIO(raw)) as opened:
|
| 305 |
+
width, height = opened.size
|
| 306 |
+
if width * height > MAX_IMAGE_PIXELS:
|
| 307 |
+
raise ValueError(f"image exceeds {MAX_IMAGE_PIXELS} pixels: {width}x{height}")
|
| 308 |
+
opened.load()
|
| 309 |
+
ready = ImageOps.exif_transpose(opened) if exif_transpose else opened
|
| 310 |
+
# A loaded image owns its pixels after the file closes.
|
| 311 |
+
return ready
|
| 312 |
+
except ValueError:
|
| 313 |
+
raise
|
| 314 |
+
except (UnidentifiedImageError, OSError, SyntaxError) as error:
|
| 315 |
+
raise ValueError(f"cannot decode image: {error}") from error
|
| 316 |
+
|
| 317 |
+
|
| 318 |
+
def _output_field(output, name: str):
|
| 319 |
+
if isinstance(output, dict):
|
| 320 |
+
value = output.get(name)
|
| 321 |
+
else:
|
| 322 |
+
value = getattr(output, name, None)
|
| 323 |
+
if value is None:
|
| 324 |
+
raise ValueError(f"processor output is missing {name!r}")
|
| 325 |
+
return value
|
| 326 |
+
|
| 327 |
+
|
| 328 |
+
def _prepare(example: dict, processor, *, image_route: str,
|
| 329 |
+
content_class: str | None = None, loaded_images=None, cache=None,
|
| 330 |
+
timer=None, device=None) -> PreparedInput:
|
| 331 |
+
"""Prepare one decision after validating an explicit registered image route."""
|
| 332 |
+
if image_route not in IMAGE_ROUTES:
|
| 333 |
+
raise ValueError(f"image_route must be one of {IMAGE_ROUTES}, got {image_route!r}")
|
| 334 |
+
try:
|
| 335 |
+
question, options = example["question"], example["options"]
|
| 336 |
+
except (KeyError, TypeError) as error:
|
| 337 |
+
raise ValueError("example must contain question and options") from error
|
| 338 |
+
sources = example.get("images")
|
| 339 |
+
if sources is None and "image" in example:
|
| 340 |
+
sources = [example["image"]]
|
| 341 |
+
if not isinstance(sources, (list, tuple)) or not sources:
|
| 342 |
+
raise ValueError("example must contain a non-empty images list")
|
| 343 |
+
|
| 344 |
+
normalised = _normalise_options(options)
|
| 345 |
+
if content_class is not None:
|
| 346 |
+
raise ValueError("content_class overrides are forbidden; routing uses observable image content")
|
| 347 |
+
# The historical native read path used decoded pixel order directly. Keep
|
| 348 |
+
# that exact behavior while capped retains its established EXIF transpose.
|
| 349 |
+
timer = timer or PhaseTimer()
|
| 350 |
+
with timer.phase("decode_load"):
|
| 351 |
+
loaded = loaded_images if loaded_images is not None else [
|
| 352 |
+
_load_image(source, exif_transpose=image_route == "capped") for source in sources
|
| 353 |
+
]
|
| 354 |
+
with timer.phase("processor_preprocess"):
|
| 355 |
+
ready_images, records = [], []
|
| 356 |
+
for image in loaded:
|
| 357 |
+
routed_class = route_content(image)
|
| 358 |
+
if image_route == "capped":
|
| 359 |
+
ready, record = resize_for_budget(image, routed_class)
|
| 360 |
+
else:
|
| 361 |
+
ready = image if image.mode == "RGB" else image.convert("RGB")
|
| 362 |
+
record = {
|
| 363 |
+
"image_route": "native",
|
| 364 |
+
"content_class": routed_class,
|
| 365 |
+
"orig_size": image.size,
|
| 366 |
+
"new_size": ready.size,
|
| 367 |
+
"image_tokens": image_tokens(ready.height, ready.width),
|
| 368 |
+
}
|
| 369 |
+
ready_images.append(ready)
|
| 370 |
+
records.append(record)
|
| 371 |
+
|
| 372 |
+
with timer.phase("tokenise_chat_template"):
|
| 373 |
+
user = render_prompt(question, normalised, len(ready_images))
|
| 374 |
+
if cache is not None:
|
| 375 |
+
prompt = cache.prompt(user)
|
| 376 |
+
elif hasattr(processor, "apply_chat_template"):
|
| 377 |
+
messages = [{"role": "system", "content": IMAGE_SYSTEM}, {"role": "user", "content": user}]
|
| 378 |
+
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
| 379 |
+
enable_thinking=False)
|
| 380 |
+
else:
|
| 381 |
+
prompt = user
|
| 382 |
+
tokenizer = getattr(processor, "tokenizer", None)
|
| 383 |
+
if tokenizer is None:
|
| 384 |
+
raise ValueError("processor must expose its tokenizer for option-letter readout")
|
| 385 |
+
token_ids = (cache.letter_ids(len(normalised)) if cache is not None else
|
| 386 |
+
hopper_prompt.letter_token_ids(tokenizer)[:len(normalised)])
|
| 387 |
+
# Fast torch image processors accept device; slow/PIL processors do not.
|
| 388 |
+
image_processor = getattr(processor, "image_processor", None)
|
| 389 |
+
fast = image_processor is not None and (
|
| 390 |
+
getattr(image_processor, "backend", None) == "torch" or
|
| 391 |
+
"Fast" in type(image_processor).__name__ or
|
| 392 |
+
any(base.__name__ == "BaseImageProcessorFast" for base in type(image_processor).__mro__))
|
| 393 |
+
kwargs = {"images_kwargs": {"device": device}} if fast and device is not None else {}
|
| 394 |
+
if cache is not None:
|
| 395 |
+
cache.tokenizer.timer = timer
|
| 396 |
+
token_before = timer.seconds["tokenise_chat_template"] if timer.seconds is not None else 0.0
|
| 397 |
+
with timer.phase("processor_preprocess"):
|
| 398 |
+
output = processor(text=[prompt], images=ready_images, padding=True, return_tensors="pt", **kwargs)
|
| 399 |
+
if timer.seconds is not None:
|
| 400 |
+
timer.seconds["processor_preprocess"] -= timer.seconds["tokenise_chat_template"] - token_before
|
| 401 |
+
return PreparedInput(input_ids=_output_field(output, "input_ids"),
|
| 402 |
+
pixel_values=_output_field(output, "pixel_values"),
|
| 403 |
+
image_grid_thw=_output_field(output, "image_grid_thw"),
|
| 404 |
+
resize_records=records, letter_token_ids=token_ids, prompt=prompt,
|
| 405 |
+
mm_token_type_ids=(output.get("mm_token_type_ids") if isinstance(output, dict)
|
| 406 |
+
else getattr(output, "mm_token_type_ids", None)),
|
| 407 |
+
image_route=image_route)
|
| 408 |
+
|
| 409 |
+
|
| 410 |
+
def prepare(example: dict, processor, content_class: str | None = None) -> PreparedInput:
|
| 411 |
+
"""Prepare the existing capped route shared by capped training and serving."""
|
| 412 |
+
return _prepare(example, processor, image_route="capped", content_class=content_class)
|
| 413 |
+
|
| 414 |
+
|
| 415 |
+
def prepare_native(example: dict, processor, **kwargs) -> PreparedInput:
|
| 416 |
+
"""Prepare processor-default full-resolution images without budget resizing."""
|
| 417 |
+
return _prepare(example, processor, image_route="native", **kwargs)
|
| 418 |
+
|
| 419 |
+
|
| 420 |
+
def prepare_for_route(example: dict, processor, route: str) -> PreparedInput:
|
| 421 |
+
"""Dispatch one example through exactly one registered image route."""
|
| 422 |
+
if route == "native":
|
| 423 |
+
return prepare_native(example, processor)
|
| 424 |
+
if route == "capped":
|
| 425 |
+
return prepare(example, processor)
|
| 426 |
+
raise ValueError(f"image_route must be one of {IMAGE_ROUTES}, got {route!r}")
|
| 427 |
+
|
| 428 |
+
|
| 429 |
+
def last_position_logits(model, prepared, *, timer=None, forward_call=None):
|
| 430 |
+
"""Run exactly one last-position vocabulary projection for every local reader."""
|
| 431 |
+
try:
|
| 432 |
+
device = next(model.parameters()).device
|
| 433 |
+
except (StopIteration, AttributeError):
|
| 434 |
+
device = None
|
| 435 |
+
|
| 436 |
+
timer = timer or PhaseTimer()
|
| 437 |
+
|
| 438 |
+
def move(value):
|
| 439 |
+
return move_to_device(value, device)
|
| 440 |
+
|
| 441 |
+
with timer.phase("h2d_copy"):
|
| 442 |
+
kwargs = {
|
| 443 |
+
"input_ids": move(prepared.input_ids),
|
| 444 |
+
"pixel_values": move(prepared.pixel_values),
|
| 445 |
+
"image_grid_thw": move(prepared.image_grid_thw),
|
| 446 |
+
"use_cache": False,
|
| 447 |
+
**({"mm_token_type_ids": move(prepared.mm_token_type_ids)}
|
| 448 |
+
if getattr(prepared, "mm_token_type_ids", None) is not None else {}),
|
| 449 |
+
"return_dict": True,
|
| 450 |
+
}
|
| 451 |
+
readout_kwargs = getattr(model, "_image_last_position_kwargs", None)
|
| 452 |
+
if readout_kwargs is None:
|
| 453 |
+
parameters = inspect.signature(model.forward).parameters
|
| 454 |
+
if "logits_to_keep" in parameters or any(
|
| 455 |
+
parameter.kind == parameter.VAR_KEYWORD for parameter in parameters.values()
|
| 456 |
+
):
|
| 457 |
+
readout_kwargs = {"logits_to_keep": 1}
|
| 458 |
+
elif "num_logits_to_keep" in parameters:
|
| 459 |
+
readout_kwargs = {"num_logits_to_keep": 1}
|
| 460 |
+
else:
|
| 461 |
+
raise ValueError("model does not expose a last-position logits readout")
|
| 462 |
+
model._image_last_position_kwargs = readout_kwargs
|
| 463 |
+
kwargs.update(readout_kwargs)
|
| 464 |
+
with timer.phase("forward"):
|
| 465 |
+
output = model(**kwargs) if forward_call is None else forward_call(lambda: model(**kwargs))
|
| 466 |
+
logits = output["logits"] if isinstance(output, dict) else output.logits
|
| 467 |
+
if logits.ndim != 3 or logits.shape[0] != 1 or logits.shape[1] != 1:
|
| 468 |
+
raise ValueError("model must return exactly [1, 1, vocabulary] logits")
|
| 469 |
+
return logits[0, 0, :]
|
| 470 |
+
|
| 471 |
+
|
| 472 |
+
def calibrated_read(model, prepared: PreparedInput, *, temperature: float) -> tuple[list[float], list[float]]:
|
| 473 |
+
"""Return raw and checkpoint-artifact-calibrated option probabilities."""
|
| 474 |
+
logits = last_position_logits(model, prepared)
|
| 475 |
+
import torch
|
| 476 |
+
|
| 477 |
+
raw = option_log_probs(logits, prepared.letter_token_ids, validate_finite=False).exp()
|
| 478 |
+
calibrated = option_log_probs(logits, prepared.letter_token_ids, temperature,
|
| 479 |
+
validate_finite=False).exp()
|
| 480 |
+
values = torch.stack((raw, calibrated)).detach().cpu().tolist()
|
| 481 |
+
if not all(math.isfinite(value) for row in values for value in row):
|
| 482 |
+
raise ValueError("option probabilities are non-finite")
|
| 483 |
+
return values[0], values[1]
|
| 484 |
+
|
| 485 |
+
|
| 486 |
+
def option_log_probs(logits_last, letter_token_ids: list[int], temperature: float = 1.0, *,
|
| 487 |
+
validate_finite: bool = True):
|
| 488 |
+
"""Differentiable FP32 log-softmax over the served option-letter rows.
|
| 489 |
+
|
| 490 |
+
Training uses this function directly and :func:`option_probs` is only its
|
| 491 |
+
exponentiated, detached presentation form. Keeping selection, validation,
|
| 492 |
+
dtype conversion and temperature here prevents a train/serve readout fork.
|
| 493 |
+
"""
|
| 494 |
+
import torch
|
| 495 |
+
|
| 496 |
+
if not math.isfinite(temperature) or temperature <= 0:
|
| 497 |
+
raise ValueError("temperature must be finite and greater than zero")
|
| 498 |
+
if not letter_token_ids:
|
| 499 |
+
raise ValueError("letter_token_ids must not be empty")
|
| 500 |
+
logits = torch.as_tensor(logits_last)
|
| 501 |
+
if logits.ndim != 1:
|
| 502 |
+
raise ValueError("logits_last must be a one-dimensional vocabulary vector")
|
| 503 |
+
if any(isinstance(index, bool) or not isinstance(index, int) or not 0 <= index < logits.numel()
|
| 504 |
+
for index in letter_token_ids):
|
| 505 |
+
raise ValueError("letter_token_ids must be valid vocabulary indices")
|
| 506 |
+
try:
|
| 507 |
+
selected = logits[list(letter_token_ids)].to(dtype=torch.float32) / temperature
|
| 508 |
+
except (IndexError, TypeError) as error:
|
| 509 |
+
raise ValueError("letter_token_ids must be valid vocabulary indices") from error
|
| 510 |
+
log_probabilities = torch.log_softmax(selected, dim=-1)
|
| 511 |
+
if validate_finite and not bool(torch.isfinite(log_probabilities).all()):
|
| 512 |
+
raise ValueError("option probabilities are non-finite")
|
| 513 |
+
return log_probabilities
|
| 514 |
+
|
| 515 |
+
|
| 516 |
+
def option_probs(logits_last, letter_token_ids: list[int], temperature: float = 1.0) -> list[float]:
|
| 517 |
+
"""FP32 softmax over only the displayed option-letter logits."""
|
| 518 |
+
values = option_log_probs(logits_last, letter_token_ids, temperature, validate_finite=False).exp().detach().cpu().tolist()
|
| 519 |
+
if not all(math.isfinite(value) for value in values):
|
| 520 |
+
raise ValueError("option probabilities are non-finite")
|
| 521 |
+
return values
|
open_decisions/scoring/__init__.py
ADDED
|
File without changes
|
open_decisions/scoring/prompt.py
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""How an example becomes a prompt. Shared by every backend, so it imports no ML framework."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
LETTERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
| 6 |
+
SYSTEM = ("You make decisions about a document under a policy. Read only what is written in the document. "
|
| 7 |
+
"Reply with the letter of the correct option and nothing else.")
|
| 8 |
+
ANSWERABLE = "Does the document state every fact that the policy needs in order to decide this case?"
|
| 9 |
+
YES_NO = [("yes", "yes"), ("no", "no")]
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def option_lines(example):
|
| 13 |
+
"""(label, text shown) per option, in the example's order."""
|
| 14 |
+
if example["kind"] == "noul":
|
| 15 |
+
return [("true", "true"), ("false", "false")]
|
| 16 |
+
return [(o["name"], o["name"] if o["name"] == o["description"] else f"{o['name']}: {o['description']}")
|
| 17 |
+
for o in example["options"]]
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def user_prompt(example, question, options):
|
| 21 |
+
listing = "\n".join(f"{LETTERS[i]}. {text}" for i, (_, text) in enumerate(options))
|
| 22 |
+
document = f"DOCUMENT\n{example['document']}\n\n" if example["document"] else "" # bare questions have none
|
| 23 |
+
return (f"POLICY\n{example['policy']}\n\n{document}"
|
| 24 |
+
f"QUESTION\n{question}\n\nOPTIONS\n{listing}")
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def prompt_ids(tokenizer, user):
|
| 28 |
+
messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}]
|
| 29 |
+
ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False)
|
| 30 |
+
return list(ids if isinstance(ids, list) else ids["input_ids"]) # some tokenizer versions return a mapping
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def letter_token_ids(tokenizer):
|
| 34 |
+
ids = [tokenizer.encode(letter, add_special_tokens=False) for letter in LETTERS]
|
| 35 |
+
if any(len(i) != 1 for i in ids):
|
| 36 |
+
raise ValueError("Every option letter must be a single token for this tokenizer.")
|
| 37 |
+
return [i[0] for i in ids]
|
requirements.lock
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Generated from the captured training-image freeze; do not edit.
|
| 2 |
+
# python: 3.12.6
|
| 3 |
+
# torch-cuda: 12.8
|
| 4 |
+
# freeze-sha256: 1470ee9c425f71586f73783ba29b540ec224a9687eb612761a347bf7b56f3194
|
| 5 |
+
accelerate==1.15.0
|
| 6 |
+
aiohappyeyeballs==2.4.3
|
| 7 |
+
aiohttp==3.10.8
|
| 8 |
+
aiosignal==1.3.1
|
| 9 |
+
annotated-doc==0.0.5
|
| 10 |
+
annotated-types==0.8.0
|
| 11 |
+
anyio==4.15.1
|
| 12 |
+
apache-tvm-ffi==0.1.12
|
| 13 |
+
attrs==24.2.0
|
| 14 |
+
causal-conv1d @ https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.7.0/causal_conv1d-1.7.0%2Bcu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=a1fcd5d022b67ddfc9e1c291f890004c1c081a03d66b532f6eebe31219168e0e
|
| 15 |
+
certifi==2024.8.30
|
| 16 |
+
click==8.5.0
|
| 17 |
+
cloudpickle==3.1.2
|
| 18 |
+
einops==0.8.2
|
| 19 |
+
fastapi==0.142.2
|
| 20 |
+
filelock==4.0.8
|
| 21 |
+
fla-core==0.5.2
|
| 22 |
+
flash-linear-attention==0.5.2
|
| 23 |
+
frozenlist==1.4.1
|
| 24 |
+
fsspec==2026.9.0
|
| 25 |
+
grpclib==0.4.7
|
| 26 |
+
h11==0.16.0
|
| 27 |
+
h2==4.1.0
|
| 28 |
+
hf-xet==1.6.0
|
| 29 |
+
hpack==4.0.0
|
| 30 |
+
httpcore==1.0.9
|
| 31 |
+
httpx==0.28.1
|
| 32 |
+
huggingface_hub==1.33.0
|
| 33 |
+
hyperframe==6.0.1
|
| 34 |
+
idna==3.10
|
| 35 |
+
Jinja2==3.1.6
|
| 36 |
+
markdown-it-py==4.2.0
|
| 37 |
+
MarkupSafe==3.0.3
|
| 38 |
+
mdurl==0.1.2
|
| 39 |
+
ml_dtypes==0.6.0
|
| 40 |
+
mpmath==1.3.0
|
| 41 |
+
multidict==6.1.0
|
| 42 |
+
networkx==3.7
|
| 43 |
+
ninja==1.13.2
|
| 44 |
+
numpy==2.5.3
|
| 45 |
+
nvidia-cublas-cu12==12.8.4.1
|
| 46 |
+
nvidia-cuda-cupti-cu12==12.8.90
|
| 47 |
+
nvidia-cuda-nvrtc-cu12==12.8.93
|
| 48 |
+
nvidia-cuda-runtime-cu12==12.8.90
|
| 49 |
+
nvidia-cudnn-cu12==9.10.2.21
|
| 50 |
+
nvidia-cufft-cu12==11.3.3.83
|
| 51 |
+
nvidia-cufile-cu12==1.13.1.3
|
| 52 |
+
nvidia-curand-cu12==10.3.9.90
|
| 53 |
+
nvidia-cusolver-cu12==11.7.3.90
|
| 54 |
+
nvidia-cusparse-cu12==12.5.8.93
|
| 55 |
+
nvidia-cusparselt-cu12==0.7.1
|
| 56 |
+
nvidia-nccl-cu12==2.27.3
|
| 57 |
+
nvidia-nvjitlink-cu12==12.8.93
|
| 58 |
+
nvidia-nvtx-cu12==12.8.90
|
| 59 |
+
opentelemetry-api==1.45.0
|
| 60 |
+
packaging==26.3
|
| 61 |
+
peft==0.21.0
|
| 62 |
+
pillow==12.3.0
|
| 63 |
+
pip==24.3.1
|
| 64 |
+
protobuf==5.29.2
|
| 65 |
+
psutil==7.2.2
|
| 66 |
+
pydantic==2.13.5
|
| 67 |
+
pydantic_core==2.46.5
|
| 68 |
+
Pygments==2.21.0
|
| 69 |
+
python-multipart==0.0.32
|
| 70 |
+
PyYAML==6.0.3
|
| 71 |
+
regex==2026.9.29
|
| 72 |
+
rich==15.0.0
|
| 73 |
+
safetensors==0.8.0
|
| 74 |
+
setuptools==84.0.0
|
| 75 |
+
shellingham==1.5.4
|
| 76 |
+
starlette==1.7.0
|
| 77 |
+
sympy==1.14.0
|
| 78 |
+
tilelang==0.1.15
|
| 79 |
+
tokenizers==0.23.2
|
| 80 |
+
torch==2.8.0
|
| 81 |
+
torch_c_dlpack_ext==0.1.5
|
| 82 |
+
torchvision==0.23.0
|
| 83 |
+
tqdm==4.70.1
|
| 84 |
+
transformers==5.17.0
|
| 85 |
+
triton==3.4.0
|
| 86 |
+
typer==0.27.2
|
| 87 |
+
typing-inspection==0.4.4
|
| 88 |
+
typing_extensions==4.16.0
|
| 89 |
+
uv==0.5.14
|
| 90 |
+
uvicorn==0.54.0
|
| 91 |
+
wheel==0.45.1
|
| 92 |
+
yarl==1.13.1
|
| 93 |
+
z3-solver==4.15.4.0
|
requirements.lock.sha256
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
974e3a19773d67b4de84842ffa4a3a537e2886240cdfb97d94216159804aa7e2 requirements.lock
|
requirements.txt
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Direct dependencies at the exact versions of the evaluated image (Python 3.12, Linux x86_64, CUDA 12.8).
|
| 2 |
+
# requirements.lock is the complete freeze.
|
| 3 |
+
torch==2.8.0
|
| 4 |
+
torchvision==0.23.0
|
| 5 |
+
transformers==5.17.0
|
| 6 |
+
tokenizers==0.23.2
|
| 7 |
+
safetensors==0.8.0
|
| 8 |
+
accelerate==1.15.0
|
| 9 |
+
huggingface_hub==1.33.0
|
| 10 |
+
hf-xet==1.6.0
|
| 11 |
+
pillow==12.3.0
|
| 12 |
+
numpy==2.5.3
|
| 13 |
+
flash-linear-attention==0.5.2
|
| 14 |
+
fla-core==0.5.2
|
| 15 |
+
causal-conv1d @ https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.7.0/causal_conv1d-1.7.0%2Bcu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl#sha256=a1fcd5d022b67ddfc9e1c291f890004c1c081a03d66b532f6eebe31219168e0e
|
| 16 |
+
triton==3.4.0
|
| 17 |
+
einops==0.8.2
|
| 18 |
+
peft==0.21.0
|