wzebrowski commited on
Commit
dda072e
·
verified ·
1 Parent(s): 788dfdc

Update JSONLite descendant training and current evaluation metrics

Browse files
Files changed (5) hide show
  1. PACKAGING_VERIFICATION.json +1 -1
  2. README.md +5 -17
  3. RUN_GUIDE.md +5 -17
  4. SHA256SUMS +4 -4
  5. TECHNICAL_REPORT.md +5 -17
PACKAGING_VERIFICATION.json CHANGED
@@ -18,5 +18,5 @@
18
  "scope": "Packaging smoke only. A plausible but repetitive continuation is not an accuracy measurement.",
19
  "matches_historical_prefix": true
20
  },
21
- "documentation_update": "Main JSONLite descendant includes additional cardinality/replay distillation SFT; base weights, tokenizer, runner and bundle unchanged."
22
  }
 
18
  "scope": "Packaging smoke only. A plausible but repetitive continuation is not an accuracy measurement.",
19
  "matches_historical_prefix": true
20
  },
21
+ "documentation_update": "Latest JSONLite joint extraction SFT and matched metrics; base weights, tokenizer, runner and bundle unchanged."
22
  }
README.md CHANGED
@@ -175,20 +175,8 @@ not establish blanket underlying-content redistribution clearance.
175
 
176
  ## Native JSON descendant update
177
 
178
- The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M)
179
- continues this pretrained base through grounded QA, requested-field and focused extraction,
180
- a 15,000-presentation native JSON foundation, and an additional focused extraction SFT.
181
- The added stage combines original JSON TRAIN replay, matched zero/one/two/three-value
182
- examples, and training-only frozen-parent distillation. The main model generates the
183
- entire JSON object itself from source text and a supplied schema, with no assembly or repair.
184
-
185
- On the same previously observed 1,444-request native JSON evaluation, complete-request
186
- correctness is **1,199/1,444 (83.03%)**, compared with 1,191/1,444 for the prior native release.
187
- Field accuracy is 91.36%, value F1 90.40%, missing-field accuracy 92.79%, and schema validity
188
- 99.79%. The canonical subset is 1,163/1,388 (83.79%); observed playground correctness is
189
- 44/54. These are descendant results, not scores of this pretrained base. Some domains and
190
- value precision/recall regress slightly, and strict automatic retention gates were not met.
191
- See the [JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md)
192
- for full training provenance, matched comparison and limitations.
193
-
194
- The base weights, tokenizer, inference runner and source bundle are unchanged by this documentation update.
 
175
 
176
  ## Native JSON descendant update
177
 
178
+ The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M) adds a joint full-weight JSON extraction SFT to its earlier grounded-QA, extraction and native JSON ancestry. The latest stage mixes original source/schema/JSON TRAIN replay, multiple-value examples and authored prose. The released main checkpoint inherits 19,200 presentations from a 21,144-presentation experiment. It directly generates the complete JSON object; validation follows without assembly or repair.
179
+
180
+ On the same previously observed 1,444-request SNIPS-derived evaluation, the descendant achieves **1,271/1,444 (88.02%) complete correctness**, **94.17% field accuracy**, **92.88% extracted-value F1**, **92.42% precision**, **93.35% recall**, **95.22% missing-field accuracy** and **100% schema validity**. The observed 60-case playground scores 49/60. Compared with the previous release, 96 requests improve and 24 regress. These are descendant results on repeatedly observed research panels, not scores of this base or a fresh blind benchmark. See the [current JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md) for the full provenance, protocol and limitations.
181
+
182
+ The pretrained base weights, tokenizer, inference runner and source bundle are unchanged.
 
 
 
 
 
 
 
 
 
 
 
 
RUN_GUIDE.md CHANGED
@@ -60,20 +60,8 @@ following or extraction. See the model card and report for measured scope.
60
 
61
  ## Native JSON descendant update
62
 
63
- The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M)
64
- continues this pretrained base through grounded QA, requested-field and focused extraction,
65
- a 15,000-presentation native JSON foundation, and an additional focused extraction SFT.
66
- The added stage combines original JSON TRAIN replay, matched zero/one/two/three-value
67
- examples, and training-only frozen-parent distillation. The main model generates the
68
- entire JSON object itself from source text and a supplied schema, with no assembly or repair.
69
-
70
- On the same previously observed 1,444-request native JSON evaluation, complete-request
71
- correctness is **1,199/1,444 (83.03%)**, compared with 1,191/1,444 for the prior native release.
72
- Field accuracy is 91.36%, value F1 90.40%, missing-field accuracy 92.79%, and schema validity
73
- 99.79%. The canonical subset is 1,163/1,388 (83.79%); observed playground correctness is
74
- 44/54. These are descendant results, not scores of this pretrained base. Some domains and
75
- value precision/recall regress slightly, and strict automatic retention gates were not met.
76
- See the [JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md)
77
- for full training provenance, matched comparison and limitations.
78
-
79
- The base weights, tokenizer, inference runner and source bundle are unchanged by this documentation update.
 
60
 
61
  ## Native JSON descendant update
62
 
63
+ The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M) adds a joint full-weight JSON extraction SFT to its earlier grounded-QA, extraction and native JSON ancestry. The latest stage mixes original source/schema/JSON TRAIN replay, multiple-value examples and authored prose. The released main checkpoint inherits 19,200 presentations from a 21,144-presentation experiment. It directly generates the complete JSON object; validation follows without assembly or repair.
64
+
65
+ On the same previously observed 1,444-request SNIPS-derived evaluation, the descendant achieves **1,271/1,444 (88.02%) complete correctness**, **94.17% field accuracy**, **92.88% extracted-value F1**, **92.42% precision**, **93.35% recall**, **95.22% missing-field accuracy** and **100% schema validity**. The observed 60-case playground scores 49/60. Compared with the previous release, 96 requests improve and 24 regress. These are descendant results on repeatedly observed research panels, not scores of this base or a fresh blind benchmark. See the [current JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md) for the full provenance, protocol and limitations.
66
+
67
+ The pretrained base weights, tokenizer, inference runner and source bundle are unchanged.
 
 
 
 
 
 
 
 
 
 
 
 
SHA256SUMS CHANGED
@@ -1,12 +1,12 @@
1
  59ee3f5b0b4e357e34a390a72febcc13eac5493fb4a4c2d7ecdf6ba5be324ff2 CITATION.cff
2
  9c6d4c439acf90f9187ba6cf11f070afb3ea1515d488b52c381adf9356a1d613 EVALUATION_RESULTS.json
3
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
4
- 27478e40538a7777e0e1f8aed83f16afa6d9cf47084f6bfd39c7f8a9df515fb9 PACKAGING_VERIFICATION.json
5
  14819fc05509546bb8218153cf8494c6a3c4e2e4744ce504bdc21ef0d7d1ed59 PRETRAINING_SAMPLES.json
6
- f97034ba3ecbbb05f200a48cb9813ea5eb9c9e46c19f04496db14b03251a3967 README.md
7
- 52863aa0d0b0fc25cad46c07830c244b9fa576f38b29fe39d2d086bbf8e78fae RUN_GUIDE.md
8
  c3bd058575a0313f716dabb8de8a3d7b729ff413dae5f74b06525c0737dcfac3 SOURCE_FILES.json
9
- 3ff056103137004972aaa69e1f50783b70534d951a905579cb72f151a4170d44 TECHNICAL_REPORT.md
10
  3235c99f09bfab6e5ee95e64a382a4975978ec7f8988d33a71d4aaeb0215bf15 THIRD_PARTY_NOTICES.md
11
  15a0e3b511b97503a611c0da838a7fde1c06ebed34c7103f7cda1782cc6378a4 checkpoint_metadata.json
12
  22eaa19ba2346558711ff5245b4a1e581a431b7b1c5df3c2342f52407ebbc74a checkpoint_provenance.json
 
1
  59ee3f5b0b4e357e34a390a72febcc13eac5493fb4a4c2d7ecdf6ba5be324ff2 CITATION.cff
2
  9c6d4c439acf90f9187ba6cf11f070afb3ea1515d488b52c381adf9356a1d613 EVALUATION_RESULTS.json
3
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
4
+ aa0e0453672382a7a0d7ec123d24ad83edde6d19909f141e8305235454d0d027 PACKAGING_VERIFICATION.json
5
  14819fc05509546bb8218153cf8494c6a3c4e2e4744ce504bdc21ef0d7d1ed59 PRETRAINING_SAMPLES.json
6
+ 2e008cfeba481b8f9b37de95df231665e04b72ecc44bf04447580a295fa10d5a README.md
7
+ b556a9c2a63f43c904af7460936cff4e6da9ece27d2877ddb8c5eac2346b7f2a RUN_GUIDE.md
8
  c3bd058575a0313f716dabb8de8a3d7b729ff413dae5f74b06525c0737dcfac3 SOURCE_FILES.json
9
+ c76d8a9799f36f917c463dadd9d77321241f28a99bd094702887ef2700f39bb9 TECHNICAL_REPORT.md
10
  3235c99f09bfab6e5ee95e64a382a4975978ec7f8988d33a71d4aaeb0215bf15 THIRD_PARTY_NOTICES.md
11
  15a0e3b511b97503a611c0da838a7fde1c06ebed34c7103f7cda1782cc6378a4 checkpoint_metadata.json
12
  22eaa19ba2346558711ff5245b4a1e581a431b7b1c5df3c2342f52407ebbc74a checkpoint_provenance.json
TECHNICAL_REPORT.md CHANGED
@@ -153,20 +153,8 @@ underlying-source redistribution clearance.
153
 
154
  ## Native JSON descendant update
155
 
156
- The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M)
157
- continues this pretrained base through grounded QA, requested-field and focused extraction,
158
- a 15,000-presentation native JSON foundation, and an additional focused extraction SFT.
159
- The added stage combines original JSON TRAIN replay, matched zero/one/two/three-value
160
- examples, and training-only frozen-parent distillation. The main model generates the
161
- entire JSON object itself from source text and a supplied schema, with no assembly or repair.
162
-
163
- On the same previously observed 1,444-request native JSON evaluation, complete-request
164
- correctness is **1,199/1,444 (83.03%)**, compared with 1,191/1,444 for the prior native release.
165
- Field accuracy is 91.36%, value F1 90.40%, missing-field accuracy 92.79%, and schema validity
166
- 99.79%. The canonical subset is 1,163/1,388 (83.79%); observed playground correctness is
167
- 44/54. These are descendant results, not scores of this pretrained base. Some domains and
168
- value precision/recall regress slightly, and strict automatic retention gates were not met.
169
- See the [JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md)
170
- for full training provenance, matched comparison and limitations.
171
-
172
- The base weights, tokenizer, inference runner and source bundle are unchanged by this documentation update.
 
153
 
154
  ## Native JSON descendant update
155
 
156
+ The current [OpenSML-JSONLite-175M-Instruct](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M) adds a joint full-weight JSON extraction SFT to its earlier grounded-QA, extraction and native JSON ancestry. The latest stage mixes original source/schema/JSON TRAIN replay, multiple-value examples and authored prose. The released main checkpoint inherits 19,200 presentations from a 21,144-presentation experiment. It directly generates the complete JSON object; validation follows without assembly or repair.
157
+
158
+ On the same previously observed 1,444-request SNIPS-derived evaluation, the descendant achieves **1,271/1,444 (88.02%) complete correctness**, **94.17% field accuracy**, **92.88% extracted-value F1**, **92.42% precision**, **93.35% recall**, **95.22% missing-field accuracy** and **100% schema validity**. The observed 60-case playground scores 49/60. Compared with the previous release, 96 requests improve and 24 regress. These are descendant results on repeatedly observed research panels, not scores of this base or a fresh blind benchmark. See the [current JSONLite technical report](https://huggingface.co/OpenSML/OpenSML-JSONLite-175M/blob/main/TECHNICAL_REPORT.md) for the full provenance, protocol and limitations.
159
+
160
+ The pretrained base weights, tokenizer, inference runner and source bundle are unchanged.