Text Generation
MLX
Safetensors
English
information-extraction
structured-output
json
small-language-model
apple-silicon
from-scratch
post-training
research
Instructions to use wzebrowski/OpenSML-JSONLite-175M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use wzebrowski/OpenSML-JSONLite-175M with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("wzebrowski/OpenSML-JSONLite-175M") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use wzebrowski/OpenSML-JSONLite-175M with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "wzebrowski/OpenSML-JSONLite-175M" --prompt "Once upon a time"
- Atomic Chat
Present released model metrics without release comparisons
Browse files- README.md +17 -17
- SHA256SUMS +1 -1
README.md
CHANGED
|
@@ -29,9 +29,9 @@ datasets:
|
|
| 29 |
|
| 30 |
[Run locally](RUN_GUIDE.md) 路 [Technical report](TECHNICAL_REPORT.md) 路 [Results](JSONLITE_RESULTS.json) 路 [Provenance](checkpoint_provenance.json)
|
| 31 |
|
| 32 |
-
**
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
## Run locally
|
| 37 |
|
|
@@ -327,23 +327,23 @@ The full inherited lineage and earlier SFT data are documented in the technical
|
|
| 327 |
|
| 328 |
## Native JSON evaluation
|
| 329 |
|
| 330 |
-
|
| 331 |
|
| 332 |
-
| Metric |
|
| 333 |
-
| --- | ---: |
|
| 334 |
-
| Entire request correct | **88.02%** |
|
| 335 |
-
| Field accuracy | **94.17%** |
|
| 336 |
-
| Extracted-value F1 | **92.88%** |
|
| 337 |
-
| Extracted-value precision | **92.42%** |
|
| 338 |
-
| Extracted-value recall | **93.35%** |
|
| 339 |
-
| Missing-field accuracy | **95.22%** |
|
| 340 |
-
| Schema-valid JSON | **100.00%** |
|
| 341 |
|
| 342 |
-
|
| 343 |
|
| 344 |
-
The separate observed 60-case playground scores **49/60 (81.67%)**
|
| 345 |
|
| 346 |
-
|
| 347 |
|
| 348 |

|
| 349 |
|
|
@@ -351,10 +351,10 @@ The main checkpoint was selected for aggregate improvement despite failing the s
|
|
| 351 |
|
| 352 |
A focused English extraction research model for documented flat schemas and mostly SNIPS fields. It can shorten, omit, invent or misassign values, confuse roles, miss list members, and fail on corrections or unfamiliar definitions. Valid JSON does not prove correct extraction. Long documents, nested objects, numeric/boolean outputs, multilingual tasks, general chat and prompt-injection robustness are outside the verified scope. `success` means structural validity and natural completion, not semantic correctness.
|
| 353 |
|
| 354 |
-
##
|
| 355 |
|
| 356 |
Selected BF16 weight SHA-256:
|
| 357 |
|
| 358 |
`781597baeb9b51d535fc27cd6f941dbc202e01d62c8d963587a3f3d565128560`
|
| 359 |
|
| 360 |
-
Includes `model.safetensors`,
|
|
|
|
| 29 |
|
| 30 |
[Run locally](RUN_GUIDE.md) 路 [Technical report](TECHNICAL_REPORT.md) 路 [Results](JSONLITE_RESULTS.json) 路 [Provenance](checkpoint_provenance.json)
|
| 31 |
|
| 32 |
+
**A small language model that generates complete JSON directly from source text and a supplied JSON Schema.** It makes one autoregressive call; parsing/schema validation follows generation. No JSON assembly, output repair, constrained decoder or secondary model is used.
|
| 33 |
|
| 34 |
+
The model was fine-tuned for grounded question answering, exact and focused field extraction, and direct JSON generation. Its final joint JSON extraction stage combines original training replay, multiple-value examples and authored prose. Weights are provided as `model.safetensors`. The pretrained ancestor is [OpenSML-Base-175M](https://huggingface.co/OpenSML/OpenSML-Base-175M).
|
| 35 |
|
| 36 |
## Run locally
|
| 37 |
|
|
|
|
| 327 |
|
| 328 |
## Native JSON evaluation
|
| 329 |
|
| 330 |
+
The released model was evaluated on 1,444 SNIPS-derived source-text and JSON Schema requests using raw greedy generation:
|
| 331 |
|
| 332 |
+
| Metric | Released model |
|
| 333 |
+
| --- | ---: |
|
| 334 |
+
| Entire request correct | **88.02%** |
|
| 335 |
+
| Field accuracy | **94.17%** |
|
| 336 |
+
| Extracted-value F1 | **92.88%** |
|
| 337 |
+
| Extracted-value precision | **92.42%** |
|
| 338 |
+
| Extracted-value recall | **93.35%** |
|
| 339 |
+
| Missing-field accuracy | **95.22%** |
|
| 340 |
+
| Schema-valid JSON | **100.00%** |
|
| 341 |
|
| 342 |
+
The model answers **1,271/1,444 requests completely correctly**. Paired-source correctness is **561/694**. The evaluation records **53 false abstentions**, **73 missed missing fields**, and **24 unsupported predicted values**. All 1,444 outputs parse, validate and stop naturally; there are no output-budget failures.
|
| 343 |
|
| 344 |
+
The separate observed 60-case playground scores **49/60 (81.67%)**. Its original 54-case subset scores 46/54; the six added three/four/five-item probes score 3/6. Three larger-list probes fail. The compact 240-request development panel scores 178/240 (74.17%). These panels measure different scopes and must not be pooled. They have prior research exposure and are not fresh blind benchmarks or an external leaderboard.
|
| 345 |
|
| 346 |
+
These aggregate scores do not establish universal retention or production reliability. Checkpoint-selection checks and remaining failures are documented in the technical report.
|
| 347 |
|
| 348 |

|
| 349 |
|
|
|
|
| 351 |
|
| 352 |
A focused English extraction research model for documented flat schemas and mostly SNIPS fields. It can shorten, omit, invent or misassign values, confuse roles, miss list members, and fail on corrections or unfamiliar definitions. Valid JSON does not prove correct extraction. Long documents, nested objects, numeric/boolean outputs, multilingual tasks, general chat and prompt-injection robustness are outside the verified scope. `success` means structural validity and natural completion, not semantic correctness.
|
| 353 |
|
| 354 |
+
## Released model identity and package
|
| 355 |
|
| 356 |
Selected BF16 weight SHA-256:
|
| 357 |
|
| 358 |
`781597baeb9b51d535fc27cd6f941dbc202e01d62c8d963587a3f3d565128560`
|
| 359 |
|
| 360 |
+
Includes `model.safetensors`, tokenizer, native MLX runner, source bundle, 60 source/schema/reference examples, raw evaluation records, checksums and provenance. Standalone verification checks generated strings and stop reasons against the saved 60-case protocol; this is packaging parity, not an independent accuracy benchmark. Source uses Apache-2.0; upstream terms apply separately; no trained-weight license is assigned by this release.
|
SHA256SUMS
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
74a074f90bd5e7a26d5f6cd5757ddc1fbfcc72e259e0b23a210fa48d91b35fdb JSONLITE_RESULTS.json
|
| 5 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
|
| 6 |
f53194e53a2af5ec857337e8213397ea4f81a7e02d2dbcf0384a00cf1cf7f76f PACKAGING_VERIFICATION.json
|
| 7 |
-
|
| 8 |
5ee8f11f0e87becf1a29c44e2b0f3dacc2c779712b635558be57dd7ef44455d3 RUN_GUIDE.md
|
| 9 |
c6191baf937bce61f1b747acfbcb975a17eef6e8a598682a751a1031d1cfee23 SOURCE_NOTICE.md
|
| 10 |
6088771ab13fa1080c0ca0f31649deb9f417597d7c2055b0f388e86f41f2c415 TECHNICAL_REPORT.md
|
|
|
|
| 4 |
74a074f90bd5e7a26d5f6cd5757ddc1fbfcc72e259e0b23a210fa48d91b35fdb JSONLITE_RESULTS.json
|
| 5 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE
|
| 6 |
f53194e53a2af5ec857337e8213397ea4f81a7e02d2dbcf0384a00cf1cf7f76f PACKAGING_VERIFICATION.json
|
| 7 |
+
6a4ca570696577a60e5a9b7ffdcda04edf0b9a75322cc4db33e3f92d8ac79931 README.md
|
| 8 |
5ee8f11f0e87becf1a29c44e2b0f3dacc2c779712b635558be57dd7ef44455d3 RUN_GUIDE.md
|
| 9 |
c6191baf937bce61f1b747acfbcb975a17eef6e8a598682a751a1031d1cfee23 SOURCE_NOTICE.md
|
| 10 |
6088771ab13fa1080c0ca0f31649deb9f417597d7c2055b0f388e86f41f2c415 TECHNICAL_REPORT.md
|