Card: state how small this build's verification is (42 questions); long-input failure measured on E2B
Browse files
README.md
CHANGED
|
@@ -25,10 +25,13 @@ This repository is the text model converted to Core ML; the conversion is descri
|
|
| 25 |
|
| 26 |
- **Correct at ≤ 512 tokens.** On 28 real requests (42 questions), no clear-cut answer changes against the fp32 PyTorch
|
| 27 |
rebuild of the same model, on an M1 Max and an M4.
|
|
|
|
|
|
|
|
|
|
| 28 |
- **Speed (median request):** M4 Neural Engine 1.8 s (MLX on the GPU: 0.63 s); M1 Max 2.9 s (MLX: 0.81 s).
|
| 29 |
- **Use it** where the GPU should stay free or power matters (iPhone, iPad). On a Mac, the MLX build is faster.
|
| 30 |
- **Not included yet:** photo, voice and video (their encoders are separate models, still to convert), and inputs longer than
|
| 31 |
-
512 tokens (2,048-token programs
|
| 32 |
|
| 33 |
## Run it
|
| 34 |
|
|
|
|
| 25 |
|
| 26 |
- **Correct at ≤ 512 tokens.** On 28 real requests (42 questions), no clear-cut answer changes against the fp32 PyTorch
|
| 27 |
rebuild of the same model, on an M1 Max and an M4.
|
| 28 |
+
- **Verification is small so far.** Those 42 questions are this build's only accuracy check. The full evaluation suites
|
| 29 |
+
and the 145-request demo smoke test that the MLX build passed have not been run on it. If you need measured accuracy
|
| 30 |
+
across tasks, use [JohnP1/d1a-e4b-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-mlx-q8).
|
| 31 |
- **Speed (median request):** M4 Neural Engine 1.8 s (MLX on the GPU: 0.63 s); M1 Max 2.9 s (MLX: 0.81 s).
|
| 32 |
- **Use it** where the GPU should stay free or power matters (iPhone, iPad). On a Mac, the MLX build is faster.
|
| 33 |
- **Not included yet:** photo, voice and video (their encoders are separate models, still to convert), and inputs longer than
|
| 34 |
+
512 tokens (2,048-token programs gave wrong answers on the M4 Neural Engine, measured with D1A-E2B's chunks).
|
| 35 |
|
| 36 |
## Run it
|
| 37 |
|