JohnP1 commited on
Commit
114f98a
·
verified ·
1 Parent(s): a1fd999

Card: state how small this build's verification is (42 questions); long-input failure measured on E2B

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -25,10 +25,13 @@ This repository is the text model converted to Core ML; the conversion is descri
25
 
26
  - **Correct at ≤ 512 tokens.** On 28 real requests (42 questions), no clear-cut answer changes against the fp32 PyTorch
27
  rebuild of the same model, on an M1 Max and an M4.
 
 
 
28
  - **Speed (median request):** M4 Neural Engine 1.8 s (MLX on the GPU: 0.63 s); M1 Max 2.9 s (MLX: 0.81 s).
29
  - **Use it** where the GPU should stay free or power matters (iPhone, iPad). On a Mac, the MLX build is faster.
30
  - **Not included yet:** photo, voice and video (their encoders are separate models, still to convert), and inputs longer than
31
- 512 tokens (2,048-token programs give wrong answers on the M4 Neural Engine).
32
 
33
  ## Run it
34
 
 
25
 
26
  - **Correct at ≤ 512 tokens.** On 28 real requests (42 questions), no clear-cut answer changes against the fp32 PyTorch
27
  rebuild of the same model, on an M1 Max and an M4.
28
+ - **Verification is small so far.** Those 42 questions are this build's only accuracy check. The full evaluation suites
29
+ and the 145-request demo smoke test that the MLX build passed have not been run on it. If you need measured accuracy
30
+ across tasks, use [JohnP1/d1a-e4b-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-mlx-q8).
31
  - **Speed (median request):** M4 Neural Engine 1.8 s (MLX on the GPU: 0.63 s); M1 Max 2.9 s (MLX: 0.81 s).
32
  - **Use it** where the GPU should stay free or power matters (iPhone, iPad). On a Mac, the MLX build is faster.
33
  - **Not included yet:** photo, voice and video (their encoders are separate models, still to convert), and inputs longer than
34
+ 512 tokens (2,048-token programs gave wrong answers on the M4 Neural Engine, measured with D1A-E2B's chunks).
35
 
36
  ## Run it
37