Shape mismatch error when running the model
Issue: Shape mismatch error when running the model
Command used:
COREAI_CHUNK_THRESHOLD=1 swift run -c release llm-runner \
--model <path_to_model>/gemma4_12b_qat_decode_int4linsym_msdpa_g8 \
--prompt "What is the capital of France?" \
--max-tokens 64 \
--chunk-size 1
Building for production...
[Computing dependencies]
Build complete! (0.86 secs)
⏳ Preparing AI asset from source...CoreAIRuntime/NDArrayDescriptor.swift:139: Fatal error: Shape at dimension 1 of 256 is not a valid substitution for source shape 1
zsh: trace trap COREAI_CHUNK_THRESHOLD=1 swift run -c release llm-runner --model --prompt
Thanks for the report, @xht033 , and I'm sorry it sat here for three and a half months.
The command came from this card, and it was wrong: Apple's llm-runner cannot run these decode-only bundles with its default engine, and it still stops at "Preparing AI asset" with the same error on Apple's current main. Two ways work, both checked today on an M4 Max with macOS 27.0 and the current files: llm-runner with --inference-engine-variant coreai-sequential --chunk-size 1 --chunk-threshold 1, or CoreAIKit's ChatSession(bundleAt:); each answered "The capital of France is Paris." with the int4 bundle. I've rewritten the card's Usage section with both. If your copy is from June, download it again: the bundles were re-saved on 2026-09-14 because the June files no longer load on macOS 27.
Thanks again for reporting it.