training-lab / eval /local_intelligence /model_candidates.md
arach's picture
Sync local-intelligence eval harness
0d1866c verified
|
Raw History Blame Contribute Delete
674 Bytes

Recommended Open Models

Current shortlist for the Talkie local-intelligence eval pack on this machine:

  1. mlx-community/NVIDIA-Nemotron-3-Nano-4B-4bit
    • Plain mlx-lm model
    • Good fit for structured JSON tasks without pushing memory too hard
  2. mlx-community/gemma-4-e4b-it-mxfp8
    • Runs through mlx-vlm
    • Stronger modern general-purpose open model, but heavier than the Nemotron 4B path
  3. mlx-community/Qwen3.5-2B-4bit
    • Existing local baseline already partly in cache
    • Useful as the control model for before/after comparisons

Suggested first pass:

  • Nemotron 4B on all 24 cards
  • Gemma 4 E4B on all 24 cards
  • Qwen3.5 2B as baseline