How should verified expert layers for local LLMs be evaluated?

#1
by genia-dev - opened

MAT Nexus is an experimental orchestration layer for local language models.

It uses bounded deterministic experts only when a result can be mechanically verified. Otherwise, the request bypasses Nexus and reaches the original LLM unchanged.

The public dashboard compares local LLMs alone against the same models assisted by Nexus. It also separates audited results, diagnostic experiments, invalidated campaigns, and inconclusive outcomes.

I would appreciate technical feedback on three questions:

  1. Which verifiable task families should be evaluated next?
  2. What additional baselines or ablations would make the comparisons stronger?
  3. Which local models should be tested after Granite, Gemma, and Llama?

Dashboard:
https://huggingface.co/spaces/genia-dev/MAT-Nexus-Benchmark

Public methodology and results:
https://github.com/sxc3030-eng/mat-nexus-showcase

Sign up or log in to comment