Spaces:
Sleeping
fix(llm): KI-175 — demote nemotron to last resort in all chains
Browse filesUser verdict on nemotron: "the worst model, barely usable, last resort."
Live admin showed nemotron elected as primary for Brain Fast + Brain Main
+ Judge — because the chain election was picking the first HEALTHY model
in the chain, and nemotron sat at position #2 (BRAIN/FAST_BRAIN) or #3
(JUDGE). Whenever qwen had a transient blip the election fell to nemotron
and the sticky-primary held it there.
Fix: reorder chains so nemotron is always LAST. It now only serves when
every other chain member is simultaneously unavailable — fail-loud is
better than serving the worst model by default.
BRAIN_CHAIN:
qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron
FAST_BRAIN_CHAIN:
qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron
JUDGE_CHAIN:
mistral-large-3 → llama-4-maverick → nemotron
(judge order was already correct; chain unchanged)
Llama-4 Maverick added as a 3rd-fallback to FAST_BRAIN_CHAIN (previously
only in BRAIN+JUDGE). All chains now have 3 healthy non-nemotron options
before nemotron is reached.
Verification: imports + chain ordering confirmed via `python -c` smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@@ -250,19 +250,21 @@ BRAIN_CHAIN = [
|
|
| 250 |
# that have been "down" for 48+ consecutive probes (qwen3.5-122b,
|
| 251 |
# gpt-oss-120b, deepseek-v4-pro) so the election pool only contains
|
| 252 |
# demonstrably-healthy NIM models.
|
|
|
|
|
|
|
|
|
|
| 253 |
# Primary: Qwen 3-Next 80B — 5/5 recent probes ok, clean JSON, multilingual
|
| 254 |
"qwen/qwen3-next-80b-a3b-instruct",
|
| 255 |
-
# 1st fallback:
|
| 256 |
-
#
|
| 257 |
-
# cross-family-grading invariant if elevated to judge.
|
| 258 |
-
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
|
| 259 |
-
# 2nd fallback: Mistral Large 3 675B — recent 3/3 probes ok, different
|
| 260 |
-
# family (mistral). Also the judge primary; only reached when both Qwen +
|
| 261 |
-
# Nemotron are unavailable.
|
| 262 |
"mistralai/mistral-large-3-675b-instruct-2512",
|
| 263 |
-
#
|
| 264 |
-
# family (meta), keeps the chain alive through
|
| 265 |
"meta/llama-4-maverick-17b-128e-instruct",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 266 |
]
|
| 267 |
|
| 268 |
# Same chain for fast brain — Qwen 80B is already fast (~2s); no need for a
|
|
@@ -281,15 +283,18 @@ FAST_BRAIN_CHAIN = [
|
|
| 281 |
# demonstrably-healthy NIM models. With election (KI-080) picking the
|
| 282 |
# actually-fastest healthy candidate per turn, chain order matters
|
| 283 |
# only for cold-start; the elector handles steady-state.
|
|
|
|
|
|
|
| 284 |
# Primary: Qwen 3-Next 80B — 5/5 recent probes ok, ~2s, multilingual,
|
| 285 |
# verified `<FF>` adherence in production traffic.
|
| 286 |
"qwen/qwen3-next-80b-a3b-instruct",
|
| 287 |
-
# 1st fallback:
|
| 288 |
-
# (fastest healthy NIM model), different family for diversity.
|
| 289 |
-
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
|
| 290 |
-
# 2nd fallback: Mistral Large 3 675B — 3/3 recent ok, different family,
|
| 291 |
-
# keeps fact-find alive through a Qwen+Nemotron simultaneous outage.
|
| 292 |
"mistralai/mistral-large-3-675b-instruct-2512",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 293 |
]
|
| 294 |
|
| 295 |
# Judge chain — non-Qwen (different family from brain primary so the judge
|
|
|
|
| 250 |
# that have been "down" for 48+ consecutive probes (qwen3.5-122b,
|
| 251 |
# gpt-oss-120b, deepseek-v4-pro) so the election pool only contains
|
| 252 |
# demonstrably-healthy NIM models.
|
| 253 |
+
# KI-175 (2026-05-15) — Nemotron demoted to LAST RESORT per operator
|
| 254 |
+
# preference: nemotron is the weakest practical NIM model and should
|
| 255 |
+
# only serve when every higher-quality model is simultaneously down.
|
| 256 |
# Primary: Qwen 3-Next 80B — 5/5 recent probes ok, clean JSON, multilingual
|
| 257 |
"qwen/qwen3-next-80b-a3b-instruct",
|
| 258 |
+
# 1st fallback: Mistral Large 3 675B — recent 3/3 probes ok, different
|
| 259 |
+
# family (mistral). Strongest fallback.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 260 |
"mistralai/mistral-large-3-675b-instruct-2512",
|
| 261 |
+
# 2nd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different
|
| 262 |
+
# family (meta), keeps the chain alive through Qwen+Mistral outage.
|
| 263 |
"meta/llama-4-maverick-17b-128e-instruct",
|
| 264 |
+
# 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — barely usable for
|
| 265 |
+
# worst-case scenarios. Only reached when Qwen + Mistral + Maverick
|
| 266 |
+
# are ALL down simultaneously. Kept in the pool for fail-loud > down.
|
| 267 |
+
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
|
| 268 |
]
|
| 269 |
|
| 270 |
# Same chain for fast brain — Qwen 80B is already fast (~2s); no need for a
|
|
|
|
| 283 |
# demonstrably-healthy NIM models. With election (KI-080) picking the
|
| 284 |
# actually-fastest healthy candidate per turn, chain order matters
|
| 285 |
# only for cold-start; the elector handles steady-state.
|
| 286 |
+
# KI-175 (2026-05-15) — Nemotron demoted to LAST RESORT per operator
|
| 287 |
+
# preference: nemotron is the weakest practical NIM model.
|
| 288 |
# Primary: Qwen 3-Next 80B — 5/5 recent probes ok, ~2s, multilingual,
|
| 289 |
# verified `<FF>` adherence in production traffic.
|
| 290 |
"qwen/qwen3-next-80b-a3b-instruct",
|
| 291 |
+
# 1st fallback: Mistral Large 3 675B — 3/3 recent ok, different family.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 292 |
"mistralai/mistral-large-3-675b-instruct-2512",
|
| 293 |
+
# 2nd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different family.
|
| 294 |
+
"meta/llama-4-maverick-17b-128e-instruct",
|
| 295 |
+
# 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — barely usable.
|
| 296 |
+
# Only reached when Qwen + Mistral + Maverick are ALL down.
|
| 297 |
+
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
|
| 298 |
]
|
| 299 |
|
| 300 |
# Judge chain — non-Qwen (different family from brain primary so the judge
|