rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
dc57a46
·
1 Parent(s): 94aeb46

fix(llm): KI-175 — demote nemotron to last resort in all chains

Browse files

User verdict on nemotron: "the worst model, barely usable, last resort."

Live admin showed nemotron elected as primary for Brain Fast + Brain Main
+ Judge — because the chain election was picking the first HEALTHY model
in the chain, and nemotron sat at position #2 (BRAIN/FAST_BRAIN) or #3
(JUDGE). Whenever qwen had a transient blip the election fell to nemotron
and the sticky-primary held it there.

Fix: reorder chains so nemotron is always LAST. It now only serves when
every other chain member is simultaneously unavailable — fail-loud is
better than serving the worst model by default.

BRAIN_CHAIN:
qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron
FAST_BRAIN_CHAIN:
qwen3-next-80b → mistral-large-3 → llama-4-maverick → nemotron
JUDGE_CHAIN:
mistral-large-3 → llama-4-maverick → nemotron
(judge order was already correct; chain unchanged)

Llama-4 Maverick added as a 3rd-fallback to FAST_BRAIN_CHAIN (previously
only in BRAIN+JUDGE). All chains now have 3 healthy non-nemotron options
before nemotron is reached.

Verification: imports + chain ordering confirmed via `python -c` smoke.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. backend/providers/nvidia_nim_llm.py +19 -14
backend/providers/nvidia_nim_llm.py CHANGED
@@ -250,19 +250,21 @@ BRAIN_CHAIN = [
250
  # that have been "down" for 48+ consecutive probes (qwen3.5-122b,
251
  # gpt-oss-120b, deepseek-v4-pro) so the election pool only contains
252
  # demonstrably-healthy NIM models.
 
 
 
253
  # Primary: Qwen 3-Next 80B — 5/5 recent probes ok, clean JSON, multilingual
254
  "qwen/qwen3-next-80b-a3b-instruct",
255
- # 1st fallback: NVIDIA Nemotron-Super 49B — recent 3/3 probes ok, ~354ms
256
- # latency, different family (nvidia) from Qwen primary → preserves
257
- # cross-family-grading invariant if elevated to judge.
258
- "nvidia/llama-3.3-nemotron-super-49b-v1.5",
259
- # 2nd fallback: Mistral Large 3 675B — recent 3/3 probes ok, different
260
- # family (mistral). Also the judge primary; only reached when both Qwen +
261
- # Nemotron are unavailable.
262
  "mistralai/mistral-large-3-675b-instruct-2512",
263
- # 3rd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different
264
- # family (meta), keeps the chain alive through a single-family outage.
265
  "meta/llama-4-maverick-17b-128e-instruct",
 
 
 
 
266
  ]
267
 
268
  # Same chain for fast brain — Qwen 80B is already fast (~2s); no need for a
@@ -281,15 +283,18 @@ FAST_BRAIN_CHAIN = [
281
  # demonstrably-healthy NIM models. With election (KI-080) picking the
282
  # actually-fastest healthy candidate per turn, chain order matters
283
  # only for cold-start; the elector handles steady-state.
 
 
284
  # Primary: Qwen 3-Next 80B — 5/5 recent probes ok, ~2s, multilingual,
285
  # verified `<FF>` adherence in production traffic.
286
  "qwen/qwen3-next-80b-a3b-instruct",
287
- # 1st fallback: NVIDIA Nemotron-Super 49B — 3/3 recent ok, ~354ms
288
- # (fastest healthy NIM model), different family for diversity.
289
- "nvidia/llama-3.3-nemotron-super-49b-v1.5",
290
- # 2nd fallback: Mistral Large 3 675B — 3/3 recent ok, different family,
291
- # keeps fact-find alive through a Qwen+Nemotron simultaneous outage.
292
  "mistralai/mistral-large-3-675b-instruct-2512",
 
 
 
 
 
293
  ]
294
 
295
  # Judge chain — non-Qwen (different family from brain primary so the judge
 
250
  # that have been "down" for 48+ consecutive probes (qwen3.5-122b,
251
  # gpt-oss-120b, deepseek-v4-pro) so the election pool only contains
252
  # demonstrably-healthy NIM models.
253
+ # KI-175 (2026-05-15) — Nemotron demoted to LAST RESORT per operator
254
+ # preference: nemotron is the weakest practical NIM model and should
255
+ # only serve when every higher-quality model is simultaneously down.
256
  # Primary: Qwen 3-Next 80B — 5/5 recent probes ok, clean JSON, multilingual
257
  "qwen/qwen3-next-80b-a3b-instruct",
258
+ # 1st fallback: Mistral Large 3 675B — recent 3/3 probes ok, different
259
+ # family (mistral). Strongest fallback.
 
 
 
 
 
260
  "mistralai/mistral-large-3-675b-instruct-2512",
261
+ # 2nd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different
262
+ # family (meta), keeps the chain alive through Qwen+Mistral outage.
263
  "meta/llama-4-maverick-17b-128e-instruct",
264
+ # 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — barely usable for
265
+ # worst-case scenarios. Only reached when Qwen + Mistral + Maverick
266
+ # are ALL down simultaneously. Kept in the pool for fail-loud > down.
267
+ "nvidia/llama-3.3-nemotron-super-49b-v1.5",
268
  ]
269
 
270
  # Same chain for fast brain — Qwen 80B is already fast (~2s); no need for a
 
283
  # demonstrably-healthy NIM models. With election (KI-080) picking the
284
  # actually-fastest healthy candidate per turn, chain order matters
285
  # only for cold-start; the elector handles steady-state.
286
+ # KI-175 (2026-05-15) — Nemotron demoted to LAST RESORT per operator
287
+ # preference: nemotron is the weakest practical NIM model.
288
  # Primary: Qwen 3-Next 80B — 5/5 recent probes ok, ~2s, multilingual,
289
  # verified `<FF>` adherence in production traffic.
290
  "qwen/qwen3-next-80b-a3b-instruct",
291
+ # 1st fallback: Mistral Large 3 675B — 3/3 recent ok, different family.
 
 
 
 
292
  "mistralai/mistral-large-3-675b-instruct-2512",
293
+ # 2nd fallback: Meta Llama-4 Maverick 17B — 5/5 probes ok, different family.
294
+ "meta/llama-4-maverick-17b-128e-instruct",
295
+ # 3rd / LAST RESORT: NVIDIA Nemotron-Super 49B — barely usable.
296
+ # Only reached when Qwen + Mistral + Maverick are ALL down.
297
+ "nvidia/llama-3.3-nemotron-super-49b-v1.5",
298
  ]
299
 
300
  # Judge chain — non-Qwen (different family from brain primary so the judge