Khanna, Videh Rakesh Rakesh Claude Sonnet 5.5 commited on
Commit
9274ccc
Β·
1 Parent(s): a688bd4

LLM chain: switch off Cerebras and HuggingFace router by default

Browse files

Both only return 402 for this account (Cerebras: card-gated trial credits gone;
HuggingFace: no inference credits left), so every call to them was wasted time
and a fail_streak. LLM_DISABLED_PROVIDERS (default cerebras,huggingface) removes
them from the provider chain; set it to an empty string to bring them back.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Files changed (3) hide show
  1. CLAUDE.md +1 -1
  2. llm_client.py +8 -1
  3. tests/test_llm_providers.py +30 -0
CLAUDE.md CHANGED
@@ -150,7 +150,7 @@ Inspired by TauricResearch/TradingAgents multi-agent debate pattern.
150
 
151
  **Provider model currency (fixed 2026-08-31):** LLM providers silently retire models, which breaks the chain with 404/410 and funnels every call to slow Ollama (the "AI keeps looping / no results" symptom). Verified-current models as of 2026-08-31: Groq dropped ALL Llama-3.x (`llama-3.3-70b-versatile` + `llama-3.1-8b-instant` now 404) β†’ use `openai/gpt-oss-120b`/`-20b` or `qwen/qwen3.8-27b`; NVIDIA `nvidia/llama-3.3-nemotron-super-49b-v1` hit EOL 2026-08-26 (410) and `meta/llama-3.1-8b-instruct` isn't served β†’ use `openai/gpt-oss-20b`; Cerebras now returns **402 payment_required** (free tier consumed) β€” benched via `_mark_daily_exhausted`. **gpt-oss are REASONING models**: without `reasoning_effort:"low"` they spend the whole `max_tokens` budget on hidden reasoning and return EMPTY `content` on the small-budget debate calls (fundamentals=200, bull/bear=320 tokens) β†’ `llm_client._payload()` injects `reasoning_effort:"low"` only for gpt-oss models. `llm_client._try_groq`/`_try_nvidia` also append a hard-coded known-good model so a stale HF Secret (e.g. `NVIDIA_MODEL`=dead) can't fully break the provider. Diagnose live provider health via `/api/provider-status?probe=1` on the Space.
152
 
153
- **Provider audit (2026-11-02):** every provider was listed and tested with an F&O-style JSON request. Free capacity is per account/model and shrinks fast: *Cerebras* 402 (free credits gone), *HuggingFace router* 402 (no credits), *OpenRouter* 429 `free-models-per-day` (50/day account-wide, so more `:free` models do not help; $10 credit unlocks 1,000/day), *Gemini* 429 per-model daily quota (and `gemini-2.5-flash`/`-lite` are 404 retired), *SambaNova* only `gemma-4-31B-it` answers (the rest 429 "high demand"; the original 401 was a bad key), *NVIDIA* mostly read-timeouts and several listed models 404 "not found for account" (`nemotron-3-super-120b-a12b`, `kimi-k2.6`, `nemotron-nano-3-30b-a3b`, `llama-3.1-nemotron-70b`). *Groq* works but has strict per-minute token limits. Verified-working then: Groq (gpt-oss-120b/20b, qwen3.8-27b), SambaNova gemma-4-31B-it, Gemini gemini-3.1-flash-lite. Defaults now put those first; `gemini-3.5-flash`/`-3.6-flash` and Gemini-hosted gemma return no JSON at the app's token budget, and `nvidia/nemotron-3.5-lightning` returned empty content. Model env vars in HF Secrets override code defaults, so update both. A 1.5B Ollama model is too slow (~35s) and breaks the F&O numeric rules, so it is not a F&O fallback.
154
 
155
  **AI unavailable root cause** β€” "⚠ AI unavailable" means ALL cloud providers failed AND Ollama failed/unavailable. Most common cause: free-tier rate limits exhausted during a 150-stock batch scan (~150–450 LLM calls). Fix: ensure OpenRouter + Groq + Cerebras + HF keys are all set in HF Spaces Secrets. The "Signals active: 0" part of the error is a separate issue β€” the stock has no technical strategy signals firing, independent of AI.
156
 
 
150
 
151
  **Provider model currency (fixed 2026-08-31):** LLM providers silently retire models, which breaks the chain with 404/410 and funnels every call to slow Ollama (the "AI keeps looping / no results" symptom). Verified-current models as of 2026-08-31: Groq dropped ALL Llama-3.x (`llama-3.3-70b-versatile` + `llama-3.1-8b-instant` now 404) β†’ use `openai/gpt-oss-120b`/`-20b` or `qwen/qwen3.8-27b`; NVIDIA `nvidia/llama-3.3-nemotron-super-49b-v1` hit EOL 2026-08-26 (410) and `meta/llama-3.1-8b-instruct` isn't served β†’ use `openai/gpt-oss-20b`; Cerebras now returns **402 payment_required** (free tier consumed) β€” benched via `_mark_daily_exhausted`. **gpt-oss are REASONING models**: without `reasoning_effort:"low"` they spend the whole `max_tokens` budget on hidden reasoning and return EMPTY `content` on the small-budget debate calls (fundamentals=200, bull/bear=320 tokens) β†’ `llm_client._payload()` injects `reasoning_effort:"low"` only for gpt-oss models. `llm_client._try_groq`/`_try_nvidia` also append a hard-coded known-good model so a stale HF Secret (e.g. `NVIDIA_MODEL`=dead) can't fully break the provider. Diagnose live provider health via `/api/provider-status?probe=1` on the Space.
152
 
153
+ **Provider audit (2026-11-02):** every provider was listed and tested with an F&O-style JSON request. Free capacity is per account/model and shrinks fast: *Cerebras* 402 (free credits gone), *HuggingFace router* 402 (no credits), *OpenRouter* 429 `free-models-per-day` (50/day account-wide, so more `:free` models do not help; $10 credit unlocks 1,000/day), *Gemini* 429 per-model daily quota (and `gemini-2.5-flash`/`-lite` are 404 retired), *SambaNova* only `gemma-4-31B-it` answers (the rest 429 "high demand"; the original 401 was a bad key), *NVIDIA* mostly read-timeouts and several listed models 404 "not found for account" (`nemotron-3-super-120b-a12b`, `kimi-k2.6`, `nemotron-nano-3-30b-a3b`, `llama-3.1-nemotron-70b`). *Groq* works but has strict per-minute token limits. Verified-working then: Groq (gpt-oss-120b/20b, qwen3.8-27b), SambaNova gemma-4-31B-it, Gemini gemini-3.1-flash-lite. Defaults now put those first; `gemini-3.5-flash`/`-3.6-flash` and Gemini-hosted gemma return no JSON at the app's token budget, and `nvidia/nemotron-3.5-lightning` returned empty content. Model env vars in HF Secrets override code defaults, so update both. A 1.5B Ollama model is too slow (~35s) and breaks the F&O numeric rules, so it is not a F&O fallback. **Cerebras and the HuggingFace router are switched off by default** (`LLM_DISABLED_PROVIDERS`, default `cerebras,huggingface`; set it to an empty string to re-enable after adding billing/credits): both only return 402, so every call to them was wasted time. The active chain is Groq β†’ NVIDIA β†’ Gemini β†’ OpenRouter β†’ SambaNova β†’ Ollama.
154
 
155
  **AI unavailable root cause** β€” "⚠ AI unavailable" means ALL cloud providers failed AND Ollama failed/unavailable. Most common cause: free-tier rate limits exhausted during a 150-stock batch scan (~150–450 LLM calls). Fix: ensure OpenRouter + Groq + Cerebras + HF keys are all set in HF Spaces Secrets. The "Signals active: 0" part of the error is a separate issue β€” the stock has no technical strategy signals firing, independent of AI.
156
 
llm_client.py CHANGED
@@ -120,7 +120,14 @@ _OLLAMA_WARMUP_BACKOFF_SECS: int = 45
120
  _OLLAMA_CHAT_TIMEOUT: int = 70 # 70s: warmup confirms model is loaded, so 70s is enough for 512 tokens
121
 
122
  # Canonical cloud provider order (original preference before runtime reordering)
123
- _CLOUD_PROVIDERS = ["groq", "nvidia", "gemini", "openrouter", "sambanova", "cerebras", "huggingface"]
 
 
 
 
 
 
 
124
  _PROVIDER_ORDER = _CLOUD_PROVIDERS
125
 
126
 
 
120
  _OLLAMA_CHAT_TIMEOUT: int = 70 # 70s: warmup confirms model is loaded, so 70s is enough for 512 tokens
121
 
122
  # Canonical cloud provider order (original preference before runtime reordering)
123
+ # Providers switched off because their free tier no longer works for this account (verified 2026-11-02):
124
+ # Cerebras answers 402 payment_required (card-gated $5 trial) and the HuggingFace router answers 402 with no
125
+ # credits left. Every call to them was wasted time and a fail_streak. Set LLM_DISABLED_PROVIDERS="" (or a
126
+ # different comma list) in the environment to bring them back, e.g. after adding billing.
127
+ _DISABLED_PROVIDERS = {name.strip().lower() for name in os.environ.get("LLM_DISABLED_PROVIDERS", "cerebras,huggingface").split(",")
128
+ if name.strip()}
129
+ _CLOUD_PROVIDERS = [name for name in ("groq", "nvidia", "gemini", "openrouter", "sambanova", "cerebras", "huggingface")
130
+ if name not in _DISABLED_PROVIDERS]
131
  _PROVIDER_ORDER = _CLOUD_PROVIDERS
132
 
133
 
tests/test_llm_providers.py ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import os
3
+ import subprocess
4
+ import sys
5
+
6
+ ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
7
+
8
+
9
+ def _chain(disabled=None):
10
+ env = {key: value for key, value in os.environ.items() if key != "LLM_DISABLED_PROVIDERS"}
11
+ if disabled is not None:
12
+ env["LLM_DISABLED_PROVIDERS"] = disabled
13
+ out = subprocess.run([sys.executable, "-c", "import json, llm_client; print(json.dumps(llm_client._CLOUD_PROVIDERS))"],
14
+ cwd=ROOT, env=env, capture_output=True, text=True, check=True).stdout
15
+ return json.loads(out)
16
+
17
+
18
+ def test_providers_with_dead_free_tiers_are_off_by_default():
19
+ chain = _chain()
20
+ assert "cerebras" not in chain and "huggingface" not in chain
21
+ assert {"groq", "nvidia", "gemini", "openrouter", "sambanova"} <= set(chain)
22
+
23
+
24
+ def test_an_empty_setting_brings_every_provider_back():
25
+ assert {"cerebras", "huggingface"} <= set(_chain(""))
26
+
27
+
28
+ def test_the_disabled_list_can_be_customised():
29
+ chain = _chain("nvidia")
30
+ assert "nvidia" not in chain and {"cerebras", "huggingface"} <= set(chain)