Spaces:
Running
Running
Khanna, Videh Rakesh Rakesh Claude Sonnet 5.5 commited on
Commit Β·
9274ccc
1
Parent(s): a688bd4
LLM chain: switch off Cerebras and HuggingFace router by default
Browse filesBoth only return 402 for this account (Cerebras: card-gated trial credits gone;
HuggingFace: no inference credits left), so every call to them was wasted time
and a fail_streak. LLM_DISABLED_PROVIDERS (default cerebras,huggingface) removes
them from the provider chain; set it to an empty string to bring them back.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- CLAUDE.md +1 -1
- llm_client.py +8 -1
- tests/test_llm_providers.py +30 -0
CLAUDE.md
CHANGED
|
@@ -150,7 +150,7 @@ Inspired by TauricResearch/TradingAgents multi-agent debate pattern.
|
|
| 150 |
|
| 151 |
**Provider model currency (fixed 2026-08-31):** LLM providers silently retire models, which breaks the chain with 404/410 and funnels every call to slow Ollama (the "AI keeps looping / no results" symptom). Verified-current models as of 2026-08-31: Groq dropped ALL Llama-3.x (`llama-3.3-70b-versatile` + `llama-3.1-8b-instant` now 404) β use `openai/gpt-oss-120b`/`-20b` or `qwen/qwen3.8-27b`; NVIDIA `nvidia/llama-3.3-nemotron-super-49b-v1` hit EOL 2026-08-26 (410) and `meta/llama-3.1-8b-instruct` isn't served β use `openai/gpt-oss-20b`; Cerebras now returns **402 payment_required** (free tier consumed) β benched via `_mark_daily_exhausted`. **gpt-oss are REASONING models**: without `reasoning_effort:"low"` they spend the whole `max_tokens` budget on hidden reasoning and return EMPTY `content` on the small-budget debate calls (fundamentals=200, bull/bear=320 tokens) β `llm_client._payload()` injects `reasoning_effort:"low"` only for gpt-oss models. `llm_client._try_groq`/`_try_nvidia` also append a hard-coded known-good model so a stale HF Secret (e.g. `NVIDIA_MODEL`=dead) can't fully break the provider. Diagnose live provider health via `/api/provider-status?probe=1` on the Space.
|
| 152 |
|
| 153 |
-
**Provider audit (2026-11-02):** every provider was listed and tested with an F&O-style JSON request. Free capacity is per account/model and shrinks fast: *Cerebras* 402 (free credits gone), *HuggingFace router* 402 (no credits), *OpenRouter* 429 `free-models-per-day` (50/day account-wide, so more `:free` models do not help; $10 credit unlocks 1,000/day), *Gemini* 429 per-model daily quota (and `gemini-2.5-flash`/`-lite` are 404 retired), *SambaNova* only `gemma-4-31B-it` answers (the rest 429 "high demand"; the original 401 was a bad key), *NVIDIA* mostly read-timeouts and several listed models 404 "not found for account" (`nemotron-3-super-120b-a12b`, `kimi-k2.6`, `nemotron-nano-3-30b-a3b`, `llama-3.1-nemotron-70b`). *Groq* works but has strict per-minute token limits. Verified-working then: Groq (gpt-oss-120b/20b, qwen3.8-27b), SambaNova gemma-4-31B-it, Gemini gemini-3.1-flash-lite. Defaults now put those first; `gemini-3.5-flash`/`-3.6-flash` and Gemini-hosted gemma return no JSON at the app's token budget, and `nvidia/nemotron-3.5-lightning` returned empty content. Model env vars in HF Secrets override code defaults, so update both. A 1.5B Ollama model is too slow (~35s) and breaks the F&O numeric rules, so it is not a F&O fallback.
|
| 154 |
|
| 155 |
**AI unavailable root cause** β "β AI unavailable" means ALL cloud providers failed AND Ollama failed/unavailable. Most common cause: free-tier rate limits exhausted during a 150-stock batch scan (~150β450 LLM calls). Fix: ensure OpenRouter + Groq + Cerebras + HF keys are all set in HF Spaces Secrets. The "Signals active: 0" part of the error is a separate issue β the stock has no technical strategy signals firing, independent of AI.
|
| 156 |
|
|
|
|
| 150 |
|
| 151 |
**Provider model currency (fixed 2026-08-31):** LLM providers silently retire models, which breaks the chain with 404/410 and funnels every call to slow Ollama (the "AI keeps looping / no results" symptom). Verified-current models as of 2026-08-31: Groq dropped ALL Llama-3.x (`llama-3.3-70b-versatile` + `llama-3.1-8b-instant` now 404) β use `openai/gpt-oss-120b`/`-20b` or `qwen/qwen3.8-27b`; NVIDIA `nvidia/llama-3.3-nemotron-super-49b-v1` hit EOL 2026-08-26 (410) and `meta/llama-3.1-8b-instruct` isn't served β use `openai/gpt-oss-20b`; Cerebras now returns **402 payment_required** (free tier consumed) β benched via `_mark_daily_exhausted`. **gpt-oss are REASONING models**: without `reasoning_effort:"low"` they spend the whole `max_tokens` budget on hidden reasoning and return EMPTY `content` on the small-budget debate calls (fundamentals=200, bull/bear=320 tokens) β `llm_client._payload()` injects `reasoning_effort:"low"` only for gpt-oss models. `llm_client._try_groq`/`_try_nvidia` also append a hard-coded known-good model so a stale HF Secret (e.g. `NVIDIA_MODEL`=dead) can't fully break the provider. Diagnose live provider health via `/api/provider-status?probe=1` on the Space.
|
| 152 |
|
| 153 |
+
**Provider audit (2026-11-02):** every provider was listed and tested with an F&O-style JSON request. Free capacity is per account/model and shrinks fast: *Cerebras* 402 (free credits gone), *HuggingFace router* 402 (no credits), *OpenRouter* 429 `free-models-per-day` (50/day account-wide, so more `:free` models do not help; $10 credit unlocks 1,000/day), *Gemini* 429 per-model daily quota (and `gemini-2.5-flash`/`-lite` are 404 retired), *SambaNova* only `gemma-4-31B-it` answers (the rest 429 "high demand"; the original 401 was a bad key), *NVIDIA* mostly read-timeouts and several listed models 404 "not found for account" (`nemotron-3-super-120b-a12b`, `kimi-k2.6`, `nemotron-nano-3-30b-a3b`, `llama-3.1-nemotron-70b`). *Groq* works but has strict per-minute token limits. Verified-working then: Groq (gpt-oss-120b/20b, qwen3.8-27b), SambaNova gemma-4-31B-it, Gemini gemini-3.1-flash-lite. Defaults now put those first; `gemini-3.5-flash`/`-3.6-flash` and Gemini-hosted gemma return no JSON at the app's token budget, and `nvidia/nemotron-3.5-lightning` returned empty content. Model env vars in HF Secrets override code defaults, so update both. A 1.5B Ollama model is too slow (~35s) and breaks the F&O numeric rules, so it is not a F&O fallback. **Cerebras and the HuggingFace router are switched off by default** (`LLM_DISABLED_PROVIDERS`, default `cerebras,huggingface`; set it to an empty string to re-enable after adding billing/credits): both only return 402, so every call to them was wasted time. The active chain is Groq β NVIDIA β Gemini β OpenRouter β SambaNova β Ollama.
|
| 154 |
|
| 155 |
**AI unavailable root cause** β "β AI unavailable" means ALL cloud providers failed AND Ollama failed/unavailable. Most common cause: free-tier rate limits exhausted during a 150-stock batch scan (~150β450 LLM calls). Fix: ensure OpenRouter + Groq + Cerebras + HF keys are all set in HF Spaces Secrets. The "Signals active: 0" part of the error is a separate issue β the stock has no technical strategy signals firing, independent of AI.
|
| 156 |
|
llm_client.py
CHANGED
|
@@ -120,7 +120,14 @@ _OLLAMA_WARMUP_BACKOFF_SECS: int = 45
|
|
| 120 |
_OLLAMA_CHAT_TIMEOUT: int = 70 # 70s: warmup confirms model is loaded, so 70s is enough for 512 tokens
|
| 121 |
|
| 122 |
# Canonical cloud provider order (original preference before runtime reordering)
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
_PROVIDER_ORDER = _CLOUD_PROVIDERS
|
| 125 |
|
| 126 |
|
|
|
|
| 120 |
_OLLAMA_CHAT_TIMEOUT: int = 70 # 70s: warmup confirms model is loaded, so 70s is enough for 512 tokens
|
| 121 |
|
| 122 |
# Canonical cloud provider order (original preference before runtime reordering)
|
| 123 |
+
# Providers switched off because their free tier no longer works for this account (verified 2026-11-02):
|
| 124 |
+
# Cerebras answers 402 payment_required (card-gated $5 trial) and the HuggingFace router answers 402 with no
|
| 125 |
+
# credits left. Every call to them was wasted time and a fail_streak. Set LLM_DISABLED_PROVIDERS="" (or a
|
| 126 |
+
# different comma list) in the environment to bring them back, e.g. after adding billing.
|
| 127 |
+
_DISABLED_PROVIDERS = {name.strip().lower() for name in os.environ.get("LLM_DISABLED_PROVIDERS", "cerebras,huggingface").split(",")
|
| 128 |
+
if name.strip()}
|
| 129 |
+
_CLOUD_PROVIDERS = [name for name in ("groq", "nvidia", "gemini", "openrouter", "sambanova", "cerebras", "huggingface")
|
| 130 |
+
if name not in _DISABLED_PROVIDERS]
|
| 131 |
_PROVIDER_ORDER = _CLOUD_PROVIDERS
|
| 132 |
|
| 133 |
|
tests/test_llm_providers.py
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import os
|
| 3 |
+
import subprocess
|
| 4 |
+
import sys
|
| 5 |
+
|
| 6 |
+
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def _chain(disabled=None):
|
| 10 |
+
env = {key: value for key, value in os.environ.items() if key != "LLM_DISABLED_PROVIDERS"}
|
| 11 |
+
if disabled is not None:
|
| 12 |
+
env["LLM_DISABLED_PROVIDERS"] = disabled
|
| 13 |
+
out = subprocess.run([sys.executable, "-c", "import json, llm_client; print(json.dumps(llm_client._CLOUD_PROVIDERS))"],
|
| 14 |
+
cwd=ROOT, env=env, capture_output=True, text=True, check=True).stdout
|
| 15 |
+
return json.loads(out)
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def test_providers_with_dead_free_tiers_are_off_by_default():
|
| 19 |
+
chain = _chain()
|
| 20 |
+
assert "cerebras" not in chain and "huggingface" not in chain
|
| 21 |
+
assert {"groq", "nvidia", "gemini", "openrouter", "sambanova"} <= set(chain)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def test_an_empty_setting_brings_every_provider_back():
|
| 25 |
+
assert {"cerebras", "huggingface"} <= set(_chain(""))
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def test_the_disabled_list_can_be_customised():
|
| 29 |
+
chain = _chain("nvidia")
|
| 30 |
+
assert "nvidia" not in chain and {"cerebras", "huggingface"} <= set(chain)
|