Spaces:
Running
Running
Commit History
-
Migrate to gpt-oss-120b, strict JSON schema, per-IP rate limiting
- Replace deprecated llama-3.3-70b-versatile with openai/gpt-oss-120b - Switch evaluation response_format from json_object to json_schema strict mode, eliminating the ~17% EvaluationParseError rate at the generation level; schema built dynamically from DEFAULT_ATTRIBUTES - Overhaul retry logic: one defensive transport retry (was 5x parse retry); EvaluationParseError propagates immediately - Add CapacityError for Groq 429 (daily limit hit), surfaced as a friendly toast instead of a raw error - Add per-IP daily session cap (default 10, env-var overridable) via gr.Request in app.py; in-memory counter resets daily - Document Groq project-level limits (10k RPD / 3M TPD) in inference.py - Update test suite: 42 tests, all passing; new schema shape and 429 tests added - Sync ROADMAP.md (Milestone 6), CLAUDE.md, README.md, ARCHITECTURE.md - Add PRISMA_UPDATE.md, GROQ_MIGRATION.md, SCHEMA_RETRY.md to .gitignore Co-authored-by: Cursor <cursoragent@cursor.com>47329ab
-
Increase parse-retry budget from 1 to 4 (5 total attempts)
Live testing showed a higher second-attempt failure rate than the original brief assumed (3 of 7 first-failures still surfaced a toast on the second attempt). Bumping max_attempts to 5 brings the expected user-visible error rate well into single-digit-percent territory at the cost of bounded extra latency on the failure path. Implementation pulls the count out to DEFAULT_MAX_ATTEMPTS in src/config.py so it lives with the other tunables, and parameterizes the constructor so tests can vary it without monkeypatching. Log strings now include the attempt number out of the budget so the production logs stay informative as we tune the value. InferenceError is still not retried. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>e1f5302
-
Retry once on EvaluationParseError to mask stochastic JSON drift
Llama 3.3 70B under json_object mode occasionally emits syntactically valid JSON that omits or mis-types the response field — observed roughly 17% in early-session testing on Groq (same pattern existed pre-migration on HF, so it's model behavior, not provider). With temperature 0.7 the failure is stochastic, so resampling at the same temperature usually parses cleanly — which matches the user-visible "press Send again, it works" pattern. generate() now retries once on EvaluationParseError. No backoff: the failure is sample-level, not transient API state. Logs both the first-attempt failure and the second-attempt outcome so the residual rate stays trackable. InferenceError is explicitly NOT retried; that needs proper rate-limit-aware backoff and is deferred to a separate session. The intended primary fix was strict json_schema mode, but Groq does not currently support it on llama-3.3-70b-versatile (only on openai/gpt-oss-* and meta-llama/llama-4-scout-* families). That choice and the …082c6bb
-
Migrate inference backend from Hugging Face to Groq
HF free-tier credits were depleted, returning 402 on every call. Groq hosts the same Llama 3.3 70B Instruct model, has no depleting credit pool (only short-term per-minute rate limits), and runs noticeably faster (~250-400 vs ~30-50 tok/sec), which keeps the running-evaluation paradigm feeling responsive. The Space stays on HF Spaces; only the model call moves. PrismaInferenceClient keeps its public surface, exception types, and JSON response_format enforcement; mocks in tests/test_inference updated to the groq SDK's chat.completions.create shape. Env var is now GROQ_API_KEY (both .env and HF Space Secrets paths called out in the startup error message). README, ARCHITECTURE, and .env.example updated accordingly. Also bundled two small Gradio 6.0 hygiene fixes seen in production logs: gr.Chatbot type="messages" is now explicit, and theme/css moved from gr.Blocks to demo.launch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>03732db
-
Add inference module with HF API wrapper
- src/inference.py: PrismaInferenceClient wraps huggingface_hub's InferenceClient with forced JSON mode (required for Llama 3.3 70B) - Typed errors: InferenceError for API issues, EvaluationParseError bubbles up from src.evaluation for parse issues - src/config.py: MODEL_ID, DEFAULT_TEMPERATURE, DEFAULT_MAX_TOKENS - tests/test_inference.py: 10 tests with mocked InferenceClient, no real API calls7db5adc -
Add evaluation module with parsing, validation, and formatting
- src/evaluation.py: parse_model_output, format_score, format_evaluation * Intensifier scale (1=not at all, 7=extremely) * Strict validation: missing fields, out-of-range scores, non-int types * Tolerates markdown code fences around JSON - src/config.py: add MIN_SCORE, MAX_SCORE, SESSION_TURN_CAP constants - tests/test_evaluation.py: 16 tests covering happy paths and error cases989cfe7
-
Initial project scaffolding
1e436e0