Fix 1 — Context builder bug (critical): truncation was dead code; re-render
now uses trimmed variables. Added get_raw_template() to prompt_loader and
wired template_str into build_context() via router. New test_context_builder.py.
Fix 2 — Reference tests for benchmark: added reference_test_code field with
8–12 held-out assert statements for all 8 tasks. validate_against_reference()
in run_benchmark.py runs final code against ground truth independently of the
QA agent's own tests. TaskResult gains reference_test_passed + elapsed_seconds;
BenchmarkSummary gains reference-validated success rate.
Fix 3 — Test suite expansion: 3 new schema validator tests (truncated JSON
salvage, multiple JSON objects, escaped quotes), 3 new sandbox tests (stack
overflow resource exhaustion, ImportError capture, empty inputs), new
test_router.py (retry-on-failure, raises after max retries). Fixed
test_graph_mock.py to reject "running" as a valid terminal status.
Fix 4 — Observability: LangSmith env detec…
Fix: move sync workflow to .github/workflows so GitHub Actions picks it up
sync_to_hf.yml was in the repo root — GitHub only reads workflow files from
.github/workflows/. Moving it here so the HF Spaces sync actually triggers
on every push to main.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
