Spaces:
Sleeping
Sleeping
Commit History
-
feat: Implement wait_for_updates action for handling delayed cases and evidence
- Added support for the "wait_for_updates" action in baseline_runner and benchmark_runner to manage scenarios with no open cases. - Enhanced the ChargebackOpsEnvironment to handle pending issuer reviews and delayed evidence, allowing agents to wait for updates when necessary. - Introduced a new marathon task scenario with wave arrivals, simulating long-horizon backlog management. - Updated tests to validate the behavior of the new wait_for_updates action and its integration with delayed evidence and issuer reviews.2dedffd
-
feat: tighten EscalationROI, add ambiguous medium case, LLM note judge wrapper
- Add pre_arb_recovery_medium headline case to raise round-2 fire rate - Tighten EscalationROIRubric concede penalty on positive-EV contestable cases - Add disjoint required/helpful evidence in headline + generator templates - Drop dispute_complexity multiplier - Heuristic switches to EV-based escalation (P(win)*amount vs $250 fee) - New LLMNoteJudgeRubric (opt-in via USE_LLM_NOTE_JUDGE) with provider fallback - Pin gemini-2.5-flash across llm_softening + baseline docs - Refresh demo_ui with multi-round panels (issuer rationale, arb ruling, P&L) - Update README architecture, AGENT.md scoring tables, RESULTS.md numbers - 86/86 tests green; headline heuristic 0.8254, escalate_all 0.7713e32a33b -
Add training notebook and benchmark runner for ChargebackOps
- Introduced `train_merchant_agent.ipynb` for end-to-end GRPO training of the merchant-side chargeback agent, including environment setup, model loading, training prompt dataset creation, and evaluation. - Created `benchmark_runner.py` to implement scripted policies for benchmarking against the trained agent, including heuristic, escalate_all, concede_all, and naive policies. - Updated `issuer_model.py` to refine rationale messages for issuer decisions. - Added unit tests for the benchmark runner and training adapter to ensure policy validity and reward computation accuracy. - Implemented training helpers in `training/__init__.py`, `env_adapter.py`, and `reward_adapter.py` to facilitate interaction with the GRPO trainer.bd00c06
-
refactor: tighten rubric discrimination + LLM path + add running doc
- Tighten acceptable_strategies on contest-optimal templates (fraud_signal_ambiguity, _PRODUCT_NOT_AS_DESCRIBED, _SERVICE_NOT_PROVIDED) so conceding a winnable case no longer earns partial strategy credit. Bad-policy floor 0.212 -> 0.199. - Expand _obvious_next_action to cover single-candidate states, remove_evidence, and all set_strategy picks. Starves the LLM of bad-call opportunities on deterministic workflow states; provider calls drop 19 -> 7 per 10-task run. - Improve LLM prompt with reason-code -> optimal-strategy mapping and explicit "candidate 0 is usually correct" guidance. - LLM baseline: 0.622 -> 0.729 (+0.107). fraud_signal_ambiguity recovered 0.408 -> 0.968, generated_nightmare_s31 0.134 -> 0.486. LLM now edges heuristic (0.729 > 0.724). - Add 28-task multi-seed stress grid to RESULTS.md (7 seeds x 4 difficulties). - Add docs/RUNNING_THE_AGENT.md with end-to-end run instructions. - Fix README Docker block: offline run no longer requires --env-file; section ref…0054f7f
-
Refactor evidence building and improve code readability in iso_adapter.py
- Enhanced formatting and readability of evidence creation in _build_evidence function. - Consolidated imports in iso_adapter.py and simulation.py for clarity. - Improved policy requirements and guidance formatting in simulation.py. - Updated task generation logic in server/app.py and chargeback_ops_environment.py for consistency. - Refined demo UI in server/demo_ui.py to include policy selection and improved layout.37bfd28
-
fix: match structured output to hackathon spec format
[START] now includes env= and model= fields. [STEP] now includes action=, done=, error= with 2dp reward. [END] now includes success=, rewards= instead of task=. API key reads HF_TOKEN or API_KEY with HF router default.0f980e9 -
fix: add [START]/[STEP]/[END] structured output to inference.py
Phase 2 validator requires structured markers on stdout. Each task prints [START] task=X, [STEP] step=N reward=R, and [END] task=X score=S steps=N with flush=True.388e3b8
-
feat: add OpenEnv overview documentation and enhance harmful evidence detection keywords
64cb3ce -
feat: harden grading, expand task catalog, add episode persistence
- Add root inference.py re-export (submission contract requires it) - Expand list_tasks() from 3 to 8 tasks (3 built-in + 5 generated) - Tighten grading: acceptable fallback 0.55->0.35, abandoned contest evidence 0.3->0.15, outcome fallback 0.6->0.4 - Add JSONL file persistence to episode_store - Add /results endpoint for episode history inspection - Remove circular grader-gaming comments from baseline_runner - Update README project layout and commands for new structure - Update AGENT.md file map for new directory paths87c40c2
-
refactor: reorganize source files into core/, evaluation/, runners/, scenarios/ directories
Moved loose root-level Python modules into logical subdirectories: - core/ — models, client, episode_store - evaluation/ — grading, agent_brutal_audit - runners/ — baseline_runner, inference - scenarios/ — simulation, case_generator, iso_adapter3816847