Spaces:
Sleeping
Sleeping
Commit History
-
Finalise OpenEnv submission: Clamp rewards to 0.0-1.0, update Gradio UI, and add Groq pipelines
Soham Banerjee192db9d
-
3 fixes: README accurate, is_adversarial in state, reset(campaign_id) (100/100)
Soham BanerjeeFix 1 — README: all stale numbers updated 128 scenarios (52 easy / 25 medium / 51 hard) Baseline table: easy=0.375, medium=0.460, hard=0.144, overall=0.300 Penalty table: graduated severity (sev-5→-0.30, sev-4→-0.15, sev-3→-0.05) Bonuses table: campaign +0.15, appeal ±0.10 Observation space documented with campaign + is_adversarial fields Episode modes: Queue + Campaign + Single-step + Appeal examples Project structure: 128 scenarios, 100-check validator, appeal in env Fix 2 — is_adversarial in state obs (_build_state_obs) state['is_adversarial'] = True|False on every reset/state call Agents can condition on it BEFORE calling step(), before the appeal Fix 3 — reset(campaign_id='camp_crypto_001') Deterministically loads all posts for a campaign in post_index order Raises ValueError on unknown campaign_id Raises ValueError if both scenario_id and campaign_id provided Validator: 92 → 100 checks (8 new)10c3c6e
-
Appeal mechanic: is_adversarial + env.appeal() 2-turn flow (92/92 checks)
Soham BanerjeeArchitecture: - is_adversarial: bool on scenario — flags posts with appeal challenges - appeal_text: user's counter-argument (social engineering attempt or valid context) - appeal_verdict: 'uphold' | 'overturn' — what correct response is - env.appeal({verdict, rationale}) → {appeal_reward, verdict_correct, gt_verdict, ...} +0.10 correct verdict (resist manipulation OR accept valid context) -0.10 wrong verdict (fooled by appeal OR stubbornly upholding wrong decision) - _pending_appeal_scenario / _appeal_used state prevents double-calls - step() exposes appeal_available / appeal_text / appeal_verdict_gt in info - is_adversarial field present in info on every step 10 adversarial scenarios: 6 × uphold — manipulative appeals: educational framing, authority claim, fiction claim post-hoc, free-speech argument, research framing, stalking 'it's my novel' claim 4 × overturn — valid context: pest control misread, Holocaust education, anti-h…d741d4b -
Fill easy GT gaps: full label×action coverage (79/79 checks)
Soham Banerjee11 new easy scenarios plugging 4 missing/thin combos: misleading/shadowban 0 → 3 (health disinfo suppression) toxic/shadowban 0 → 3 (chronic low-grade toxicity) toxic/warn 1 → 4 (first-offence mild insults) misleading/escalate 1 → 3 (repeat high-harm misinformation) Easy tier: 37 → 48 scenarios | Overall: 107 → 118 scenarios Validator: 66 → 79 checks + all 4 easy labels each ≥ 2 examples + all 5 easy actions each ≥ 2 examples + 4 gap combos each ≥ 2 examples, asserted individuallya4c538a -
Graduated severity penalty: sev-5→-0.30, sev-4→-0.15, sev-3→-0.05 (66/66)
Soham BanerjeeReplaces flat -0.20 (only at sev-5) with graduated table: 5 → -0.30 imminent/coordinated/physical threats 4 → -0.15 doxxing, targeted harassment 3 → -0.05 repeated toxic behaviour ≤2 → 0.00 mild, allow/warn defensible Breakdown key renamed: missed_critical_threat_penalty → missed_threat_penalty Validator: 61 → 66 checks (all 4 severity levels + old-key-gone asserted)94717ed -
Cross-post campaign mechanic: campaign_id in state, +0.15 escalate-all bonus (61/61 checks)
Soham BanerjeeArchitecture: - campaign_id/campaign_post_index/campaign_total_posts injected into state obs so agent can reason about coordinated behaviour across posts - Queue mode: 33% chance of campaign episode vs standard mixed queue - _build_state_obs() enriches state dict; state() and reset() both use it - +0.15 bonus on final step when agent escalates ALL posts in a campaign - episode_actions tracked; bonus requires ALL actions == 'escalate' - campaign_bonus_{earned,value} exposed in info 3 campaigns (9 new scenarios): camp_crypto_001 — coordinated pump-and-dump spam (easy→medium→hard) camp_doxx_002 — escalating doxxing/harassment threat (easy→medium→hard) camp_disinfo_003 — coordinated health misinformation (easy→medium→hard) Dataset: 98 → 107 scenarios (37 easy / 25 medium / 45 hard) Validator: 53 → 61 checks — full campaign mechanic tested748cef6 -
10 ambiguous hard scenarios + full valid_actions test suite (53/53 checks)
Soham BanerjeeNew scenarios (scen_hard_36 to 40), all remove/shadowban grey areas: 36 — coded xenophobia dogwhistle (shadowban|remove) 37 — dehumanising repeat offender, moderate policy (remove|shadowban) 38 — non-English hate speech, strict policy (shadowban|remove) 39 — coordinated bot-network amplification, severity-5 (remove|shadowban) 40 — hate speech with satire claim (shadowban|remove) Hard tier: 37 → 42 scenarios | Ambiguous total: 5 → 10 Validator expanded to 53 checks: - both remove/shadowban score equally on ambiguous scenarios - wrong action (allow) does NOT score full - ambiguous count ≥ 10 asserted - coordination cluster count ≥ 2 asserted941d83d -
v2 docs & validation: README rewrite, openenv.yaml v2.0, validator 47/47
Soham Banerjee- README: accurate v2 docs (75 scenarios, queue mode, -0.3..1.0 reward, all 4 penalties, inference.py format, validate command) - openenv.yaml: version 2.0.0, penalty schema, queue episode_type, updated num_scenarios (25/20/30), measured baseline scores - validate_openenv.py: 47-check test suite (YAML, dataset, live API, reward bounds) — all passing68d61d8