Spaces:
Sleeping
Sleeping
Commit History
-
Fix campaign IDs: load dynamically from env instead of hardcoded values
Soham Banerjeee8094c5 -
Update README path to server/app.py structure and dependencies
Soham Banerjeec769b2e -
Upgrade Gradio UI theme and styling aesthetics
Soham Banerjee7c416c2 -
Fix campaign detection reset logic in UI
Soham Banerjeee177d10 -
Add permissive trailing slashes for API endpoints
Soham Banerjee54de384 -
Fix Dockerfile CMD to use server.app
Soham Banerjee7bf9d4b -
Merge branch 'main' of https://github.com/oki-dokii/Meta into bruh
Soham Banerjeeef72aeb -
Fix validate-submission issues: pyproject.toml setup, server entrypoint, app path, and uv.lock
Soham Banerjee29ae803 -
Merge branch 'bruh'
Soham Banerjeede331a4 -
fix: Appended the required score=<score> to [END] stdout logs for OpenEnv compatibility
Soham Banerjeeb377684 -
Merge branch 'bruh'
Soham Banerjeea054d58 -
chore: Strict compliance with OpenAI client AST init
Soham Banerjee494f108 -
Merge branch 'bruh'
Soham Banerjeeed47fce -
chore: strict compliance with OpenEnv inference env variables ast-validation
Soham Banerjeede96010 -
docs: Add architecture and core loop Mermaid diagrams
Soham Banerjee4f6ae75 -
docs: Add architecture and core loop Mermaid diagrams
Soham Banerjee47c3d7b -
Update README.md
Soham BanerjeeUnverified signature58cc6be -
docs: Fix penalties header to remove negative reward reference
Soham Banerjee6e93689 -
docs: Update reward constraints copy to 0.0-1.0 in UI and README
Soham Banerjeee486f66
-
Fix page unresponsiveness by removing demo.load() events
Jashandeep Singh +1: CopilotCRITICAL FIX: Removed demo.load() calls that were causing page freezes/hangs. These were triggering heavy operations on every page load. Changes: - Removed demo.load() for Try It tab (line 303) - Removed demo.load() for Campaign Detection tab (line 429) - Users now select from dropdown to load scenarios (cleaner UX) - Added helpful initial messages to guide users This fixes the 'Pages Unresponsive' dialog issue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>e18dfa2 -
Fix event handlers by moving functions to module level
Jashandeep Singh +1: CopilotCRITICAL FIX: Functions load_campaign() and submit_campaign() were defined inside the tab context, causing event handlers to fail. Moved both functions to module level (lines 127-173) alongside other tab handler functions. This fixes the 'buttons don't respond at all' issue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>783ba73 -
Simplify theme/CSS for Gradio 6.0 and improve performance
Jashandeep Singh +1: Copilot- Move theme/css to launch() for Gradio 6.0 compatibility - Remove backdrop-filter blur that causes performance issues - Simplify theme (remove complex gradients and custom fonts) - Remove heavy box-shadow and transform CSS rules - Keep minimal styling for better UI responsiveness Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>a617197 -
Fix Gradio theme/css initialization to make buttons work
Jashandeep Singh +1: CopilotMove theme and css parameters from demo.launch() to gr.Blocks() constructor. This fixes the issue where buttons and UI components were not rendering properly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>bb0d19d -
Fix Gradio 6.0 deprecation warnings and campaign loading
Jashandeep Singhb35b650 -
Add comprehensive multi-provider documentation and tests
Jashandeep Singh7f9923b -
Add multi-provider support: Groq (default), OpenAI, Grok
Jashandeep Singhd36acd9 -
Add Groq API support alongside OpenAI and Grok
Jashandeep Singh2a479c8 -
Fix README color for HF
Jashandeep Singh39f65af -
Deploy to HF Spaces
Jashandeep Singh +1: CopilotCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>ff6af76 -
Finalise OpenEnv submission: Clamp rewards to 0.0-1.0, update Gradio UI, and add Groq pipelines
Soham Banerjee192db9d
-
app.py: Fix Gradio 6 deprecations and unused imports
Soham Banerjee04de9f3 -
inference.py: Groq default + dynamic scenario loading + campaign/adversarial prompt hints
Soham Banerjee- Default endpoint: api.groq.com/openai/v1 (was api.openai.com/v1) - Default model: llama-3.3-70b-versatile (was gpt-4o-mini) - API key fallback chain: GROQ_API_KEY → HF_TOKEN → OPENAI_API_KEY - TASKS built dynamically from moderation_benchmark.json — covers all 128 scenarios including scen_adv_*, camp_*, scen_easy_33+ that hardcoded ranges missed - build_prompt: injects CAMPAIGN ALERT when campaign_id in state - build_prompt: injects ADVERSARIAL SCENARIO hint when is_adversarial=True - Smoke test: 3/3 easy = 1.0, hard = 0.60 on llama-3.3-70b-versatile2421327 -
README: add judge tip for deterministic reset(campaign_id=...) under campaign section
Soham Banerjee64f2c91 -
Restore moderation_benchmark.json from d741d4b (128 scenarios, 100/100 checks)
Soham BanerjeeData regression fix: the merge at abf8abc replaced the full 128-scenario benchmark with an 85-scenario version. Restored from the d741d4b commit which had all features intact: 128 scenarios (52 easy / 25 medium / 51 hard) 10 adversarial scenarios (appeal mechanic) 10 ambiguous hard scenarios (valid_actions) 3 campaign episodes (9 posts) 100/100 validator checks passingb2860e4 -
Merge + 3 fixes: README accurate (128 scen / real baselines), is_adversarial in state, reset(campaign_id) (100/100 checks)
Soham Banerjeefa17b3c -
3 fixes: README accurate, is_adversarial in state, reset(campaign_id) (100/100)
Soham BanerjeeFix 1 — README: all stale numbers updated 128 scenarios (52 easy / 25 medium / 51 hard) Baseline table: easy=0.375, medium=0.460, hard=0.144, overall=0.300 Penalty table: graduated severity (sev-5→-0.30, sev-4→-0.15, sev-3→-0.05) Bonuses table: campaign +0.15, appeal ±0.10 Observation space documented with campaign + is_adversarial fields Episode modes: Queue + Campaign + Single-step + Appeal examples Project structure: 128 scenarios, 100-check validator, appeal in env Fix 2 — is_adversarial in state obs (_build_state_obs) state['is_adversarial'] = True|False on every reset/state call Agents can condition on it BEFORE calling step(), before the appeal Fix 3 — reset(campaign_id='camp_crypto_001') Deterministically loads all posts for a campaign in post_index order Raises ValueError on unknown campaign_id Raises ValueError if both scenario_id and campaign_id provided Validator: 92 → 100 checks (8 new)10c3c6e
-
2b 3a
DayalGupta03abf8abc -
Appeal mechanic: is_adversarial + env.appeal() 2-turn flow (92/92 checks)
Soham BanerjeeArchitecture: - is_adversarial: bool on scenario — flags posts with appeal challenges - appeal_text: user's counter-argument (social engineering attempt or valid context) - appeal_verdict: 'uphold' | 'overturn' — what correct response is - env.appeal({verdict, rationale}) → {appeal_reward, verdict_correct, gt_verdict, ...} +0.10 correct verdict (resist manipulation OR accept valid context) -0.10 wrong verdict (fooled by appeal OR stubbornly upholding wrong decision) - _pending_appeal_scenario / _appeal_used state prevents double-calls - step() exposes appeal_available / appeal_text / appeal_verdict_gt in info - is_adversarial field present in info on every step 10 adversarial scenarios: 6 × uphold — manipulative appeals: educational framing, authority claim, fiction claim post-hoc, free-speech argument, research framing, stalking 'it's my novel' claim 4 × overturn — valid context: pest control misread, Holocaust education, anti-h…d741d4b -
Fill easy GT gaps: full label×action coverage (79/79 checks)
Soham Banerjee11 new easy scenarios plugging 4 missing/thin combos: misleading/shadowban 0 → 3 (health disinfo suppression) toxic/shadowban 0 → 3 (chronic low-grade toxicity) toxic/warn 1 → 4 (first-offence mild insults) misleading/escalate 1 → 3 (repeat high-harm misinformation) Easy tier: 37 → 48 scenarios | Overall: 107 → 118 scenarios Validator: 66 → 79 checks + all 4 easy labels each ≥ 2 examples + all 5 easy actions each ≥ 2 examples + 4 gap combos each ≥ 2 examples, asserted individuallya4c538a -
Graduated severity penalty: sev-5→-0.30, sev-4→-0.15, sev-3→-0.05 (66/66)
Soham BanerjeeReplaces flat -0.20 (only at sev-5) with graduated table: 5 → -0.30 imminent/coordinated/physical threats 4 → -0.15 doxxing, targeted harassment 3 → -0.05 repeated toxic behaviour ≤2 → 0.00 mild, allow/warn defensible Breakdown key renamed: missed_critical_threat_penalty → missed_threat_penalty Validator: 61 → 66 checks (all 4 severity levels + old-key-gone asserted)94717ed -
Cross-post campaign mechanic: campaign_id in state, +0.15 escalate-all bonus (61/61 checks)
Soham BanerjeeArchitecture: - campaign_id/campaign_post_index/campaign_total_posts injected into state obs so agent can reason about coordinated behaviour across posts - Queue mode: 33% chance of campaign episode vs standard mixed queue - _build_state_obs() enriches state dict; state() and reset() both use it - +0.15 bonus on final step when agent escalates ALL posts in a campaign - episode_actions tracked; bonus requires ALL actions == 'escalate' - campaign_bonus_{earned,value} exposed in info 3 campaigns (9 new scenarios): camp_crypto_001 — coordinated pump-and-dump spam (easy→medium→hard) camp_doxx_002 — escalating doxxing/harassment threat (easy→medium→hard) camp_disinfo_003 — coordinated health misinformation (easy→medium→hard) Dataset: 98 → 107 scenarios (37 easy / 25 medium / 45 hard) Validator: 53 → 61 checks — full campaign mechanic tested748cef6 -
10 ambiguous hard scenarios + full valid_actions test suite (53/53 checks)
Soham BanerjeeNew scenarios (scen_hard_36 to 40), all remove/shadowban grey areas: 36 — coded xenophobia dogwhistle (shadowban|remove) 37 — dehumanising repeat offender, moderate policy (remove|shadowban) 38 — non-English hate speech, strict policy (shadowban|remove) 39 — coordinated bot-network amplification, severity-5 (remove|shadowban) 40 — hate speech with satire claim (shadowban|remove) Hard tier: 37 → 42 scenarios | Ambiguous total: 5 → 10 Validator expanded to 53 checks: - both remove/shadowban score equally on ambiguous scenarios - wrong action (allow) does NOT score full - ambiguous count ≥ 10 asserted - coordination cluster count ≥ 2 asserted941d83d -
Close all 4 scoring gaps (+~6 pts)
Soham BanerjeeGap 1 — Easy warn/escalate GT actions: 5 new scenarios Gap 2 — Easy misleading label: 2 new scenarios Gap 3 — Ambiguous hard scenarios: 5 new with valid_actions list _compute_reward now checks valid_actions for multi-correct scoring Gap 4 — Cross-post coordination: 6 clustered scenarios (2 clusters x 3 posts) Queue mode now 25% cluster episodes w/ 0.10 bonus for handling all correctly Dataset: 75 → 93 scenarios (34 easy / 22 medium / 37 hard) 2 coordination clusters: spam_cluster_A, harassment_cluster_B 47/47 validation checks still passing9bc46b3 -
v2 docs & validation: README rewrite, openenv.yaml v2.0, validator 47/47
Soham Banerjee- README: accurate v2 docs (75 scenarios, queue mode, -0.3..1.0 reward, all 4 penalties, inference.py format, validate command) - openenv.yaml: version 2.0.0, penalty schema, queue episode_type, updated num_scenarios (25/20/30), measured baseline scores - validate_openenv.py: 47-check test suite (YAML, dataset, live API, reward bounds) — all passing68d61d8 -
Update inference.py: expand TASKS to all 75 scenarios (25 easy / 20 medium / 30 hard)
Soham Banerjee- Updated scenario_ids ranges to match expanded benchmark dataset - baseline_results.json refreshed (75 scenarios, overall mean 0.459)2426958 -
phase 1 and phase 2a
DayalGupta0378d0a45
-
ContentModerationEnv v1.0 — complete OpenEnv benchmark
Soham Banerjee- 60-scenario benchmark dataset (easy/medium/hard tiers) - Full OpenEnv spec: step() / reset() / state() API - openenv.yaml manifest, Pydantic v2 typed models - Reproducible lexical baseline (no API keys required) - 3-step LLM pipeline (Gemini-only version included) - Gradio app for Hugging Face Spaces deployment - Dockerfile, README with full setup instructions2a39e79