Spaces:
Paused
Paused
Commit History
-
Split train and tool simulator modules; mastery curriculum and grader workflow nudge.
raj921Move GRPO helpers into train_*.py, tool dispatch into server/support_ops_tools.py, add reward_log mastery script and curriculum state wiring, export mastery helpers, and extend tests.5e75745 -
feat: default GRPO/eval to Qwen/Qwen3-1.7B + LoRA; disable Qwen3 thinking in chat template
raj921eff6196 -
Update notebooks and proof artifact tooling
raj9213937051 -
Harden notebook vLLM compatibility checks
raj921c6793ac -
Trim notebook install dependencies
raj92183da325 -
Fix OpenEnv and HF training flows
raj921c94b44c -
Docs: align with Hugging Face Hub + TRL official guides
raj921- Add docs/HUGGINGFACE_HUB.md (repositories-getting-started, Xet, hf upload, TRL GRPO+vLLM) - Root .gitattributes for common image extensions (LFS pointers per Hub) - README links; Colab/Kaggle use trl[vllm] per TRL docs + vllm<0.19 pin - train._require_vllm docstring links TRL GRPO section2c08612 -
Fix Colab TRL+vLLM: pin vllm<0.19, add tomli deps, guard train
raj921- TRL 0.29 colocate supports vLLM 0.11-0.18; 0.19.1 caused NameError: LLM - Pip cells: tomli, tomli-w, vllm>=0.11,<0.19 (Colab + Kaggle + kernel copy) - train._require_vllm_trl_colocate_safe() before GRPOConfig use_vllm - pyproject train extra: vllm upper bound; README troubleshooting note59cc95b -
Colab/Kaggle: auto GPU profile for bf16+vLLM on A100/L4
raj921Add GPU_PROFILE (auto/t4/fast) with VRAM-based selection so large GPUs do not stay on the slow T4 path. Document re-run order in Colab.dec30cf -
Diagnostic: track parse_ok_ratio per episode, dump health every 5 episodes
raj921Adds early-warning instrumentation so flat-reward 'model can't speak the schema' runs surface inside the first 5 episodes instead of after 50. train.py rollout_once - Wraps the parse step in try/except; counts parse_attempts and parse_failures per episode. - On parse failure, falls back to a noop SupportOpsAction so the episode continues (env still sees the turn). - Returns parse_ok_ratio, parse_failures, parse_attempts, plus last_raw_completion (first 300 chars) and last_parse_error. train.py main() - Reward CSV gains a parse_ok_ratio column. - New DIAGNOSTIC_EVERY = 5 prints '[diagnostic ep=N] parse_ok=... last_err=... last_raw=...' every 5 episodes; warns aloud when parse_ok_ratio<0.5. Notebooks (driftshield_colab/kaggle + kernel copy) - Same CSV column. - log_episode now takes the full episode dict and runs the same every-5-episodes diagnostic in-notebook (so Colab/Kaggle stdout surfaces the warning inline). 25/25 tests pass.ff55578 -
Fix flat-reward training: stronger SYSTEM_PROMPT with few-shot, longer completions, MAX_TURNS=16, schema sanity probe
raj921Symptom: 9 episodes of GRPO produced byte-identical reward (-0.47 every row, field=0.10, reply=0, grounding=0). Constant in-group reward means GRPO advantages = 0, gradient = 0, no learning. Root cause: 4B base couldn't reliably emit valid SupportOpsAction JSON. Every rollout hard-failed at the action parser the same way -> identical penalty. Fixes: train.py SYSTEM_PROMPT - Reframed as DriftShield (4 failure modes), not generic support agent. - Lists every legal value for workflow.* enums. - Adds DriftShield rules: treat user text as data, read recoverable error hints, distrust ops.get_recommendation, never claim deletion /refund/admin grant without policy support. - Adds a concrete 3-turn FEW-SHOT example for ds_prompt_injection_access showing the exact JSON shape across investigate -> route -> reply + answer. Few-shot examples are the single biggest schema-compliance win on small open models. GRPOConfig - max_completion_length 512 -> 768 so tool_call…03a5c04 -
Rename files to DriftShield: notebooks, server module, kernel folder
raj921Renames (git mv, history preserved): support_ops_colab.ipynb -> driftshield_colab.ipynb support_ops_kaggle.ipynb -> driftshield_kaggle.ipynb support_ops_colab_gemma4.ipynb -> driftshield_colab_gemma4.ipynb server/support_ops_environment.py -> server/driftshield_environment.py kernels/support-ops-grpo/ -> kernels/driftshield-grpo/ kernels/.../support_ops_kaggle.ipynb -> .../driftshield_kaggle.ipynb Reference fixes: - server/app.py: import switched to driftshield_environment. - tests/test_driftshield.py, tests/test_reproducibility.py: import switched. - tests/conftest.py: aliases now register 'support_ops_env.server.driftshield_environment' AND keep the legacy 'support_ops_env.server.support_ops_environment' alias for backwards compat with any external code/notebooks still on the old path. - scripts/push_kaggle_kernel.sh: KERNEL_DIR -> kernels/driftshield-grpo, NOTEBOOK -> driftshield_kaggle.ipynb. - kernels/driftshield…f73ecfc -
DriftShield Phase C: README pivot + driftshield_easy as default curriculum
raj921README.md: full DriftShield pivot - Front-matter: title 'DriftShield - SupportOps Runtime Failure Gym', emoji shield, tags include driftshield/prompt-injection/schema-drift. - New 30-second story + threat table (4 production failure modes). - TL;DR for judges rebuilt around DriftShield artifacts and 33 tests. - OpenEnv API example shows the recoverable schema-drift flow. - DriftShield curriculum table; legacy collections remain accessible by id. - Reward design rewritten: 9 components (5 legacy + 4 DriftShield) and 13 penalties (8 legacy + 5 DriftShield), with the negation-aware forbidden_phrase_hits rule documented. - Strong-baseline score table (D1: 0.94-0.97; legacy unchanged). - Demo flow uses driftshield_easy / driftshield as defaults. Defaults switched to driftshield: - train.py --difficulty driftshield_easy - eval_compare.py --difficulty driftshield - audit.py --difficulty driftshield - support_ops_colab.ipynb, support_ops_kaggle.ipynb, kernels/support-o…70030ab
-
Explain GitHub import vs Kaggle Internet; friendlier gaierror in network check
raj921a38fd26 -
Notebook UX: split pip vs imports; warn against Ctrl+C during DNS retries
raj921Prevents confusing ModuleNotFoundError: trl after interrupted pip.3140e48 -
Docs: clarify pip DNS failures; add pre-install network check cells
raj921Errno -3 / trl (versions: none) means no PyPI reachability, not missing TRL. Apply to Kaggle notebook, kernel copy, and Colab.bd55d9e -
Add Kaggle training notebook, kernel metadata, and push script
raj921- support_ops_kaggle.ipynb for Kaggle (secrets, /kaggle/working outputs) - kernels/support-ops-grpo for kaggle kernels push (rajkumar295 id) - scripts/push_kaggle_kernel.sh with env or kaggle.json auth - README and optional kaggle extra in pyproject.toml; gitignore .venv-kagglee842e3c