Phase 2 traded off MEDIUM-tier rogue-catch (20% → 0%) for resolution
gains. Phase 3 needs to recover both. The new flag scales BOTH the
OversightRubric catch-bonus (+50) and false-positive penalty (-75) by
the same factor so calibration pressure is preserved while the
absolute gradient signal on flag_rogue actions is amplified.
* compute_step_reward(rogue_bonus_multiplier=1.0) — backwards-compatible
* make_reward_fn forwards multiplier into _score_completion which
re-builds an OversightRubric on the fly when multiplier != 1.0
* CLI flag --rogue-bonus-multiplier (default 1.0)
* scripts/jobs_grpo_train.sh: GRPO_ROGUE_MULTIPLIER env var
Smoke-tested: 2.0× yields catch=+100, FP=-150, oversight reward stream
exactly doubles. 110/110 unit tests still green (the existing tests
all use the default multiplier=1.0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add A/B comparison Job for trained-policy showdown
scripts/ab_compare.sh evaluates two LoRA repos on the same 540-episode
sweep, builds a side-by-side report and an overlay plot, and uploads
both to whichever repo wins on summed trained mean reward across tiers.
Used to pick between the Phase-1 and Phase-2 LoRAs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase-1 submission upgrade for the hackathon: previous run used the
TRL default LR (5e-6) with EASY-only data and saw a flat reward curve
+ KL=0.14. Three new knobs let us re-target without code edits:
* --learning-rate (default 5e-6, set 2e-5 to break the flat-reward
plateau without touching anything else)
* --temperature (default 0.7, set 0.8 for more exploration in Phase 2)
* --curriculum-schedule "easy:200,medium:200,hard:200" — pre-rolls a
step-budget tier sequence so GRPO sees increasing difficulty over
training instead of EASY for all 600 steps. Falls back to the old
--start-tier behavior when the flag isn't passed.
scripts/jobs_grpo_train.sh forwards GRPO_LR / GRPO_TEMP /
GRPO_CURRICULUM env vars; everything else is unchanged. 110/110 unit
tests pass (helper is a no-op when the schedule env var isn't set).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add HF Jobs entry-point for post-training evaluation
scripts/jobs_evaluate.sh: pull the LoRA from helloAK96/chaosops-grpo-lora,
run chaosops.train.evaluate across all (policy × tier × failure_type),
regenerate a labelled comparison_curve.png, upload back to the model repo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add transformers-backend GRPO loader (no triton/Unsloth dep) + fix Jobs deps
The first HF Jobs validation hit two image-level issues:
* pip's resolution pulled torch 2.11+cu130 into a CUDA-12.4 image, so
bitsandbytes failed to load (libnvJitLink.so.13 missing)
* Unsloth requires triton, which JIT-compiles a CUDA helper at runtime
and needs `cc` — the slim pytorch image doesn't ship one
Changes:
* grpo_train.py: add load_transformers_model() (plain transformers +
peft + bnb 4-bit) and load_model() dispatcher with auto-fallback
* CLI flag --backend {auto,unsloth,transformers}
* scripts/jobs_grpo_train.sh: pin torch 2.4.1+cu124, bitsandbytes 0.43.3,
upper-bound transformers/trl/peft/datasets/accelerate to versions that
ship CUDA-12 wheels; default GRPO_BACKEND=transformers so triton
isn't needed in the Job container
110/110 tests still green; new code paths only execute on the GPU side.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
scripts/jobs_grpo_train.sh runs inside the Job container with the Space
repo mounted read-only at /data. Symlinks /data into /tmp/chaosops so
`import chaosops` resolves, installs unsloth/trl/peft/bitsandbytes,
runs grpo_train, renders a labelled learning_curve.png, and (when
GRPO_PUSH_TO_HUB=1) uploads the LoRA + metrics to
helloAK96/chaosops-grpo-lora.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
