Spaces:
Sleeping
Sleeping
Commit History
-
fix: dtype kwarg (torch_dtype deprecated), vLLM max_model_len=4096
- grpo_train: torch_dtype → dtype (transformers 4.45+ renamed it) - secalign_agent: add max_model_len=4096 to vLLM LLM() init; vLLM was trying to allocate KV cache for full 131072-token context (16 GiB) but only 5.4 GiB is free after Qwen bf16 occupies ~3 GiB. With max_model_len=4096 the KV cache drops to ~500 MB — fits easily. SecAlign agent tasks never need more than 4096 tokens. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>aaa7c61 -
fix: max_completion_length 512→128, firewall circuit-breaker
- max_completion_length 512→128: completions/clipped_ratio was 1.0 on every step (all payloads truncated). 128 tokens is enough for any injection payload and cuts per-step generation time ~4x. - FirewallWrapper: convert _scan_user from @staticmethod to instance method; add _fw_user_broken circuit-breaker so after the first llamafirewall scan error the fallback to PG2 is silent for the rest of the session (was logging on every step, ~10+ times per training step). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>b7d3a14 -
fix: drop BnB 4-bit, load attacker in plain bf16
BitsAndBytes 4-bit pins lm_head (and other non-convert modules) to fp32 regardless of the dtype= kwarg, which collides with bf16 hidden states inside GRPO's generate() path -> 'expected Float, found BFloat16' at F.linear. The post-load cast loop did not stick reliably under device_map='auto' + tied embeddings. Qwen2.5-1.5B in bf16 is ~3 GB on A100 -- 4-bit was unnecessary. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>6aebb94 -
fix: dtype kwarg + cast lm_head/embeds to bfloat16 to fix GRPO generate
33bf00a -
fix: add torch_dtype=bfloat16 to prevent Float/BFloat16 mismatch in GRPO
b42adcc -
fix: GRPO batch_size must be divisible by num_generations (1→4)
0d411fb -
fix: skip Unsloth in GRPO trainer (grpo_accumulated_loss signature mismatch)
Unsloth 2025.11.x patches TRL's GRPOTrainer with an incompatible internal signature. Switch attacker to plain transformers+PEFT — reward eval dominates latency so the generation speedup doesn't matter. Also lower SecAlign vLLM gpu_memory_utilization 0.70→0.55 to fit alongside the Qwen training model on A100-40GB. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>17a9ff7 -
fix: stub GuidedDecodingParams for vLLM 0.19+ / TRL compatibility
vLLM 0.19.1 moved GuidedDecodingParams; TRL's grpo_trainer imports it at module load time. We don't use vLLM-guided decoding so a type stub is safe. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>deab900 -
Phase 5: training pipeline — client, GRPO trainer, eval, baselines (23 handcrafted attacks)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>550a83e
-
Phase 0: bootstrap
Create directory tree (env/, train/, tests/, scenarios/, notebooks/, demo/, docs/plots/), pyproject.toml with Mac-only base deps plus [gpu] and [demo] optional groups, .gitignore, README skeleton, and the colab_runner.ipynb bridge with 8 labeled cells (stubbed until their phases). Verified pip install -e ".[demo]" succeeds and `import env` works on the 3.11 venv. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>15bf5e6