flowingpurplecrane commited on
Commit
fafd3c8
·
verified ·
1 Parent(s): 36add26

method-robots-rebn-0826-0821 README: steps [20, 25, 30, 35, 40]

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3.5-4B-Base
3
+ library_name: peft
4
+ ---
5
+
6
+ # method-robots-rebn-0826-0821
7
+
8
+ LoRA adapters from mt-moral-alignment run `method-robots-rebn-0826-0821`, one subfolder per checkpoint,
9
+ converted from verl FSDP shards. Per-episode logs:
10
+ https://huggingface.co/datasets/agentic-moral-alignment/mtma/tree/main/runs/train/method-robots-rebn-0826-0821
11
+
12
+ | step | load |
13
+ |---|---|
14
+ | 20 | `subfolder="step_20"` |
15
+ | 25 | `subfolder="step_25"` |
16
+ | 30 | `subfolder="step_30"` |
17
+ | 35 | `subfolder="step_35"` |
18
+ | 40 | `subfolder="step_40"` |
19
+
20
+ ```python
21
+ base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B-Base")
22
+ model = PeftModel.from_pretrained(base, "agentic-moral-alignment/method-robots-rebn-0826-0821", subfolder="step_40")
23
+ ```
24
+
25
+ WARNING: vLLM cannot serve these adapters on Qwen3.5 -- hybrid-GDN LoRA is a
26
+ silent no-op (vllm#49354). Merge first: `model.merge_and_unload().save_pretrained(...)`.
27
+
28
+ ## Frozen run config
29
+
30
+ ```yaml
31
+ # frozen for method-robots-rebn-0826-0821 (2026-08-26T13:44:32Z, git d403f7a)
32
+ STAGE: train
33
+ MODEL: Qwen/Qwen3.5-4B-Base
34
+ GPUS: 1
35
+ TP: 1
36
+ MAX_SEQS: 64
37
+ GPU_MEM_UTIL: 0.72
38
+ GAMES: robots
39
+ BATCH_GAME: 4
40
+ START_SAMPLE: True
41
+ START_RESAMPLE_STEP: True
42
+ TURN_CAP: 64
43
+ N: 8
44
+ STEPS: 40
45
+ LR: 1.5e-05
46
+ TOP_P: 0.95
47
+ ROLLOUT_SEED: 42
48
+ FRAMING: game-goals
49
+ LAM: 1
50
+ TASK_WEIGHT: 0
51
+ MORAL: deon_soft
52
+ RETURN_FN: reward_to_go
53
+ BASELINE_FN: batch_mean
54
+ NORMALIZE_FN: batch_std
55
+ GAMMA: 1.0
56
+ OMEGA: 0
57
+ TD_LAMBDA: 0.8
58
+ THINKING: True
59
+ MAX_TURN_TOKENS: 4096
60
+ ENV_TIMEOUT_S: 60
61
+ AGENT_WORKERS: 2
62
+ REWARD_WORKERS: 1
63
+ BYPASS: True
64
+ GRAD_CKPT: True
65
+ CALC_ENTROPY: True
66
+ SAVE_FREQ: 5
67
+ NOSET: 0
68
+ PUSH: 0
69
+ PROMPT_LEN: 4096
70
+ RESPONSE_LEN: 40960
71
+ MAX_MODEL_LEN: 45056
72
+ TOKEN_LEN_PER_GPU: 49152
73
+ LORA_RANK: 32
74
+ LORA_ALPHA: 64
75
+ ```