SC-repro
Collection
่ฟๆฏๆๅค็ฐ SC ็ๅฎ้ช โข 10 items โข Updated
Text Generation
Transformers
Safetensors
qwen3
reinforcement-learning
score-centering
code-generation
experimental
conversational
text-generation-inference
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Palind/Qwen3-4B-SC-CodeAgent-500Steps")
model = AutoModelForCausalLM.from_pretrained("Palind/Qwen3-4B-SC-CodeAgent-500Steps", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))Quick Links
Qwen3-4B-SC-CodeAgent-500Steps
่ฟๆฏๅๅง Qwen/Qwen3-4B ็ป Score Centering๏ผSC๏ผๅผๆญฅไปฃ็ ไปปๅก่ฎญ็ปๅพๅฐ็ๆข็ดขๆงๆฃๆฅ็นใไป็ฌฌ 200 ๆญฅ่ฎญ็ปๆฃๆฅ็นๆขๅคๆจกๅใไผๅๅจ็ญ็ถๆ๏ผ็ปง็ปญ่ฎญ็ป่ณ็ฌฌ 500 ๆญฅ๏ผๅๅธไธบ BF16 ๆจ็ๆ้ใ200 ๆญฅ็ๆฌไฟ็็ฌ็ซไปๅบใ
ๆฌไปๅบๅๅธ็ๆฏ็ฌฌ 500 ๆญฅ๏ผไธๆฏๅผๅ้ไธ่กจ็ฐๆๅฅฝ็ๆฃๆฅ็นใ ๅฎ้ๅๅค็ฐๅฎ้ชๅๅๆ่ฎญ็ป่กไธบ๏ผ็ฎๅไธ่ฝๆฎๆญคๅฎฃ็งฐ้็จไปฃ็ agent ่ฝๅๆ SC ็ฎๆณไผๅฟใ
ๆๆ
ไปฅไธๅไธบๅไธๅบๅฎ 256 ้ข TACO ๅผๅ้ไธ็็ปๆใๆฏ้ขๅช้ๆ ทไธๆฌก๏ผn=1ใtemperature=0.6ใtop_p=1.0ใtop_k=-1๏ผๆฏ 25 ๆญฅ่ฏไผฐไธๆฌกใ
| ๆญฅๆฐ | ๅ จๆต่ฏ้่ฟ้ขๆฐ | ้ข็ฎๅ็กฎ็ | ้้ขๆต่ฏ้่ฟ็ๅๅผ | ๆๅฏๆๅไปฃ็ |
|---|---|---|---|---|
| 200 | 130/256 | 50.78% | 56.11% | 61.33% |
| 350 | 133/256 | 51.95% | 61.59% | 74.61% |
| 475 | 135/256 | 52.73% | 63.57% | 89.84% |
| 500 | 118/256 | 46.09% | 60.79% | 91.80% |
็ฌฌ 475 ๆญฅๆฏๅทฒ่ฎฐๅฝ้ช่ฏ็นไธญ็ๆ้ซๅ็กฎ็๏ผไฝๅฝๆถๆฒกๆไฟๅญๆ้๏ผไธ่ฝไป่ฏไผฐๆฅๅฟๆขๅค่ฏฅๆจกๅใๆฌๆฌกๅๅธไธไปฅ็ฌฌ 475 ๆญฅๅๆฐไปฃ่กจ็ฌฌ 500 ๆญฅๆ้ใ
acc ่ฆๆฑไธ้ข็ๆ็ปไปฃ็ ้่ฟๅ
จ้จไฟๅญๆต่ฏ๏ผ่ฎญ็ปๅฅๅฑ pass_rate ๆฏ่ฏฅ้ข้่ฟๆต่ฏ็ๆฏไพ๏ผๅ้้ขๅนณๅใไธค่
ไธๅ๏ผไป 200 ๅฐ 500 ๆญฅ๏ผๅนณๅๆต่ฏ้่ฟ็ไธๅ๏ผไฝๅ
จๅฏน้ขๆฐไธ้ใformat_ok ๅช่กจ็คบๆซๆฌกๅฉๆๅ็ญไธญ่ฝๆๅไปฃ็ ๏ผไธไปฃ่กจไปฃ็ ๆญฃ็กฎใ
ๅ จ็จ 32,000 ๆก่ฎญ็ป่ฝจ่ฟนไธญ๏ผๅชๆ 41 ๆก๏ผ็บฆ 0.13%๏ผๅฎ้ ๆง่ก่ฟๅทฅๅ ท๏ผๆ็ปๅผๅ้ๅทฅๅ ทไฝฟ็จ็ไธบ 0ใๆๅ 50 ๆญฅ็่ฎญ็ปๆชๆญ็ๅนณๅไธบ **8.34%**๏ผ็ฌฌ 200 ๆญฅไนๅๆๅ 50 ๆญฅไธบ 40.81%ใๆ ผๅผไธๆชๆญๆๆๆนๅ๏ผไฝๆฒกๆๅฝขๆ็จณๅฎ็ๅทฅๅ ทไฝฟ็จ่กไธบใ
ๅฎๆดๆฒ็บฟๆฐๅผ่ง้ๆญฅๆๆ ใSwanLab๏ผ1โ200 ๆญฅใ201โ500 ๆญฅใ
ๆนๆณไธๆฐๆฎ
- ๆกๆถไธ่ตๆบ๏ผverl ๅ็
ToolAgentLoop+ ่ชๅฎไนใๆ ็ถๆ็run_python๏ผ8 ร A100 80GB๏ผ4 ๅผ FSDP2 ่ฎญ็ปใ4 ๅผ vLLM rolloutใไธๆฏ HarborใTerminal-Bench ๆ SWE ็ฏๅขใ - ็ฎๆ ๏ผREINFORCE ๅฝขๅผ็็ญ็ฅๆๅคฑ + top-128 ่ฟไผผ SC๏ผๆฏ้ข 4 ไธชๅ็ญ๏ผๅฅๅฑๅๅป็ปๅ ๅๅผ๏ผไธๆๆ ๅๅทฎๅฝไธๅใไธไฝฟ็จ TISใRSใPPO clippingใcriticใKL ๆฉ็ฝๆๅทฅๅ ท่ฐ็จๅฅๅฑใ
- ไผๅ๏ผAdamW๏ผๆๅฎๅญฆไน ็
1e-6๏ผweight decay=0๏ผๆขฏๅบฆ่ๆฐ่ฃๅช 1.0๏ผBF16 ่ฎก็ฎใFP32 ไธปๅๆฐใๆฏ่ฎญ็ปๆญฅ 16 ้ข ร 4 ๆก่ฝจ่ฟนใ - ๅผๆญฅ๏ผ่ช็ถ
separate_async๏ผๆฏๆญฅๅๆญฅๆ้๏ผmax_off_policy_threshold=4ใdrop๏ผไธๆฏไบบไธบ็ปๆๆๆ ทๆฌๅบๅฎๅขๅ 4 ๆญฅๅปถ่ฟใ - ้ฟๅบฆไธ้ๆ ท๏ผthinking ๆจกๅผ๏ผprompt ไธ้ 4,096ใๅๅบไธ้ 8,192 tokens๏ผๆๅค 6 ไธชๅฉๆ่ฝฎๆฌกใ่ฎญ็ปไฝฟ็จ temperature=1.0ใtop-p=1.0๏ผไธๅฏ็จ็ๆ top-k ๆชๆญใSC ็ top-128 ๆฏๆฆ็่ฎฐๅฝๅฎฝๅบฆใ
- ๆฐๆฎ๏ผTACO ็ๅบๅฎ่ฎญ็ปๅ็ 02ใ03ใ04๏ผ็ปๆ ๅ่พๅ
ฅ/่พๅบๆฅๅฃ็ญ้ใ่บซไปฝๅป้ๅ้็ฆปๅ่่งฃๆ ก้ช๏ผๅๅไธบ 2,000 ้ข train + 256 ้ข devใ่ฟไธๆฏๅฎๆน TACO test๏ผไนไธๆฏๆดไธช TACO ็้ๆบๆฝๆ ทใไธๆธธ revision๏ผ
d593ed0a2becbbc952230bb89be09189bf1056dcใ
้่ๆต่ฏๅชไพๆ็ปๅค้ข๏ผไธไฝไธบๅทฅๅ ทๅ้ฆใPython ็จๅบๅจ Landlock + seccomp ้็ฆป็ฏๅขไธญ่ฟ่ก๏ผ้ๅถ็ฝ็ปใๆไปถๅ่ฟ็จ่ฎฟ้ฎ๏ผๅๆต่ฏ 3 ็งใๆฏ้ข็ดฏ่ฎกๆง่ก้ข็ฎ 90 ็ง๏ผๆ้ไธ่ฎกๅ ฅ่ฏฅ้ข็ฎใๅฎๆดๅๆฐ่ง้ ็ฝฎๆ่ฆใ
ไฝฟ็จ่ๅดไธๅฑ้
ๆฌไปๅบๅ ๅซ BF16 ๆจ็ๆ้ใtokenizerใๆๆ ๅ้ ็ฝฎๆ่ฆ๏ผไธๅซไผๅๅจ็ถๆๆๅฎๆด agent harnessใๅทฒ้ๅผ ้ๆ ธ้ชๅฏผๅบ็ปๆไธๆบๅ็็ BF16 ่ฝฌๆขไธ่ด๏ผๅนถๅฎๆๆฌๅฐๅ ่ฝฝๅ็ญๅๅๆฃๆฅ๏ผ่ฏฆ่ง่ฝฌๆขๆธ ๅใไปฅไธๅๆฐๆฅ่ช่ฎญ็ปๆ้ด่ฏๆต๏ผๅฏผๅบๅๆช้ๆฐ่ฟ่กๅฎๆดๅผๅ้ใ
ๅๆฌก่ฟ่ก๏ผๆฒกๆ step-0 ๅบ็บฟใๅน้ ็ๆ SC ๅฏน็ งๆๅค็งๅญ้ๅค๏ผๅผๅ้ๅทฒ่ขซๅๅค่ฏไผฐ๏ผๅฐๆ LiveCodeBench ๅๆฐใไธ่ฝๆฎๆญคๅคๆญ SC ไผๅฃๆ้็จ agent ่ฝๅ๏ผไนไธ่ฝไป ๅญๆๅไธๆฌกๅๆฐไธ้ๆญ่จ็จณๅฎ้ๅใ
ๅ่๏ผScore Centering ่ฎบๆใverlใQwen3-4Bใ
- Downloads last month
- 28
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Palind/Qwen3-4B-SC-CodeAgent-500Steps") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)