Updated the Dockerfile to include additional directories for data and scripts, ensuring a more organized structure. Added new training notebooks for DeepSeek-R1 and Nemotron-3 models, providing end-to-end reproducibility for SFT and RLVR training processes. Revised the README.md to include links to the new notebooks and improved clarity on model training and evaluation steps. Enhanced the .gitignore file to exclude unnecessary files and directories, streamlining the project structure.
chore: increase max prompt and sequence lengths in training scripts
Updated the `max-prompt-length` in `train_rlvr_genral.py` and `max-seq-length` in `train_sft_genral.py` from 2048 to 4096 to accommodate larger input sizes, enhancing the training capabilities for RLVR and SFT tasks.
feat: add new training jobs and scripts for DeepSeek and Nemotron models
Introduced new YAML job specifications for training DeepSeek and Nemotron models, including environment configurations and command setups. Added corresponding shell scripts for job submission and a README.md to guide users on running the jobs. Implemented new training scripts for RLVR and SFT processes, enhancing the overall training framework and providing clear instructions for model training and evaluation.
refactor: update training scripts and environment setup for Qwen3 model
Modified training scripts to transition from Qwen2.5 to Qwen3, including updates to model and adapter paths in `train_rlvr.py` and `train_grpo.py`. Removed legacy environment variables from setup scripts and README.md, introducing new command-line arguments for stub workers and LLM judge control. Enhanced documentation to reflect these changes and improve user guidance for the updated training process.
refactor: update training scripts and documentation for SFT and RLVR processes
Revised training scripts to replace GRPO with RLVR as the default training path, enhancing the workflow for SFT and RLVR tasks. Updated README.md and runbook to reflect changes in command usage, including new parameters for data preparation and training. Improved clarity on the training process and added details for using the new `run_data_pipeline.py` script. Adjusted example commands and parameters to ensure consistency and better user guidance.
feat: add environment setup and requirements for Lightning AI H100 training
Introduced new files for setting up the Lightning AI H100 environment, including a comprehensive README.md detailing the installation process, requirements_lightning_h100.txt for pip installations, and setup_lightning_h100.sh for automated setup. Added requirements_frozen.txt to capture the exact environment configuration. Included evaluation results in JSONL format and summary files for model performance comparison, enhancing the training and evaluation workflow for SFT, GRPO, and RLVR tasks.
feat: update README and runbook for SFT and GRPO training enhancements
Revised the README.md and runbook to reflect changes in the SFT and GRPO training processes, including default input/output paths and parameters. Updated example commands for data preparation and training scripts to incorporate new defaults for merging datasets. Added details on the new `e1_m1_h1_examples.jsonl` file and clarified the training steps for improved user guidance. Introduced new JSONL files for verification results and summaries to enhance tracking of model performance.
feat: update evaluation results and training scripts for Qwen2.5-7B-Instruct
Updated evaluation metrics and added new JSONL files for the Qwen2.5-7B-Instruct model, including detailed task performance data. Enhanced training scripts to reflect changes in output paths and parameters, improving the training process for GRPO and SFT. Updated visualizations and metadata to ensure comprehensive tracking of model performance across various tasks.
Enhanced training scripts for SFT and GRPO to utilize Unsloth and Hugging Face TRL, including new parameters for max steps. Introduced a judge-rerunnable notebook for streamlined training and evaluation processes. Updated README.md and runbook to reflect these changes and provide clearer instructions for users.
feat: add evaluation and plotting scripts for CORP-ENV
Introduced eval.py for evaluating CORP-ENV policies with various models and outputting results in JSONL format. Added plot_results.py to generate visualizations from evaluation data, including average terminal rewards and invalid action rates. Updated .gitignore to exclude new plan files and added optional dependencies for plotting and training in pyproject.toml. Enhanced README.md with examples for evaluation and plotting usage, and included a new runbook for Lightning AI and Hugging Face integration.
