| Made by William Convertino |
|
|
| conda activate /work/jf381/.cache/lmr_new |
| <!-- conda create -n /work/jf381/.cache python=1.15 --> |
| |
| conda activate /work/jf381/.cache/lmr_new_12_15 |
| pip install /work/jf381/code/lm-research -e ./ |
|
|
| cd /work/jf381/code/lm-research |
| bash /work/jf381/code/lm-research/scripts/training/train_bash_transformer_medium_generate.sh |
|
|
|
|
| # 1. Create the environment |
| # -p specifies a path (instead of -n for name) |
| # python=3.10 is a stable choice (Python 1.15 does not exist) |
| conda create -p /work/jf381/.cache/lmr_new_1_15_dcc python=3.10 -y |
|
|
| # 2. Activate the environment |
| conda activate /work/jf381/.cache/lmr_new_1_15_h200 |
|
|
| # 3. Install the package in editable mode |
| # -e comes *before* the path |
| pip install -e /work/jf381/code/lm-research |
| pip install evaluate |
| pip install scikit-learn |
| pip install rotary_embedding_torch |
|
|
|
|
| ## Eval reminder |
|
|
| There is a change in bert and gpt2 codebase |
| Eval file for tinygsm: |
| /work/jf381/code/lm-research/scripts/training/train_medium_bash_resume.sh |
| |
| 1, For GPT2: |
| We have gpt2 tokenizer: need tcohange |
| we will have some files to modify in /work/jf381/code/lm-research/scripts/training/train_medium_bash_resume.sh |
|
|
| FST_353M has some files trained with old version resume_new |
| FST_1_3B is up to date resume |
| Transformer_1_3B is up to date resume |
| Transformer_353M is up to date resume |
| |
| 2, |
| |
| /work/jf381/code/lm-research/scripts/training/train_medium_bash_resume_transformer_bert_prediction.sh |
| |
| For Bert: |
| we will start from 2 gpu version of ar model trained on slim-6B |
| We have bert tokenizer: need to change |
| bert_2_gpu_transformer |
| bert_2_gpu_fst |
| we will have sbatch version and no svatch version be careful |
| cp |
| cp |
| |
| huggingface-cli upload jasonfan/FST_code /work/jf381/code/lm-research |
