Hugging Face's logo

hulehule
/
pllm2-full-dump

Safetensors
Model card Files
xet
 Community
pllm2-full-dump / notes /progress.md
yutong.song
Full PLLM2.0 dump
4b1637c
preview code
|
Raw History Blame Contribute Delete
2.76 kB

July 8:

  • link to HDFS
  • HDFS address: /mnt/hdfs/pllm/yutong
  • uploaded two benchmarks
  • downloaded the long_lamp datasets

HDFS删除文件： hadoop fs -rm -r /hdfs/文件夹/路径 删除单个文件： hadoop fs -rm /hdfs/文件/路径

baseline training for abtract genration:

PYTHONPATH=. python3 train_llm.py
--train_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--validation_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--test_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_test.json
--model_name google/flan-t5-base
--task generation_abstract
--output_dir /mnt/hdfs/pllm/yutong/long_output

#多卡训练 torchrun --nproc_per_node=4 --master_port=29502 train_llm.py
--train_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--validation_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--test_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_test.json
--model_name google/flan-t5-base
--task generation_abstract
--output_dir /mnt/hdfs/pllm/yutong/long_output

#改成llama 3.1 8B 版本 PYTHONPATH=. python3 train_llama.py
--train_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--validation_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_train.json
--test_data /mnt/hdfs/pllm/yutong/long_lamp/abstract_test.json
--model_name meta-llama/Meta-Llama-3.1-8B
--task generation_abstract
--output_dir /mnt/hdfs/pllm/yutong/long_output_llama

pip install huggingface_hub huggingface-cli login huggingface token: https://llama3-1.llamameta.net/*?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiNWUwOGc5MGRmY3hxcWN3eG5pOWtod2hxIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvbGxhbWEzLTEubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MjE5NDMxMX19fV19&Signature=JYdafzZYXEVmrFyp3DASI%7EdmID5eLnR%7EW0UonzxCEEdKymy6U2UYvHfA5xLIWY1oupVhRtLhVmy6Ro1UdfFYDLKGVGUed3piEbFMSfAj4jsycBXx9dHoAG4cwtq6kjMl50lNnD4PCGBV3IKO3hHyCSJ8GTYNaROGo9u2qOr6%7E--iCAI8jGv4jKLHjcR4flR9omhkMLxD7-g9L96PN8wEEkupayDmqCXk4lZ1KOX-h%7EityoB4m1-J5Y-zZVNUBti0My3hxrPTNn79Zs8v8-bORN1384OY3LgwMVK3CBTewUODGa6L4HKvRdY6mgATk9x06Gw-CdWLcoiJczpG44r4EA__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=24251018147862817

HDFS 删除文件： hdfs dfs -rm -r harunava/home/byte_tiktok_content_ecology/user/yutong /mnt/hdfs/pllm/abstract_train.json hdfs dfs -rm /mnt/hdfs/pllm/abstract_test.json

HDFS 删除文件： hdfs dfs -rm -r harunava/home/byte_tiktok_content_ecology/user/yutong /mnt/hdfs/pllm/abstract_train.json hdfs dfs -rm /mnt/hdfs/pllm/abstract_test.json

July 16:

  • baseline model using Llama 3.1
  • model 2: RL for retrieval refinement

July 17:

  • 重新下载lamp数据集
  • run long- writer在long_lamp数据集上