# AnyMo Project

# Dependencies and Installation
```bash
pip install -r requirements.txt
pip install sentence_transformers==5.3.0

# 多机训练
apt install perftest infiniband-diags libibverbs-dev ibutils rdma-core ibverbs-utils -y
pip install deepspeed
```

Download Pre-trained Models and put these models under `checkpoints` folder. 

```text
checkpoints/
├── distilbert-base-uncased
├── flan-t5-xl
└── sentence-transformers-paraphrase-MiniLM-L6-v2
```

# Prepare Data


tar -cf - work_dirs/ \
| pv \
| pigz -p 64 \
| split -b 16G - work_dirs.tar.gz.part_

tar -cf - work_dirs | pv | pigz -p 24 > work_dirs.tar.gz