File size: 638 Bytes
8b3f571
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# AnyMo Project

# Dependencies and Installation
```bash
pip install -r requirements.txt
pip install sentence_transformers==5.3.0

# 多机训练
apt install perftest infiniband-diags libibverbs-dev ibutils rdma-core ibverbs-utils -y
pip install deepspeed
```

Download Pre-trained Models and put these models under `checkpoints` folder. 

```text
checkpoints/
├── distilbert-base-uncased
├── flan-t5-xl
└── sentence-transformers-paraphrase-MiniLM-L6-v2
```

# Prepare Data


tar -cf - work_dirs/ \
| pv \
| pigz -p 64 \
| split -b 16G - work_dirs.tar.gz.part_

tar -cf - work_dirs | pv | pigz -p 24 > work_dirs.tar.gz