Organize the model guide around use, inputs, and reproduction
Browse files
README.md
CHANGED
|
@@ -73,20 +73,24 @@ configuration and training recipe:
|
|
| 73 |
```bash
|
| 74 |
hf download risenyard/egms-qa-dataset --repo-type dataset \
|
| 75 |
--local-dir release/egms-qa-dataset
|
| 76 |
-
python -m
|
| 77 |
--release-dir release/egms-qa-dataset --target-root .
|
| 78 |
hf download risenyard/egms-qa-encoder --include '*.json' \
|
| 79 |
-
--local-dir
|
| 80 |
-
python -m
|
| 81 |
-
--
|
| 82 |
```
|
| 83 |
|
| 84 |
-
|
| 85 |
-
`
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
## Evaluation
|
| 92 |
|
|
|
|
| 73 |
```bash
|
| 74 |
hf download risenyard/egms-qa-dataset --repo-type dataset \
|
| 75 |
--local-dir release/egms-qa-dataset
|
| 76 |
+
python -m egms_encoder.install_data \
|
| 77 |
--release-dir release/egms-qa-dataset --target-root .
|
| 78 |
hf download risenyard/egms-qa-encoder --include '*.json' \
|
| 79 |
+
--local-dir data/encoder/checkpoint
|
| 80 |
+
python -m egms_encoder.pretrain \
|
| 81 |
+
--output-dir outputs/my_encoder --device cuda:0
|
| 82 |
```
|
| 83 |
|
| 84 |
+
Training reads `config.json`, `training_args.json`, and the train-fitted
|
| 85 |
+
`normalization.json` from `data/encoder/checkpoint/`. These files specify the
|
| 86 |
+
architecture, masking, sampling, loss, optimization, and validation settings.
|
| 87 |
+
The output includes `best.safetensors` for inference, `latest.pt` for resuming,
|
| 88 |
+
and the matching configuration and normalization files. The
|
| 89 |
+
[Encoder guide](https://github.com/risenyard/egms-qa/blob/main/src/egms_encoder/README.md)
|
| 90 |
+
provides resume and token-extraction commands for the trained model.
|
| 91 |
+
|
| 92 |
+
Checkpoint selection uses validation loss. Numerical reproduction depends on
|
| 93 |
+
the software, hardware, and random state.
|
| 94 |
|
| 95 |
## Evaluation
|
| 96 |
|