risenyard commited on
Commit
3f92010
·
verified ·
1 Parent(s): 03afb27

Organize the model guide around use, inputs, and reproduction

Browse files
Files changed (1) hide show
  1. README.md +14 -10
README.md CHANGED
@@ -73,20 +73,24 @@ configuration and training recipe:
73
  ```bash
74
  hf download risenyard/egms-qa-dataset --repo-type dataset \
75
  --local-dir release/egms-qa-dataset
76
- python -m egms_qa.release install \
77
  --release-dir release/egms-qa-dataset --target-root .
78
  hf download risenyard/egms-qa-encoder --include '*.json' \
79
- --local-dir release/egms-qa-encoder
80
- python -m egms_qa.reproduce encoder \
81
- --model-dir release/egms-qa-encoder --output-dir outputs/encoder_pretrain
82
  ```
83
 
84
- The runner starts training from scratch using `config.json`,
85
- `training_args.json`, and the train-fitted `normalization.json`. These files
86
- specify the architecture, masking, sampling, loss, optimization, and validation
87
- settings. Add `--dry-run` to inspect the resolved command. Checkpoint selection
88
- uses validation loss, and numerical reproduction depends on the software,
89
- hardware, and random state.
 
 
 
 
90
 
91
  ## Evaluation
92
 
 
73
  ```bash
74
  hf download risenyard/egms-qa-dataset --repo-type dataset \
75
  --local-dir release/egms-qa-dataset
76
+ python -m egms_encoder.install_data \
77
  --release-dir release/egms-qa-dataset --target-root .
78
  hf download risenyard/egms-qa-encoder --include '*.json' \
79
+ --local-dir data/encoder/checkpoint
80
+ python -m egms_encoder.pretrain \
81
+ --output-dir outputs/my_encoder --device cuda:0
82
  ```
83
 
84
+ Training reads `config.json`, `training_args.json`, and the train-fitted
85
+ `normalization.json` from `data/encoder/checkpoint/`. These files specify the
86
+ architecture, masking, sampling, loss, optimization, and validation settings.
87
+ The output includes `best.safetensors` for inference, `latest.pt` for resuming,
88
+ and the matching configuration and normalization files. The
89
+ [Encoder guide](https://github.com/risenyard/egms-qa/blob/main/src/egms_encoder/README.md)
90
+ provides resume and token-extraction commands for the trained model.
91
+
92
+ Checkpoint selection uses validation loss. Numerical reproduction depends on
93
+ the software, hardware, and random state.
94
 
95
  ## Evaluation
96