risenyard commited on
Commit
9e254ce
·
verified ·
1 Parent(s): 5afd0dd

Use the published GitHub and Dataset paths

Browse files
Files changed (1) hide show
  1. README.md +16 -6
README.md CHANGED
@@ -69,7 +69,8 @@ a forecasting result.
69
 
70
  ## Use
71
 
72
- Install the code and download the checkpoint:
 
73
 
74
  ```bash
75
  git clone https://github.com/risenyard/egms-qa
@@ -78,21 +79,30 @@ pip install -e .
78
 
79
  hf download risenyard/egms-qa-encoder \
80
  --local-dir data/encoder/checkpoint
 
 
 
 
 
 
81
  ```
82
 
83
- Extract tokens from an already prepared compatible NPZ tile collection:
84
 
85
  ```bash
86
  python -m egms_encoder.extract_tokens \
87
  --checkpoint data/encoder/checkpoint/encoder.safetensors \
88
  --model-config data/encoder/checkpoint/config.json \
89
- --manifest path/to/split.parquet \
90
- --data-config path/to/data_config.json \
 
91
  --output-dir outputs/tokens
92
  ```
93
 
94
- The manifest and data config describe the user's tiles; they are not properties
95
- of the checkpoint. In the released dataset, stored `[0,294)` is bitwise equal
 
 
96
  to `[8,302)` on the original 304-step prepared axis; the data config retains
97
  the original offset for physical-time calculations. The released training data
98
  and reference manifest are available separately in
 
69
 
70
  ## Use
71
 
72
+ Install the published code, then download the current Encoder and Dataset
73
+ trees into the runtime locations used by that code:
74
 
75
  ```bash
76
  git clone https://github.com/risenyard/egms-qa
 
79
 
80
  hf download risenyard/egms-qa-encoder \
81
  --local-dir data/encoder/checkpoint
82
+ hf download risenyard/egms-qa-dataset \
83
+ --repo-type dataset --local-dir release/egms-qa-dataset
84
+ python -m egms_qa.release audit \
85
+ --release-dir release/egms-qa-dataset
86
+ python -m egms_qa.release install \
87
+ --release-dir release/egms-qa-dataset --target-root .
88
  ```
89
 
90
+ Extract tokens from the released tiles:
91
 
92
  ```bash
93
  python -m egms_encoder.extract_tokens \
94
  --checkpoint data/encoder/checkpoint/encoder.safetensors \
95
  --model-config data/encoder/checkpoint/config.json \
96
+ --normalization data/encoder/checkpoint/normalization.json \
97
+ --manifest data/encoder/manifest/split.parquet \
98
+ --data-config data/encoder/manifest/data_config.json \
99
  --output-dir outputs/tokens
100
  ```
101
 
102
+ For another compatible NPZ collection, replace only `--manifest` and
103
+ `--data-config` with that collection's contract paths. These files describe the
104
+ tiles and are not properties of the checkpoint. In the released dataset,
105
+ stored `[0,294)` is bitwise equal
106
  to `[8,302)` on the original 304-step prepared axis; the data config retains
107
  the original offset for physical-time calculations. The released training data
108
  and reference manifest are available separately in