Use the published GitHub and Dataset paths
Browse files
README.md
CHANGED
|
@@ -69,7 +69,8 @@ a forecasting result.
|
|
| 69 |
|
| 70 |
## Use
|
| 71 |
|
| 72 |
-
Install the code
|
|
|
|
| 73 |
|
| 74 |
```bash
|
| 75 |
git clone https://github.com/risenyard/egms-qa
|
|
@@ -78,21 +79,30 @@ pip install -e .
|
|
| 78 |
|
| 79 |
hf download risenyard/egms-qa-encoder \
|
| 80 |
--local-dir data/encoder/checkpoint
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
```
|
| 82 |
|
| 83 |
-
Extract tokens from
|
| 84 |
|
| 85 |
```bash
|
| 86 |
python -m egms_encoder.extract_tokens \
|
| 87 |
--checkpoint data/encoder/checkpoint/encoder.safetensors \
|
| 88 |
--model-config data/encoder/checkpoint/config.json \
|
| 89 |
-
--
|
| 90 |
-
--
|
|
|
|
| 91 |
--output-dir outputs/tokens
|
| 92 |
```
|
| 93 |
|
| 94 |
-
|
| 95 |
-
|
|
|
|
|
|
|
| 96 |
to `[8,302)` on the original 304-step prepared axis; the data config retains
|
| 97 |
the original offset for physical-time calculations. The released training data
|
| 98 |
and reference manifest are available separately in
|
|
|
|
| 69 |
|
| 70 |
## Use
|
| 71 |
|
| 72 |
+
Install the published code, then download the current Encoder and Dataset
|
| 73 |
+
trees into the runtime locations used by that code:
|
| 74 |
|
| 75 |
```bash
|
| 76 |
git clone https://github.com/risenyard/egms-qa
|
|
|
|
| 79 |
|
| 80 |
hf download risenyard/egms-qa-encoder \
|
| 81 |
--local-dir data/encoder/checkpoint
|
| 82 |
+
hf download risenyard/egms-qa-dataset \
|
| 83 |
+
--repo-type dataset --local-dir release/egms-qa-dataset
|
| 84 |
+
python -m egms_qa.release audit \
|
| 85 |
+
--release-dir release/egms-qa-dataset
|
| 86 |
+
python -m egms_qa.release install \
|
| 87 |
+
--release-dir release/egms-qa-dataset --target-root .
|
| 88 |
```
|
| 89 |
|
| 90 |
+
Extract tokens from the released tiles:
|
| 91 |
|
| 92 |
```bash
|
| 93 |
python -m egms_encoder.extract_tokens \
|
| 94 |
--checkpoint data/encoder/checkpoint/encoder.safetensors \
|
| 95 |
--model-config data/encoder/checkpoint/config.json \
|
| 96 |
+
--normalization data/encoder/checkpoint/normalization.json \
|
| 97 |
+
--manifest data/encoder/manifest/split.parquet \
|
| 98 |
+
--data-config data/encoder/manifest/data_config.json \
|
| 99 |
--output-dir outputs/tokens
|
| 100 |
```
|
| 101 |
|
| 102 |
+
For another compatible NPZ collection, replace only `--manifest` and
|
| 103 |
+
`--data-config` with that collection's contract paths. These files describe the
|
| 104 |
+
tiles and are not properties of the checkpoint. In the released dataset,
|
| 105 |
+
stored `[0,294)` is bitwise equal
|
| 106 |
to `[8,302)` on the original 304-step prepared axis; the data config retains
|
| 107 |
the original offset for physical-time calculations. The released training data
|
| 108 |
and reference manifest are available separately in
|