risenyard commited on
Commit
f75a631
·
verified ·
1 Parent(s): 5c2ef80

Add masked-reconstruction evaluation results

Browse files
Files changed (2) hide show
  1. README.md +21 -0
  2. eval_results.json +30 -0
README.md CHANGED
@@ -40,12 +40,33 @@ weights remain frozen when training the language-model translator.
40
  | `config.json` | architecture and input contract used to construct the encoder |
41
  | `normalization.json` | input mean, standard deviation, and residual scale |
42
  | `training_args.json` | optional pretraining recipe and checkpoint-selection record |
 
43
 
44
  `encoder.safetensors`, `config.json`, and `normalization.json` are sufficient
45
  for inference. `training_args.json` is provenance for training reproduction;
46
  the optimizer and mixed-precision scaler are not distributed. EGMS
47
  measurements are not included in this model repository.
48
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
  ## Use
50
 
51
  Install the code and download the checkpoint:
 
40
  | `config.json` | architecture and input contract used to construct the encoder |
41
  | `normalization.json` | input mean, standard deviation, and residual scale |
42
  | `training_args.json` | optional pretraining recipe and checkpoint-selection record |
43
+ | `eval_results.json` | machine-readable masked-reconstruction evaluation |
44
 
45
  `encoder.safetensors`, `config.json`, and `normalization.json` are sufficient
46
  for inference. `training_args.json` is provenance for training reproduction;
47
  the optimizer and mixed-precision scaler are not distributed. EGMS
48
  measurements are not included in this model repository.
49
 
50
+ ## Evaluation
51
+
52
+ The released checkpoint was evaluated on 1,000 held-out tiles containing
53
+ 2,047,451 point histories. The same central 88-step interval, 30% of the
54
+ 294-step input, was masked for every point in a tile.
55
+
56
+ | metric | value |
57
+ |---|---:|
58
+ | normalized MSE | 0.0702 |
59
+ | MSE | 2.279 mm² |
60
+ | RMSE | 1.510 mm |
61
+ | MAE | 1.007 mm |
62
+ | pooled EGMS residual | 1.433 mm |
63
+ | per-point RMSE P10 / P50 / P90 | 0.54 / 1.03 / 2.43 mm |
64
+
65
+ The masked-reconstruction RMSE is close to the pooled residual level of the
66
+ source observations. This result indicates that the frozen representation
67
+ retains the dominant temporal structure under this masking protocol. It is not
68
+ a forecasting result.
69
+
70
  ## Use
71
 
72
  Install the code and download the checkpoint:
eval_results.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema_version": "egms-qa-encoder-evaluation-1.0",
3
+ "evaluation": "masked_reconstruction",
4
+ "dataset": "risenyard/egms-qa-dataset",
5
+ "split": "test",
6
+ "sample": {
7
+ "tiles": 1000,
8
+ "point_histories": 2047451,
9
+ "input_steps": 294
10
+ },
11
+ "masking": {
12
+ "strategy": "central_synchronized_block",
13
+ "masked_steps": 88,
14
+ "masked_fraction": 0.3
15
+ },
16
+ "aggregation": "point_weighted_over_masked_observations",
17
+ "metrics": {
18
+ "normalized_mse": 0.0702,
19
+ "mse_mm2": 2.279,
20
+ "rmse_mm": 1.51,
21
+ "mae_mm": 1.007,
22
+ "pooled_egms_residual_mm": 1.433,
23
+ "per_point_rmse_mm": {
24
+ "p10": 0.54,
25
+ "p50": 1.03,
26
+ "p90": 2.43
27
+ }
28
+ },
29
+ "scope": "Masked reconstruction of held-out observations; not forecasting."
30
+ }