amini-ai commited on
Commit
a570cdc
·
verified ·
1 Parent(s): d7ac02b

Update model card with architecture, usage, and research context

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -8,12 +8,13 @@ tags:
8
  - earth-observation
9
  - self-supervised-learning
10
  - masked-pretraining
 
11
  inference: false
12
  ---
13
 
14
  # AminiTech/amini-fm
15
 
16
- Amini FM is a transformer checkpoint for learning representations of satellite pixel time series. It accepts 48 time positions and 10 numerical channels per pixel. Its six-layer encoder uses temporal attention and cross-channel attention, with a hidden width of 512.
17
 
18
  The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
19
 
@@ -53,15 +54,15 @@ Classification, regression, and forecasting require an appropriate downstream mo
53
 
54
  Provide finite float32 values with shape `[batch_size, 48, 10]`, preserving chronological order and consistent channel definitions.
55
 
56
- The exported model specifies a 48-position context and 10 channels. A release-specific channel schema, normalization statistics, and temporal preprocessing manifest are not distributed with the checkpoint.
57
 
58
  Use the channel order, feature definitions, normalization, and missing-data handling associated with the training pipeline for your application. The model applies internal mean scaling.
59
 
60
  ## Training and related research
61
 
62
- The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches. The released artifacts do not include a dataset manifest linking this revision to a particular corpus and split.
63
 
64
- Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study describes a compact channel-independent model. This release enables cross-channel attention; the study does not establish performance for this larger checkpoint. [ICLR 2025 ML4RS paper](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf)
65
 
66
  ## Usage
67
 
@@ -126,13 +127,12 @@ The pretraining class applies random input masking even in evaluation mode. The
126
 
127
  ## Evaluation and limitations
128
 
129
- Release-specific numerical benchmarks are not reported in this card. The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.
130
 
131
  - Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
132
  - Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
133
  - Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
134
  - Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
135
- - Exact training data provenance and preprocessing are incompletely documented for this revision.
136
 
137
  ## License
138
 
 
8
  - earth-observation
9
  - self-supervised-learning
10
  - masked-pretraining
11
+ - open-weights
12
  inference: false
13
  ---
14
 
15
  # AminiTech/amini-fm
16
 
17
+ Amini FM is an open-weights transformer model for learning representations of satellite pixel time series. It accepts 48 time positions and 10 numerical channels per pixel. Its six-layer encoder uses temporal attention and cross-channel attention, with a hidden width of 512.
18
 
19
  The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
20
 
 
54
 
55
  Provide finite float32 values with shape `[batch_size, 48, 10]`, preserving chronological order and consistent channel definitions.
56
 
57
+ Each input channel should have a consistent feature definition and position. Temporal resampling should produce 48 positions per sequence.
58
 
59
  Use the channel order, feature definitions, normalization, and missing-data handling associated with the training pipeline for your application. The model applies internal mean scaling.
60
 
61
  ## Training and related research
62
 
63
+ The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.
64
 
65
+ Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study presents a compact channel-independent model; this larger release adds cross-channel attention. Its downstream performance should be evaluated for the intended task. [ICLR 2025 ML4RS paper](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf)
66
 
67
  ## Usage
68
 
 
127
 
128
  ## Evaluation and limitations
129
 
130
+ The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.
131
 
132
  - Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
133
  - Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
134
  - Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
135
  - Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
 
136
 
137
  ## License
138