Transformers
Safetensors
patchtst
time-series
remote-sensing
earth-observation
self-supervised-learning
masked-pretraining
open-weights
Instructions to use AminiTech/Amini_FM_V0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AminiTech/Amini_FM_V0 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, PatchTSTForPretraining tokenizer = AutoTokenizer.from_pretrained("AminiTech/Amini_FM_V0") model = PatchTSTForPretraining.from_pretrained("AminiTech/Amini_FM_V0", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update model card with architecture, usage, and research context
Browse files
README.md
CHANGED
|
@@ -7,12 +7,13 @@ tags:
|
|
| 7 |
- earth-observation
|
| 8 |
- self-supervised-learning
|
| 9 |
- masked-pretraining
|
|
|
|
| 10 |
inference: false
|
| 11 |
---
|
| 12 |
|
| 13 |
# AminiTech/Amini_FM_V0
|
| 14 |
|
| 15 |
-
Amini FM V0 is a compact transformer
|
| 16 |
|
| 17 |
The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
|
| 18 |
|
|
@@ -63,13 +64,13 @@ features = [
|
|
| 63 |
|
| 64 |
It sorts observations chronologically, groups them by pixel, standardizes each pixel's channels with `StandardScaler`, and linearly interpolates each sequence to 512 positions. This external preprocessing is followed by the model's internal standard-deviation scaling.
|
| 65 |
|
| 66 |
-
|
| 67 |
|
| 68 |
## Training and related research
|
| 69 |
|
| 70 |
-
The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.
|
| 71 |
|
| 72 |
-
Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study
|
| 73 |
|
| 74 |
## Usage
|
| 75 |
|
|
@@ -134,13 +135,12 @@ The pretraining class applies random input masking even in evaluation mode. The
|
|
| 134 |
|
| 135 |
## Evaluation and limitations
|
| 136 |
|
| 137 |
-
|
| 138 |
|
| 139 |
- Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
|
| 140 |
- Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
|
| 141 |
- Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
|
| 142 |
- Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
|
| 143 |
-
- Exact training data provenance and preprocessing are incompletely documented for this revision.
|
| 144 |
|
| 145 |
## License
|
| 146 |
|
|
|
|
| 7 |
- earth-observation
|
| 8 |
- self-supervised-learning
|
| 9 |
- masked-pretraining
|
| 10 |
+
- open-weights
|
| 11 |
inference: false
|
| 12 |
---
|
| 13 |
|
| 14 |
# AminiTech/Amini_FM_V0
|
| 15 |
|
| 16 |
+
Amini FM V0 is a compact open-weights transformer model for learning representations of satellite pixel time series. It accepts 512 time positions and 10 numerical channels per pixel.
|
| 17 |
|
| 18 |
The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
|
| 19 |
|
|
|
|
| 64 |
|
| 65 |
It sorts observations chronologically, groups them by pixel, standardizes each pixel's channels with `StandardScaler`, and linearly interpolates each sequence to 512 positions. This external preprocessing is followed by the model's internal standard-deviation scaling.
|
| 66 |
|
| 67 |
+
Keep feature definitions, channel order, and scaling consistent when adapting this preprocessing recipe to your data.
|
| 68 |
|
| 69 |
## Training and related research
|
| 70 |
|
| 71 |
+
The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.
|
| 72 |
|
| 73 |
+
Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study presents experiments on the compact model family. [ICLR 2025 ML4RS paper](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf)
|
| 74 |
|
| 75 |
## Usage
|
| 76 |
|
|
|
|
| 135 |
|
| 136 |
## Evaluation and limitations
|
| 137 |
|
| 138 |
+
The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.
|
| 139 |
|
| 140 |
- Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
|
| 141 |
- Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
|
| 142 |
- Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
|
| 143 |
- Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
|
|
|
|
| 144 |
|
| 145 |
## License
|
| 146 |
|