Transformers
Safetensors
patchtst
time-series
remote-sensing
earth-observation
self-supervised-learning
masked-pretraining
open-weights
Instructions to use AminiTech/amini-fm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AminiTech/amini-fm with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, PatchTSTForPretraining tokenizer = AutoTokenizer.from_pretrained("AminiTech/amini-fm") model = PatchTSTForPretraining.from_pretrained("AminiTech/amini-fm", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update model card with architecture, usage, and research context
Browse files
README.md
CHANGED
|
@@ -8,12 +8,13 @@ tags:
|
|
| 8 |
- earth-observation
|
| 9 |
- self-supervised-learning
|
| 10 |
- masked-pretraining
|
|
|
|
| 11 |
inference: false
|
| 12 |
---
|
| 13 |
|
| 14 |
# AminiTech/amini-fm
|
| 15 |
|
| 16 |
-
Amini FM is
|
| 17 |
|
| 18 |
The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
|
| 19 |
|
|
@@ -53,15 +54,15 @@ Classification, regression, and forecasting require an appropriate downstream mo
|
|
| 53 |
|
| 54 |
Provide finite float32 values with shape `[batch_size, 48, 10]`, preserving chronological order and consistent channel definitions.
|
| 55 |
|
| 56 |
-
|
| 57 |
|
| 58 |
Use the channel order, feature definitions, normalization, and missing-data handling associated with the training pipeline for your application. The model applies internal mean scaling.
|
| 59 |
|
| 60 |
## Training and related research
|
| 61 |
|
| 62 |
-
The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.
|
| 63 |
|
| 64 |
-
Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study
|
| 65 |
|
| 66 |
## Usage
|
| 67 |
|
|
@@ -126,13 +127,12 @@ The pretraining class applies random input masking even in evaluation mode. The
|
|
| 126 |
|
| 127 |
## Evaluation and limitations
|
| 128 |
|
| 129 |
-
|
| 130 |
|
| 131 |
- Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
|
| 132 |
- Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
|
| 133 |
- Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
|
| 134 |
- Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
|
| 135 |
-
- Exact training data provenance and preprocessing are incompletely documented for this revision.
|
| 136 |
|
| 137 |
## License
|
| 138 |
|
|
|
|
| 8 |
- earth-observation
|
| 9 |
- self-supervised-learning
|
| 10 |
- masked-pretraining
|
| 11 |
+
- open-weights
|
| 12 |
inference: false
|
| 13 |
---
|
| 14 |
|
| 15 |
# AminiTech/amini-fm
|
| 16 |
|
| 17 |
+
Amini FM is an open-weights transformer model for learning representations of satellite pixel time series. It accepts 48 time positions and 10 numerical channels per pixel. Its six-layer encoder uses temporal attention and cross-channel attention, with a hidden width of 512.
|
| 18 |
|
| 19 |
The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
|
| 20 |
|
|
|
|
| 54 |
|
| 55 |
Provide finite float32 values with shape `[batch_size, 48, 10]`, preserving chronological order and consistent channel definitions.
|
| 56 |
|
| 57 |
+
Each input channel should have a consistent feature definition and position. Temporal resampling should produce 48 positions per sequence.
|
| 58 |
|
| 59 |
Use the channel order, feature definitions, normalization, and missing-data handling associated with the training pipeline for your application. The model applies internal mean scaling.
|
| 60 |
|
| 61 |
## Training and related research
|
| 62 |
|
| 63 |
+
The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches.
|
| 64 |
|
| 65 |
+
Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study presents a compact channel-independent model; this larger release adds cross-channel attention. Its downstream performance should be evaluated for the intended task. [ICLR 2025 ML4RS paper](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf)
|
| 66 |
|
| 67 |
## Usage
|
| 68 |
|
|
|
|
| 127 |
|
| 128 |
## Evaluation and limitations
|
| 129 |
|
| 130 |
+
The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.
|
| 131 |
|
| 132 |
- Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
|
| 133 |
- Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
|
| 134 |
- Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
|
| 135 |
- Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
|
|
|
|
| 136 |
|
| 137 |
## License
|
| 138 |
|