Update model card with architecture, usage, and research context

README.md

@@ -1,3 +1,144 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+library_name: transformers
+tags:
+  - patchtst
+  - time-series
+  - remote-sensing
+  - earth-observation
+  - self-supervised-learning
+  - masked-pretraining
+inference: false
+---
+
+# AminiTech/amini-600k-v1
+
+Amini 600K V1 is a compact transformer checkpoint for learning representations of satellite pixel time series. It accepts 48 time positions and 10 numerical channels per pixel, with shared encoder weights across channels.
+
+The model is part of Amini's work on self-supervised representation learning for Earth observation. It adapts PatchTST to numerical spectral-band and index sequences, learning through reconstruction of randomly masked temporal patches. The released checkpoint contains an encoder and a patch reconstruction head.
+
+## Model details
+
+| Property | Value |
+| --- | --- |
+| Developer | AminiTech / Amini.ai |
+| Architecture | `PatchTSTForPretraining` |
+| Model size | Approximately 0.6M |
+| Input shape | `[batch_size, 48, 10]` |
+| Encoder layers | 3 |
+| Hidden width | 128 |
+| Attention heads | 4 |
+| Feed-forward width | 512 |
+| Patch length / stride | 12 / 12 |
+| Cross-channel attention | Disabled |
+| Normalization | `batchnorm` |
+| Internal scaling | `std` |
+| Masking | Random patches; configured ratio 0.4 |
+| Reconstruction objective | Mean squared error |
+| CLS token | Enabled |
+| Weight format / precision | Safetensors / float32 |
+| Exported Transformers version | `4.51.3` |
+
+Architecture details refer to [this checkpoint configuration](https://huggingface.co/AminiTech/amini-600k-v1/blob/870d71be731c46e97916f8f63f7c164a305cff9e/config.json). Context length counts positions after preprocessing; it does not specify a calendar duration.
+
+## Uses
+
+- Generate pixel time-series embeddings for similarity search and clustering.
+- Train downstream land-use/land-cover classifiers with labeled data.
+- Explore temporal patterns in agricultural and environmental data.
+
+Classification, regression, and forecasting require an appropriate downstream model and task-specific evaluation. The reconstruction head does not produce land-cover labels or future forecasts. The `prediction_length` field in the configuration does not establish a trained forecasting capability.
+
+## Input and preprocessing
+
+Provide finite float32 values with shape `[batch_size, 48, 10]`, preserving chronological order and consistent channel definitions.
+
+The exported model specifies a 48-position context and 10 channels. A release-specific channel schema, normalization statistics, and temporal preprocessing manifest are not distributed with the checkpoint.
+
+Use the channel order, feature definitions, normalization, and missing-data handling associated with the training pipeline for your application. The model applies internal standard-deviation scaling.
+
+## Training and related research
+
+The exported configuration records self-supervised random patch masking and MSE reconstruction. The mask ratio is a configuration value; the realized fraction depends on the discrete number of patches. The released artifacts do not include a dataset manifest linking this revision to a particular corpus and split.
+
+Related research describes Sentinel-2 median biweekly composites from 2020-2024 and 50,000 chips across Africa. The companion study describes the compact model family. It does not specify the exact Hub revision used in its experiments. [ICLR 2025 ML4RS paper](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf)
+
+## Usage
+
+Install PyTorch, Transformers with PatchTST support, and Safetensors. The example below uses synthetic input to demonstrate the API. Real inputs require the preprocessing described above.
+
+### Extract embeddings
+
+```python
+import torch
+from transformers import PatchTSTModel
+
+MODEL_ID = "AminiTech/amini-600k-v1"
+REVISION = "870d71be731c46e97916f8f63f7c164a305cff9e"
+
+encoder = PatchTSTModel.from_pretrained(
+    MODEL_ID,
+    revision=REVISION,
+    use_safetensors=True,
+    do_mask_input=False,
+).eval()
+
+past_values = torch.randn(
+    1, encoder.config.context_length, encoder.config.num_input_channels,
+    dtype=torch.float32,
+)
+observed_mask = torch.ones_like(past_values, dtype=torch.bool)
+
+with torch.inference_mode():
+    hidden = encoder(
+        past_values=past_values,
+        past_observed_mask=observed_mask,
+    ).last_hidden_state
+
+    # Shape: [batch, channels, patch_tokens_including_cls, hidden_width]
+    patch_hidden = hidden[:, :, 1:, :]  # Exclude the prepended CLS token.
+    per_channel_embeddings = patch_hidden.mean(dim=2)
+    pixel_embeddings = per_channel_embeddings.mean(dim=1)
+
+print(pixel_embeddings.shape)  # torch.Size([1, 128])
+```
+
+Loading the bare encoder omits the reconstruction head and may produce an unused-weight warning. This example averages patch embeddings and then channels; pooling should match the downstream training protocol. Channel-level or CLS-token embeddings can also be used.
+
+### Run masked reconstruction
+
+```python
+from transformers import PatchTSTForPretraining
+
+model = PatchTSTForPretraining.from_pretrained(
+    MODEL_ID, revision=REVISION, use_safetensors=True,
+).eval()
+
+with torch.inference_mode():
+    outputs = model(
+        past_values=past_values,
+        past_observed_mask=observed_mask,
+    )
+    reconstruction_loss = outputs.loss
+```
+
+The pretraining class applies random input masking even in evaluation mode. The encoder example explicitly disables masking for repeatable embeddings. Missing values must be represented consistently with the applicable preprocessing; the all-ones mask above is for fully populated synthetic input.
+
+## Evaluation and limitations
+
+Release-specific numerical benchmarks are not reported in this card. The usage examples have been checked with the released weights on CPU using PyTorch 2.6.0 and Transformers 4.48.3; this checks loading and forward-pass behavior, not downstream accuracy.
+
+- Clouds, missing observations, interpolation, sensor differences, and feature scaling can affect embeddings.
+- Transfer across regions, seasons, sensors, and acquisition conditions requires evaluation on representative data.
+- Random pixel splits can contain spatially related examples; geographic and temporal holdouts are useful when assessing generalization.
+- Reconstruction loss alone does not establish downstream classification, crop-yield prediction, calibrated change detection, or causal relationships.
+- Exact training data provenance and preprocessing are incompletely documented for this revision.
+
+## License
+
+This checkpoint is released under the Apache License 2.0, as declared in its repository metadata.
+
+## References
+
+- [Self-Supervised Representation Learning on Remote Sensing Pixel Time Series with Patch-Based Masking](https://ml-for-rs.github.io/iclr2025/camera_ready/papers/52.pdf), ICLR 2025 ML4RS Workshop.
+- [Hugging Face PatchTST documentation](https://huggingface.co/docs/transformers/model_doc/patchtst).