Release original SeisLM encoder with MIT license and verified portable weights

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Tianlin Liu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,100 @@
+---
+license: mit
+library_name: seislm
+pipeline_tag: feature-extraction
+tags:
+  - seismology
+  - seismic-waveforms
+  - self-supervised-learning
+  - arxiv:2410.15765
+---
+
+# SeisLM base
+
+Pretrained seismic waveform encoder from [SeisLM: a Foundation Model for
+Seismic Waveforms](https://arxiv.org/abs/2410.15765).
+Code and installation: https://github.com/liutianlin0121/seisLM
+
+This is an encoder with the original self-supervised pretraining heads. It
+does **not** include a trained phase-picking or earthquake-detection head.
+Downstream adaptation is needed for those tasks. It is not a Transformers
+AutoModel artifact or a hosted inference endpoint.
+
+## Architecture and provenance
+
+- Parameters (including pretraining heads): 11,359,568
+- Transformer layers: 6
+- Contextual feature dimension: 240
+- Original checkpoint SHA-256: `6e97c53c562e9ba6bc96df85d8d208b89ff84e80a25468825f29bb5bace5eacb`
+- Export: unchanged state tensors in safetensors plus the architecture
+  configuration embedded in the original checkpoint. Optimizer state omitted.
+- Training sources recorded in the original checkpoint: ETHZ, GEOFON, STEAD,
+  MLAAPDE, InstanceCounts, Iquique, PNW, OBST2024.
+
+## Input and output
+
+Inputs: float32 tensors `[batch, 3, samples]`, components **ZNE**, **100 Hz**.
+Reference window: 3001 samples. Demean each component, then divide by its
+population standard deviation plus `1e-10` (replace zero standard deviation
+by 1). No automatic resampling, filtering, or component filling is performed.
+See `preprocessing.json` for the contract and missing-component caveat.
+
+Contextual features have shape `[batch, reduced_time, 240]`.
+For 3001 input samples, reduced_time is 749. These are features, not P/S picks.
+
+## Loading
+
+A compatible Python wheel is included so this release can be used before
+the source changes are merged or published to PyPI. Install with Python 3.11/3.12:
+
+```bash
+python -m pip install "https://huggingface.co/ltlltl2006/seislm-base/resolve/main/seislm-0.2.0-py3-none-any.whl"
+```
+
+For reproducible installation, replace `main` in the wheel URL with the reviewed
+model-repository commit. For GPU-specific PyTorch installation, use the
+[PyTorch selector](https://pytorch.org/get-started/locally/).
+
+```python
+import torch
+from seisLM import load_pretrained, encode
+
+model = load_pretrained("ltlltl2006/seislm-base")
+# Synthetic installation smoke example, not a scientific evaluation:
+x = torch.randn(1, 3, 3001, generator=torch.Generator().manual_seed(2024))
+x = x - x.mean(dim=-1, keepdim=True)
+std = x.std(dim=-1, correction=0, keepdim=True)
+x = x / (torch.where(std == 0, torch.ones_like(std), std) + 1e-10)
+features = encode(model, x)
+print(features.shape)
+```
+
+Set `revision` to a published model-repository commit for repeatable downloads.
+A local bundle directory also works offline.
+
+## Refactoring regression check
+
+The exported weights were compared with the original Lightning-loaded checkpoint
+on fixed complete/noise/zero-channel inputs using CPU and NVIDIA GB10 CUDA,
+PyTorch 2.14.1+cu130, Python 3.12.3. All eight checked fields (encoder features,
+convolutional features, projected/quantized features, codebook perplexity, and
+three masked-loss values) matched exactly, with maximum absolute error 0.
+This is numerical parity on a regression fixture, not a downstream benchmark.
+See the source repository's regression documentation for the protocol.
+
+## Evaluation and limitations
+
+The original paper evaluates fine-tuned task models, not this bare encoder.
+The paper's downstream scores cannot be inferred from successful loading or
+numerical parity of this artifact. Local adaptation and held-out evaluation
+are needed before relying on predictions for a new region or instrument.
+
+## License
+
+These pretrained weights are released under the MIT license, as confirmed by
+the author. See LICENSE. The code is also distributed under MIT.
+
+## Citation
+
+Tianlin Liu et al. (2024). SeisLM: a Foundation Model for Seismic Waveforms.
+arXiv:2410.15765. https://doi.org/10.48550/arXiv.2410.15765

config.json

@@ -0,0 +1,70 @@
+{
+  "format_version": 1,
+  "architecture": "MultiDimWav2Vec2ForPreTraining",
+  "model_config": {
+    "activation_dropout": 0.0,
+    "apply_spec_augment": true,
+    "attention_dropout": 0.0,
+    "codevector_dim": 256,
+    "contrastive_logits_temperature": 0.1,
+    "conv_bias": true,
+    "conv_dim": [
+      256,
+      256
+    ],
+    "conv_embed": true,
+    "conv_kernel": [
+      3,
+      3
+    ],
+    "conv_stride": [
+      2,
+      2
+    ],
+    "diversity_loss_weight": 0.1,
+    "do_stable_layer_norm": true,
+    "feat_extract_norm": "layer",
+    "feat_proj_dropout": 0.0,
+    "feat_quantizer_dropout": 0.0,
+    "hidden_dropout": 0.0,
+    "hidden_size": 240,
+    "initializer_range": 0.02,
+    "input_dim": 3,
+    "intermediate_size": 3072,
+    "layer_norm_eps": 1e-05,
+    "layerdrop": 0.0,
+    "mask_feature_length": 10,
+    "mask_feature_min_masks": 0,
+    "mask_feature_prob": 0.0,
+    "mask_time_length": 10,
+    "mask_time_min_masks": 2,
+    "mask_time_prob": 0.65,
+    "num_attention_heads": 12,
+    "num_codevector_groups": 2,
+    "num_codevectors_per_group": 320,
+    "num_conv_pos_embedding_groups": 16,
+    "num_conv_pos_embeddings": 128,
+    "num_feat_extract_layers": 2,
+    "num_hidden_layers": 6,
+    "num_negatives": 100,
+    "output_attentions": false,
+    "output_hidden_size": 768,
+    "output_hidden_states": false,
+    "pad_token_id": 0,
+    "proj_codevector_dim": 256,
+    "rotary_pos_embed": false,
+    "scale_logits_in_quantization": true,
+    "use_rms_norm": false
+  },
+  "preprocessing": {
+    "sampling_rate_hz": 100,
+    "component_order": "ZNE",
+    "tensor_layout": "batch, component, sample",
+    "dtype": "float32",
+    "reference_window_samples": 3001,
+    "normalization": "per-component demean, divide by population std + 1e-10",
+    "zero_std": "replace zero std by 1 before division",
+    "missing_components": "Legacy training randomly copies a present component into missing channels. Supply complete components for deterministic inference; no filling is performed by the loader or encode()."
+  },
+  "source_checkpoint_sha256": "6e97c53c562e9ba6bc96df85d8d208b89ff84e80a25468825f29bb5bace5eacb"
+}

manifest.json

@@ -0,0 +1,12 @@
+{
+  "model": "base",
+  "repo_id": "ltlltl2006/seislm-base",
+  "files": {
+    "LICENSE": "bbb6bbefbbd4b4fe7d56e7cf9fa3a591e22b35a064eb4a8342fecc64fc02ed11",
+    "README.md": "4d1f4568254ffc9e6130dbcfc5080ab913b9f14ec9058ba69032d3b344f41f58",
+    "config.json": "ddebdc2bdc5483b4c3f927fece90ed5b9984c8754a90c0d5aff181a37ac4820a",
+    "model.safetensors": "327b726ed779bab9284dc6a9023d36fa1822a07c6a0a33ed6b25953f17599ce6",
+    "preprocessing.json": "571abfda3ae0734b0b561d70c4a613ae432291b890b45019881c452c4a06689d",
+    "seislm-0.2.0-py3-none-any.whl": "c04f262c1bddd26a5b1628a62614042a6492f862697e0c4f1546d7c17c242681"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:327b726ed779bab9284dc6a9023d36fa1822a07c6a0a33ed6b25953f17599ce6
+size 45452496

preprocessing.json

@@ -0,0 +1,10 @@
+{
+  "sampling_rate_hz": 100,
+  "component_order": "ZNE",
+  "tensor_layout": "batch, component, sample",
+  "dtype": "float32",
+  "reference_window_samples": 3001,
+  "normalization": "per-component demean, divide by population std + 1e-10",
+  "zero_std": "replace zero std by 1 before division",
+  "missing_components": "Legacy training randomly copies a present component into missing channels. Supply complete components for deterministic inference; no filling is performed by the loader or encode()."
+}