lhallee commited on
Commit
a5aeffd
·
verified ·
1 Parent(s): b3e55d2

Update FastPLMs files

Browse files
README.md CHANGED
@@ -12,7 +12,7 @@ tags:
12
 
13
  # Synthyra/Profluent-E1-300M
14
 
15
- This checkpoint packages the FastPLMs `E1` implementation.
16
 
17
  Accepted inputs are raw amino-acid sequences prepared by the native E1 adapter.
18
  Supported Transformers entry points are `AutoConfig`, `AutoModel`,
@@ -31,9 +31,7 @@ Supported Transformers entry points are `AutoConfig`, `AutoModel`,
31
  | Attention variants | Supported: `sdpa`, `flex_attention` |
32
  | Compliance | Declared: exact release evidence is required |
33
 
34
- A supported interface is not a pretrained downstream predictor. Classification
35
- heads start untrained, and declared compliance metadata is not a claim that an
36
- arbitrary local build passed its release gate.
37
 
38
  ## Install and platform requirements
39
 
@@ -44,12 +42,12 @@ python -m pip install -r \
44
  "https://huggingface.co/Synthyra/Profluent-E1-300M/resolve/main/requirements.txt"
45
  ```
46
 
47
- The FastPLMs implementation itself is embedded in the model repository and loaded
48
- by Transformers through `trust_remote_code=True`.
49
 
50
- Python 3.11-3.14, PyTorch 2.13, and Transformers 5.13 are required. The declared CPU gate covers tiny offline contracts; published checkpoint throughput and parity require the documented device tier. The Hub quick start below requires network
51
- access on first download. For an air-gapped run, first build the manifest-pinned
52
- local artifact and use the offline form shown in the example.
53
 
54
  ## Quick start
55
 
@@ -65,22 +63,22 @@ model = AutoModel.from_pretrained(
65
  ```
66
 
67
  For offline validation, replace `model_id` with the manifest-built
68
- `dist/hub/Profluent-E1-300M` path and pass `local_files_only=True`.
69
 
70
  ## Attention and compliance
71
 
72
  The quick start selects `sdpa` explicitly. Declared variants are `sdpa`, `flex_attention`. An unavailable requested
73
- backend raises instead of silently switching implementations.
74
- `output_attentions=True` may use the documented, one-call eager fallback solely
75
- to materialize attention tensors; the configured backend remains unchanged.
76
 
77
- This family declares the `compliance` tier. Release evidence binds the exact
78
  checkpoint, backend, dtype, hardware, inputs, and reference revision.
79
 
80
  ## Dataset embeddings
81
 
82
- The shared embedding mixin preserves input order and biological-position
83
- masking. It accepts sequences, identified records, mappings, or a FASTA path:
84
 
85
  ```python
86
  pooled = model.embed_dataset(
@@ -97,13 +95,13 @@ print(residues[0].tensor.shape) # (l, d)
97
  ```
98
 
99
  Set `output` and `format="safetensors"` or `"sqlite"` for transactional,
100
- bounded-memory persistence. Resume verifies input order, model state, tokenizer
101
- policy, backend, dtype, and pooling configuration before appending.
102
 
103
  ## Downstream classification
104
 
105
- Both downstream AutoClasses reuse the checkpoint backbone and initialize a new,
106
- untrained `classifier`. Sequence labels have shape `(b,)`; residue labels have
107
  shape `(b, l)` and use `-100` outside biological positions:
108
 
109
  ```python
@@ -140,7 +138,7 @@ print(token_output.logits.shape) # (b, l, 3)
140
 
141
  ## PEFT fine-tuning
142
 
143
- Install the direct training dependencies, then attach LoRA to the loaded checkpoint:
144
 
145
  ```bash
146
  python -m pip install "datasets>=4.8,<5" "peft>=0.19,<0.20"
@@ -161,17 +159,17 @@ peft_model = get_peft_model(
161
  )
162
  ```
163
 
164
- This checkpoint advertises a classification head, so the separately trained
165
- `classifier` is saved with the adapter.
166
  All FastPLMs checkpoints follow the Transformers `PreTrainedModel` contract and
167
- can be adapted with PEFT. The ESM2-specific shipped CLI is an example, not a
168
  support boundary. Record the target modules, base revision, data identity, and
169
  trainable parameter scope.
170
 
171
  ## Test-time training
172
 
173
  TTT samples masked views of one protein and updates only injected low-rank
174
- adapters. Base checkpoint weights remain frozen:
175
 
176
  ```python
177
  from transformers import AutoModelForMaskedLM
@@ -189,14 +187,14 @@ ttt_model.ttt_reset()
189
  print(metrics)
190
  ```
191
 
192
- Persisted adapters retain their deterministic reset state. TTT adds latency
193
- and memory, can worsen an output, and does not establish biological function.
194
 
195
  ## Tokenizer-free E1 input
196
 
197
- E1 has no tokenizer. The model retains native raw-sequence preparation,
198
- boundary tokens, sequence positions, and retrieval-augmented context behavior.
199
- The ordinary representation path accepts sequences directly:
200
 
201
  ```python
202
  result = model.embed_dataset(
@@ -207,9 +205,9 @@ result = model.embed_dataset(
207
  print(result[0].tensor.shape)
208
  ```
209
 
210
- Lower-level masked-language-model calls must use the E1 batch preparer rather
211
- than an `AutoTokenizer`. E1 launch messages and distributed legal files retain
212
- the attribution required by the upstream agreement.
213
 
214
  ## Runtime contract
215
 
@@ -229,8 +227,8 @@ the attribution required by the upstream agreement.
229
  ## Release record
230
 
231
  - FastPLMs weights: `Synthyra/Profluent-E1-300M`
232
- - Runtime revision: recorded separately in the built artifact and published commit
233
- - Source-tree and runtime-bundle SHA-256: recorded in `provenance.json`
234
  - Official checkpoint: `Profluent-Bio/E1-300m`
235
  - Artifact source: `fast`
236
  - State transform: `e1_to_fastplms_v1`
@@ -238,19 +236,17 @@ the attribution required by the upstream agreement.
238
  - Release tiers: `check`, `compliance`, `feature`, `artifact`, `benchmark`
239
  - Unresolved required file identities: `0`
240
 
241
- `provenance.json` records exact file identities, conversion, source revisions,
242
- legal texts, schema, and attestations. A nonzero unresolved count blocks release.
243
 
244
  ## Validation boundary
245
 
246
- Declared tiers compare applicable configuration, tokenizer behavior, state,
247
- and representative inference with the pinned reference. Metadata alone does
248
- not claim a build passed, a backend is faster, or an output is biologically
249
- valid.
250
 
251
  ## License
252
 
253
  Checkpoint terms: [Profluent-E1 Clickthrough License Agreement](https://github.com/Profluent-AI/E1/blob/main/LICENSE). The Hub model-card identifier is
254
- `other`. Applicable source licenses, notices, attribution,
255
- and conversion records are distributed with the local artifact. Review them
256
- before use.
 
12
 
13
  # Synthyra/Profluent-E1-300M
14
 
15
+ This checkpoint contains the FastPLMs `E1` implementation.
16
 
17
  Accepted inputs are raw amino-acid sequences prepared by the native E1 adapter.
18
  Supported Transformers entry points are `AutoConfig`, `AutoModel`,
 
31
  | Attention variants | Supported: `sdpa`, `flex_attention` |
32
  | Compliance | Declared: exact release evidence is required |
33
 
34
+ A supported interface is not a pretrained downstream predictor. Classification heads start untrained. Compliance metadata does not show that a local build passed its release gate.
 
 
35
 
36
  ## Install and platform requirements
37
 
 
42
  "https://huggingface.co/Synthyra/Profluent-E1-300M/resolve/main/requirements.txt"
43
  ```
44
 
45
+ The FastPLMs implementation itself is embedded in the model repository.
46
+ Transformers loads it through `trust_remote_code=True`.
47
 
48
+ This model requires Python 3.11-3.14, PyTorch 2.13, and Transformers 5.13. The CPU gate covers small offline tests. Published checkpoint throughput and parity require the documented device tier. The Hub quick start needs network access for
49
+ the first download. For an air-gapped run, build the manifest-pinned local
50
+ artifact first and use the offline example.
51
 
52
  ## Quick start
53
 
 
63
  ```
64
 
65
  For offline validation, replace `model_id` with the manifest-built
66
+ `dist/hub/Profluent-E1-300M` path. Pass `local_files_only=True`.
67
 
68
  ## Attention and compliance
69
 
70
  The quick start selects `sdpa` explicitly. Declared variants are `sdpa`, `flex_attention`. An unavailable requested
71
+ backend raises. It does not silently change implementation.
72
+ `output_attentions=True` can use the documented one-call eager fallback to
73
+ materialize attention tensors. The configured backend does not change.
74
 
75
+ This family declares the `compliance` tier. Release evidence identifies the
76
  checkpoint, backend, dtype, hardware, inputs, and reference revision.
77
 
78
  ## Dataset embeddings
79
 
80
+ The shared embedding mixin keeps input order and biological-position masking.
81
+ It accepts sequences, identified records, mappings, or a FASTA path:
82
 
83
  ```python
84
  pooled = model.embed_dataset(
 
95
  ```
96
 
97
  Set `output` and `format="safetensors"` or `"sqlite"` for transactional,
98
+ bounded-memory storage. Resume checks input order, model state, tokenizer
99
+ policy, backend, dtype, and pooling configuration before it appends data.
100
 
101
  ## Downstream classification
102
 
103
+ Both downstream AutoClasses use the checkpoint backbone and create a new,
104
+ untrained `classifier`. Sequence labels have shape `(b,)`. Residue labels have
105
  shape `(b, l)` and use `-100` outside biological positions:
106
 
107
  ```python
 
138
 
139
  ## PEFT fine-tuning
140
 
141
+ Install the training dependencies. Then attach LoRA to the loaded checkpoint:
142
 
143
  ```bash
144
  python -m pip install "datasets>=4.8,<5" "peft>=0.19,<0.20"
 
159
  )
160
  ```
161
 
162
+ This checkpoint advertises a classification head. Save the separately trained
163
+ `classifier` with the adapter.
164
  All FastPLMs checkpoints follow the Transformers `PreTrainedModel` contract and
165
+ can use PEFT. The ESM2-specific shipped CLI is an example, not a
166
  support boundary. Record the target modules, base revision, data identity, and
167
  trainable parameter scope.
168
 
169
  ## Test-time training
170
 
171
  TTT samples masked views of one protein and updates only injected low-rank
172
+ adapters. Base checkpoint weights stay frozen:
173
 
174
  ```python
175
  from transformers import AutoModelForMaskedLM
 
187
  print(metrics)
188
  ```
189
 
190
+ Saved adapters retain their deterministic reset state. TTT adds latency and
191
+ memory, can worsen an output, and does not show biological function.
192
 
193
  ## Tokenizer-free E1 input
194
 
195
+ E1 has no tokenizer. The model keeps native raw-sequence preparation, boundary
196
+ tokens, sequence positions, and retrieval-augmented context behavior. The
197
+ ordinary representation path accepts sequences directly:
198
 
199
  ```python
200
  result = model.embed_dataset(
 
205
  print(result[0].tensor.shape)
206
  ```
207
 
208
+ Lower-level masked-language-model calls must use the E1 batch preparer, not an
209
+ `AutoTokenizer`. E1 launch messages and distributed legal files keep the
210
+ attribution required by the upstream agreement.
211
 
212
  ## Runtime contract
213
 
 
227
  ## Release record
228
 
229
  - FastPLMs weights: `Synthyra/Profluent-E1-300M`
230
+ - Runtime revision: recorded in the built artifact and published commit
231
+ - Source-tree and runtime-bundle SHA-256: recorded in the source record
232
  - Official checkpoint: `Profluent-Bio/E1-300m`
233
  - Artifact source: `fast`
234
  - State transform: `e1_to_fastplms_v1`
 
236
  - Release tiers: `check`, `compliance`, `feature`, `artifact`, `benchmark`
237
  - Unresolved required file identities: `0`
238
 
239
+ The source record records exact file identities, conversion, source revisions,
240
+ legal texts, schema, and attestations. A nonzero unresolved count blocks a release.
241
 
242
  ## Validation boundary
243
 
244
+ Declared tiers compare configuration, tokenizer behavior, state, and
245
+ representative inference with the pinned reference. Metadata does not show that
246
+ a build passed, that a backend is faster, or that an output is biologically valid.
 
247
 
248
  ## License
249
 
250
  Checkpoint terms: [Profluent-E1 Clickthrough License Agreement](https://github.com/Profluent-AI/E1/blob/main/LICENSE). The Hub model-card identifier is
251
+ `other`. The local artifact contains applicable source
252
+ licenses, notices, attribution, and conversion records. Review them before use.
 
THIRD_PARTY_NOTICES.md CHANGED
@@ -46,7 +46,7 @@ explicitly defines the repository release as including pretrained DPLM1 and
46
  DPLM2 weights, and the same revision carries the complete
47
  [Apache-2.0 license](https://github.com/bytedance/dplm/blob/8a2e15e53416b4536f03f79ad1f6f6a9cbd5e19d/LICENSE).
48
  FastPLMs records both checkpoint families as Apache-2.0 and distributes the
49
- verbatim license plus `LICENSES/dplm/PROVENANCE.md`. Converted weights retain
50
  those terms and remain subject to the ordinary artifact and publication gates.
51
 
52
  ## Biohub
@@ -80,7 +80,7 @@ TorchMetrics, Lightning Utilities, and NVIDIA DLLogger. Their exact versions or
80
  revision are pinned in `docker/constraints/esmfold.txt`; OpenFold imports them
81
  eagerly, and FastPLMs production code does not depend on them. DLLogger's exact
82
  source identity and installed-license handling are recorded in
83
- `LICENSES/dllogger/PROVENANCE.md`.
84
 
85
  ## ProteinTTT
86
 
@@ -93,7 +93,7 @@ revision-specific provenance are under `LICENSES/protein-ttt/`.
93
  For every supported family, `src/fastplms/models.toml` records an immutable
94
  official checkpoint revision, an immutable FastPLMs checkpoint revision, file
95
  digests, a named state transformation, and a mechanism-level conversion record.
96
- Generated artifacts reproduce that record in `provenance.json`. A release or
97
  artifact build must fail when a required file identity, legal text, attribution
98
  notice, modified-file notice, upstream revision, or conversion record is absent
99
  or differs from its manifest digest.
 
46
  DPLM2 weights, and the same revision carries the complete
47
  [Apache-2.0 license](https://github.com/bytedance/dplm/blob/8a2e15e53416b4536f03f79ad1f6f6a9cbd5e19d/LICENSE).
48
  FastPLMs records both checkpoint families as Apache-2.0 and distributes the
49
+ verbatim license plus `LICENSES/dplm/SOURCE_RECORD.md`. Converted weights retain
50
  those terms and remain subject to the ordinary artifact and publication gates.
51
 
52
  ## Biohub
 
80
  revision are pinned in `docker/constraints/esmfold.txt`; OpenFold imports them
81
  eagerly, and FastPLMs production code does not depend on them. DLLogger's exact
82
  source identity and installed-license handling are recorded in
83
+ `LICENSES/dllogger/SOURCE_RECORD.md`.
84
 
85
  ## ProteinTTT
86
 
 
93
  For every supported family, `src/fastplms/models.toml` records an immutable
94
  official checkpoint revision, an immutable FastPLMs checkpoint revision, file
95
  digests, a named state transformation, and a mechanism-level conversion record.
96
+ Generated artifacts reproduce that record in `source-record.json`. A release or
97
  artifact build must fail when a required file identity, legal text, attribution
98
  notice, modified-file notice, upstream revision, or conversion record is absent
99
  or differs from its manifest digest.
fastplms/models.toml CHANGED
@@ -88,7 +88,7 @@ license_files = ["LICENSE"]
88
  license_digests = ["LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30"]
89
  distribution_files = [
90
  "LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
91
- "PROVENANCE.md=sha256:a659f74be9073cf1ad2d2f7071531ca56959b421f111152cf4c41184ace5970e",
92
  ]
93
 
94
  [[upstreams]]
@@ -122,7 +122,7 @@ license_files = ["LICENSE"]
122
  license_digests = ["LICENSE=sha256:da6d3703ed11cbe42bd212c725957c98da23cbff1998c05fa4b3d976d1a58e93"]
123
  distribution_files = [
124
  "LICENSE=sha256:da6d3703ed11cbe42bd212c725957c98da23cbff1998c05fa4b3d976d1a58e93",
125
- "PROVENANCE.md=sha256:950adb94daf15e646ddf226dacfe2a8e77801aa0793e439a9a3490a48eb666e7",
126
  ]
127
 
128
  [[upstreams]]
@@ -136,7 +136,7 @@ license_digests = ["LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c1
136
  distribution_files = [
137
  "LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
138
  "MODIFICATIONS.md=sha256:fd6f0aa1086a0c996cf967b326d18e965660cda0ad5c7f36a3474a8490720da3",
139
- "PROVENANCE.md=sha256:48c903db43a217a3126afaefbac60b7ddac7efda2dfcc0cbff0bffc7d6c30081",
140
  ]
141
 
142
  [[upstreams]]
@@ -149,7 +149,7 @@ license_files = ["LICENSE"]
149
  license_digests = ["LICENSE=sha256:bb01e7d5554f9e2e117172e56551452f68a7818df7bc8e71cd7a776a1d4ba3df"]
150
  distribution_files = [
151
  "LICENSE=sha256:bb01e7d5554f9e2e117172e56551452f68a7818df7bc8e71cd7a776a1d4ba3df",
152
- "PROVENANCE.md=sha256:dc641c37353c2efd50ccbdb316ca4aae495ec02c1563e0e15bac92f75fc482e5",
153
  ]
154
 
155
  [families.esm2]
@@ -187,7 +187,8 @@ reference_adapter = "tests.parity.support.reference_adapters.esm_plusplus"
187
  attention = ["eager", "sdpa", "flex_attention", "flash_attention_2", "flash_attention_3"]
188
  dtypes = ["float32", "bfloat16"]
189
  bf16_execution = "static_parameters"
190
- precisions = ["default"]
 
191
  vram_tier = "sequence"
192
  checkpoint_license = "MIT"
193
  hub_license = "mit"
@@ -267,7 +268,7 @@ checkpoint_license = "Apache-2.0"
267
  hub_license = "apache-2.0"
268
  weights_publication_allowed = true
269
  state_transform = "dplm_to_fastplms_v1"
270
- conversion_provenance = "Input: the pinned official DPLM1 checkpoint. Transformation: apply dplm_to_fastplms_v1, omitting the unused absolute-position table for rotary checkpoints and materializing the tied input/output embedding values as independent tensors. Output: the pinned Synthyra DPLM checkpoint. Validation: release parity compares exact state identity after the declared transform, tokenizer behavior, generation, and inference. License basis: the pinned ByteDance DPLM Apache-2.0 LICENSE and README explicitly scope the repository release to the pretrained DPLM1 and DPLM2 weights; immutable evidence is recorded in LICENSES/dplm/PROVENANCE.md. Limitation: redistribution remains subject to Apache-2.0 and the pinned source record; no broader rights are inferred."
271
  representative = "dplm_150m"
272
  documentation = "docs/models.md#dplm"
273
  test_tiers = ["check", "compliance", "feature", "artifact", "benchmark"]
@@ -291,7 +292,7 @@ checkpoint_license = "Apache-2.0"
291
  hub_license = "apache-2.0"
292
  weights_publication_allowed = true
293
  state_transform = "dplm2_to_fastplms_v1"
294
- conversion_provenance = "Input: the pinned official DPLM2 checkpoint. Transformation: apply dplm2_to_fastplms_v1, retaining the independent language-model head and trained encoder contact head while omitting the unused absolute-position table for rotary checkpoints. Output: the pinned Synthyra DPLM2 checkpoint. Validation: release parity compares exact keys and values after the declared omission, non-aliasing, tokenizer behavior, generation, and inference. License basis: the pinned ByteDance DPLM Apache-2.0 LICENSE and README explicitly scope the repository release to the pretrained DPLM1 and DPLM2 weights; immutable evidence is recorded in LICENSES/dplm/PROVENANCE.md. Limitation: no head exception is permitted by this source record, and redistribution remains subject to Apache-2.0."
295
  representative = "dplm2_150m"
296
  documentation = "docs/models.md#dplm2"
297
  test_tiers = ["check", "compliance", "feature", "artifact", "benchmark"]
 
88
  license_digests = ["LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30"]
89
  distribution_files = [
90
  "LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
91
+ "SOURCE_RECORD.md=sha256:a659f74be9073cf1ad2d2f7071531ca56959b421f111152cf4c41184ace5970e",
92
  ]
93
 
94
  [[upstreams]]
 
122
  license_digests = ["LICENSE=sha256:da6d3703ed11cbe42bd212c725957c98da23cbff1998c05fa4b3d976d1a58e93"]
123
  distribution_files = [
124
  "LICENSE=sha256:da6d3703ed11cbe42bd212c725957c98da23cbff1998c05fa4b3d976d1a58e93",
125
+ "SOURCE_RECORD.md=sha256:950adb94daf15e646ddf226dacfe2a8e77801aa0793e439a9a3490a48eb666e7",
126
  ]
127
 
128
  [[upstreams]]
 
136
  distribution_files = [
137
  "LICENSE=sha256:cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
138
  "MODIFICATIONS.md=sha256:fd6f0aa1086a0c996cf967b326d18e965660cda0ad5c7f36a3474a8490720da3",
139
+ "SOURCE_RECORD.md=sha256:48c903db43a217a3126afaefbac60b7ddac7efda2dfcc0cbff0bffc7d6c30081",
140
  ]
141
 
142
  [[upstreams]]
 
149
  license_digests = ["LICENSE=sha256:bb01e7d5554f9e2e117172e56551452f68a7818df7bc8e71cd7a776a1d4ba3df"]
150
  distribution_files = [
151
  "LICENSE=sha256:bb01e7d5554f9e2e117172e56551452f68a7818df7bc8e71cd7a776a1d4ba3df",
152
+ "SOURCE_RECORD.md=sha256:dc641c37353c2efd50ccbdb316ca4aae495ec02c1563e0e15bac92f75fc482e5",
153
  ]
154
 
155
  [families.esm2]
 
187
  attention = ["eager", "sdpa", "flex_attention", "flash_attention_2", "flash_attention_3"]
188
  dtypes = ["float32", "bfloat16"]
189
  bf16_execution = "static_parameters"
190
+ precisions = ["default", "fp8"]
191
+ experimental_precisions = ["fp8"]
192
  vram_tier = "sequence"
193
  checkpoint_license = "MIT"
194
  hub_license = "mit"
 
268
  hub_license = "apache-2.0"
269
  weights_publication_allowed = true
270
  state_transform = "dplm_to_fastplms_v1"
271
+ conversion_provenance = "Input: the pinned official DPLM1 checkpoint. Transformation: apply dplm_to_fastplms_v1, omitting the unused absolute-position table for rotary checkpoints and materializing the tied input/output embedding values as independent tensors. Output: the pinned Synthyra DPLM checkpoint. Validation: release parity compares exact state identity after the declared transform, tokenizer behavior, generation, and inference. License basis: the pinned ByteDance DPLM Apache-2.0 LICENSE and README explicitly scope the repository release to the pretrained DPLM1 and DPLM2 weights; immutable evidence is recorded in LICENSES/dplm/SOURCE_RECORD.md. Limitation: redistribution remains subject to Apache-2.0 and the pinned source record; no broader rights are inferred."
272
  representative = "dplm_150m"
273
  documentation = "docs/models.md#dplm"
274
  test_tiers = ["check", "compliance", "feature", "artifact", "benchmark"]
 
292
  hub_license = "apache-2.0"
293
  weights_publication_allowed = true
294
  state_transform = "dplm2_to_fastplms_v1"
295
+ conversion_provenance = "Input: the pinned official DPLM2 checkpoint. Transformation: apply dplm2_to_fastplms_v1, retaining the independent language-model head and trained encoder contact head while omitting the unused absolute-position table for rotary checkpoints. Output: the pinned Synthyra DPLM2 checkpoint. Validation: release parity compares exact keys and values after the declared omission, non-aliasing, tokenizer behavior, generation, and inference. License basis: the pinned ByteDance DPLM Apache-2.0 LICENSE and README explicitly scope the repository release to the pretrained DPLM1 and DPLM2 weights; immutable evidence is recorded in LICENSES/dplm/SOURCE_RECORD.md. Limitation: no head exception is permitted by this source record, and redistribution remains subject to Apache-2.0."
296
  representative = "dplm2_150m"
297
  documentation = "docs/models.md#dplm2"
298
  test_tiers = ["check", "compliance", "feature", "artifact", "benchmark"]
fastplms/models/e1/attention.py CHANGED
@@ -322,11 +322,7 @@ def varlen_flex_attention_func(
322
  seqlens_k = cu_seqlens_k[1:] - cu_seqlens_k[:-1] # (n,)
323
  block_mask = block_mask_creator(seqlens_q, seqlens_k)
324
 
325
- packed_lengths = (
326
- *(int(length) for length in seqlens_q.tolist()),
327
- -1,
328
- *(int(length) for length in seqlens_k.tolist()),
329
- )
330
  fn = _get_flex_attention_fn(
331
  device=query_states.device,
332
  dtype=query_states.dtype,
@@ -351,6 +347,21 @@ def varlen_flex_attention_func(
351
  return attn_output # (b, l_q, h, d)
352
 
353
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
354
  def _get_unpad_data(sequence_ids: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, int]:
355
  """Return packed indices and run lengths for the non-padding sequence IDs."""
356
 
@@ -384,6 +395,7 @@ def _get_unpad_data(sequence_ids: torch.Tensor) -> tuple[torch.Tensor, torch.Ten
384
  )
385
 
386
 
 
387
  def _unpad_input(
388
  query_layer: torch.Tensor,
389
  key_layer: torch.Tensor,
 
322
  seqlens_k = cu_seqlens_k[1:] - cu_seqlens_k[:-1] # (n,)
323
  block_mask = block_mask_creator(seqlens_q, seqlens_k)
324
 
325
+ packed_lengths = _packed_lengths_cache_key(seqlens_q, seqlens_k)
 
 
 
 
326
  fn = _get_flex_attention_fn(
327
  device=query_states.device,
328
  dtype=query_states.dtype,
 
347
  return attn_output # (b, l_q, h, d)
348
 
349
 
350
+ @torch.compiler.disable
351
+ def _packed_lengths_cache_key(
352
+ query_lengths: torch.Tensor,
353
+ key_lengths: torch.Tensor,
354
+ ) -> tuple[int, ...]:
355
+ """Convert packed lengths to an immutable Flex cache key eagerly."""
356
+
357
+ return (
358
+ *(int(length) for length in query_lengths.tolist()),
359
+ -1,
360
+ *(int(length) for length in key_lengths.tolist()),
361
+ )
362
+
363
+
364
+ @torch.compiler.disable
365
  def _get_unpad_data(sequence_ids: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, int]:
366
  """Return packed indices and run lengths for the non-padding sequence IDs."""
367
 
 
395
  )
396
 
397
 
398
+ @torch.compiler.disable
399
  def _unpad_input(
400
  query_layer: torch.Tensor,
401
  key_layer: torch.Tensor,
fastplms/models/e1/modeling_e1.py CHANGED
@@ -135,6 +135,54 @@ def _get_logger():
135
  return logging.get_logger(__name__)
136
 
137
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
  _TOKENIZER_LOAD_CONTEXT: ContextVar[dict[str, Any] | None] = ContextVar(
139
  "fastplms_e1_tokenizer_load_context",
140
  default=None,
@@ -319,6 +367,19 @@ class RotaryPositionalEmbedding(nn.Module):
319
  self.cos_cached = angles.cos()
320
  self.sin_cached = angles.sin()
321
 
 
 
 
 
 
 
 
 
 
 
 
 
 
322
  def forward(
323
  self,
324
  q: torch.Tensor,
@@ -328,10 +389,7 @@ class RotaryPositionalEmbedding(nn.Module):
328
  ) -> tuple[torch.Tensor, torch.Tensor]:
329
  # q, k: (b, l, h, d)
330
  device, dtype = q.device, q.dtype
331
- seq_len = position_ids.max().item() + 1 if seq_len is None else seq_len
332
-
333
- if seq_len > self.max_seq_len_cached:
334
- self._set_sin_cos_cache(seq_len=seq_len, device=device)
335
 
336
  # Selecting by position inserts a head axis for broadcasting.
337
  idxs = position_ids.to(device)
@@ -717,8 +775,7 @@ class Attention(nn.Module):
717
  if cached_len < 0:
718
  raise ValueError(f"E1 cached KV length {kv_len} is shorter than query length {q_len}.")
719
  first_sequence_id = query_sequence_ids[:, :1]
720
- if bool(first_sequence_id.eq(-1).any()):
721
- raise ValueError("E1 cached queries must start with a non-padding sequence token.")
722
  cached_sequence_ids = first_sequence_id.expand(-1, cached_len)
723
  key_sequence_ids = torch.cat((cached_sequence_ids, query_sequence_ids), dim=-1)
724
  return query_sequence_ids, key_sequence_ids
@@ -1314,33 +1371,12 @@ class FAST_E1_ENCODER(E1PreTrainedModel, EmbeddingMixin):
1314
  within_positions = within_seq_position_ids.long()
1315
  global_positions = global_position_ids.long()
1316
  sequence_numbers = sequence_ids.long()
1317
- lowest_position, highest_position = torch.aminmax(within_positions)
1318
- if (
1319
- lowest_position.item() < -1
1320
- or highest_position.item() >= self.config.max_num_positions_within_seq
1321
- ):
1322
- raise ValueError(
1323
- "Position ids must be in the range "
1324
- f"[-1, {self.config.max_num_positions_within_seq}); got max "
1325
- f"{highest_position.item()} and min {lowest_position.item()}"
1326
- )
1327
- lowest_global, highest_global = torch.aminmax(global_positions)
1328
- if (
1329
- lowest_global.item() < -1
1330
- or highest_global.item() >= self.config.max_num_positions_global
1331
- ):
1332
- raise ValueError(
1333
- "Global position ids must be in the range "
1334
- f"[-1, {self.config.max_num_positions_global}); got max "
1335
- f"{highest_global.item()} and min {lowest_global.item()}"
1336
- )
1337
- lowest_sequence, highest_sequence = torch.aminmax(sequence_numbers)
1338
- if lowest_sequence.item() < -1 or highest_sequence.item() >= self.config.max_num_sequences:
1339
- raise ValueError(
1340
- "Sequence ids must be in the range "
1341
- f"[-1, {self.config.max_num_sequences}); got max "
1342
- f"{highest_sequence.item()} and min {lowest_sequence.item()}"
1343
- )
1344
 
1345
  if inputs_embeds is None:
1346
  if input_ids is None:
 
135
  return logging.get_logger(__name__)
136
 
137
 
138
+ @torch.compiler.disable
139
+ def _validate_cached_global_query_start(first_sequence_id: torch.Tensor) -> None:
140
+ """Keep cached-query validation eager while preserving padding semantics."""
141
+
142
+ if bool(first_sequence_id.eq(-1).any()):
143
+ raise ValueError("E1 cached queries must start with a non-padding sequence token.")
144
+
145
+
146
+ @torch.compiler.disable
147
+ def _validate_biological_indices(
148
+ within_positions: torch.Tensor,
149
+ global_positions: torch.Tensor,
150
+ sequence_numbers: torch.Tensor,
151
+ config: E1Config,
152
+ ) -> None:
153
+ """Validate E1's sequence and position conventions outside compiled graphs."""
154
+
155
+ lowest_position, highest_position = torch.aminmax(within_positions)
156
+ minimum_position = int(lowest_position.item())
157
+ maximum_position = int(highest_position.item())
158
+ if minimum_position < -1 or maximum_position >= config.max_num_positions_within_seq:
159
+ raise ValueError(
160
+ "Position ids must be in the range "
161
+ f"[-1, {config.max_num_positions_within_seq}); got max "
162
+ f"{maximum_position} and min {minimum_position}"
163
+ )
164
+
165
+ lowest_global, highest_global = torch.aminmax(global_positions)
166
+ minimum_global = int(lowest_global.item())
167
+ maximum_global = int(highest_global.item())
168
+ if minimum_global < -1 or maximum_global >= config.max_num_positions_global:
169
+ raise ValueError(
170
+ "Global position ids must be in the range "
171
+ f"[-1, {config.max_num_positions_global}); got max "
172
+ f"{maximum_global} and min {minimum_global}"
173
+ )
174
+
175
+ lowest_sequence, highest_sequence = torch.aminmax(sequence_numbers)
176
+ minimum_sequence = int(lowest_sequence.item())
177
+ maximum_sequence = int(highest_sequence.item())
178
+ if minimum_sequence < -1 or maximum_sequence >= config.max_num_sequences:
179
+ raise ValueError(
180
+ "Sequence ids must be in the range "
181
+ f"[-1, {config.max_num_sequences}); got max "
182
+ f"{maximum_sequence} and min {minimum_sequence}"
183
+ )
184
+
185
+
186
  _TOKENIZER_LOAD_CONTEXT: ContextVar[dict[str, Any] | None] = ContextVar(
187
  "fastplms_e1_tokenizer_load_context",
188
  default=None,
 
367
  self.cos_cached = angles.cos()
368
  self.sin_cached = angles.sin()
369
 
370
+ @torch.compiler.disable
371
+ def _ensure_sin_cos_cache(
372
+ self,
373
+ position_ids: torch.Tensor,
374
+ seq_len: int | None,
375
+ device: torch.device,
376
+ ) -> None:
377
+ """Resize the data-dependent rotary cache outside compiled graphs."""
378
+
379
+ required_length = int(position_ids.max().item()) + 1 if seq_len is None else seq_len
380
+ if required_length > self.max_seq_len_cached:
381
+ self._set_sin_cos_cache(seq_len=required_length, device=device)
382
+
383
  def forward(
384
  self,
385
  q: torch.Tensor,
 
389
  ) -> tuple[torch.Tensor, torch.Tensor]:
390
  # q, k: (b, l, h, d)
391
  device, dtype = q.device, q.dtype
392
+ self._ensure_sin_cos_cache(position_ids, seq_len, device)
 
 
 
393
 
394
  # Selecting by position inserts a head axis for broadcasting.
395
  idxs = position_ids.to(device)
 
775
  if cached_len < 0:
776
  raise ValueError(f"E1 cached KV length {kv_len} is shorter than query length {q_len}.")
777
  first_sequence_id = query_sequence_ids[:, :1]
778
+ _validate_cached_global_query_start(first_sequence_id)
 
779
  cached_sequence_ids = first_sequence_id.expand(-1, cached_len)
780
  key_sequence_ids = torch.cat((cached_sequence_ids, query_sequence_ids), dim=-1)
781
  return query_sequence_ids, key_sequence_ids
 
1371
  within_positions = within_seq_position_ids.long()
1372
  global_positions = global_position_ids.long()
1373
  sequence_numbers = sequence_ids.long()
1374
+ _validate_biological_indices(
1375
+ within_positions,
1376
+ global_positions,
1377
+ sequence_numbers,
1378
+ self.config,
1379
+ )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1380
 
1381
  if inputs_embeds is None:
1382
  if input_ids is None:
fastplms/models/e1/retrieval.py CHANGED
@@ -890,7 +890,7 @@ class HomologueSearcher:
890
  """
891
 
892
  _PROVENANCE_SCHEMA_VERSION = 1
893
- _PROVENANCE_FILENAME = "search-provenance.json"
894
 
895
  def __init__(
896
  self,
 
890
  """
891
 
892
  _PROVENANCE_SCHEMA_VERSION = 1
893
+ _PROVENANCE_FILENAME = "search-record.json"
894
 
895
  def __init__(
896
  self,
fastplms_bundle.py CHANGED
The diff for this file is too large to render. See raw diff
 
modeling_fastplms.py CHANGED
@@ -12,7 +12,7 @@ from zipfile import ZIP_DEFLATED, ZipFile
12
 
13
  from .fastplms_bundle import RUNTIME_DATA, RUNTIME_HASH
14
 
15
- if RUNTIME_HASH != "a8dda20d6a4a55e6808c7c2f2c9477c113edaa7d6ec6ef57f4a898cd47b73d48":
16
  raise RuntimeError("FastPLMs runtime identity differs from the bridge.")
17
 
18
  _RUNTIME_TEMPORARIES = []
 
12
 
13
  from .fastplms_bundle import RUNTIME_DATA, RUNTIME_HASH
14
 
15
+ if RUNTIME_HASH != "16de6217cddadfeb81d6086a8c0f273176854dc99a0b3443c89cbc998f516aba":
16
  raise RuntimeError("FastPLMs runtime identity differs from the bridge.")
17
 
18
  _RUNTIME_TEMPORARIES = []