20eddibae commited on
Commit
3cc8bf6
·
verified ·
1 Parent(s): 8dc7b38

Update card: seeded effect size, retired inversion claim, TCRdist own-readout draw

Browse files
Files changed (1) hide show
  1. README.md +46 -11
README.md CHANGED
@@ -11,15 +11,40 @@ tags:
11
  pipeline_tag: feature-extraction
12
  ---
13
 
14
- # SCEPTR-AC (TCRdist soft-label champion)
 
 
 
15
 
16
  A paired-chain αβ **T-cell receptor (TCR) language model** that maps a TCR to a
17
  128-dimensional vector such that TCRs binding the same epitope land close
18
- together. It is the champion of a controlled scaling study built on
19
  [SCEPTR](https://github.com/yutanagano/sceptr) (Nagano et al. 2025, *Cell
20
- Systems*), and is the first learned encoder in this study to **match TCRdist**
21
- on the paper's zero-shot epitope-retrieval benchmark while embedding at a
22
- fraction of the cost.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  - **Architecture:** 6-layer Pre-LN / NormFormer transformer encoder,
25
  d_model = 128, 8 heads, `CdrSimpleEmbedder` (embedding_dim = 29), CLS-token
@@ -50,10 +75,20 @@ This is the model's headline use case. Give it **k known binders** for an
50
  epitope (the reference set) and it scores **any query TCR** for how strongly
51
  it is specific to that epitope — no epitope sequence and no classifier
52
  retraining required. It is the epitope-free, learned analogue of TCRMatch and
53
- tcrdist3 meta-clonotypes, and on the paper's SOTA slate it leads every
54
- baseline (NN AUROC@50 = 0.801, prototype AUROC@50 = 0.860; a distinctive
55
- +0.058 gain from prototype scoring that the alignment/kernel baselines do not
56
- get).
 
 
 
 
 
 
 
 
 
 
57
 
58
  ```python
59
  import numpy as np
@@ -109,7 +144,7 @@ epitopes:
109
 
110
  | Model | AUROC@50 | AUROC@200 |
111
  |---|---|---|
112
- | **SCEPTR-AC champion (this model)** | **0.7945** | **0.8138** |
113
  | TCRdist (reference) | 0.7787 | 0.8020 |
114
  | SCEPTR (base AC, published) | 0.7360 | 0.7459 |
115
  | CDR3 Levenshtein | 0.6989 | — |
@@ -122,7 +157,7 @@ checkpoint with the command in the repo README.
122
  predicts binding only by retrieval against a labeled reference set.
123
  - The scaling study found this recipe is **plateaued**: deeper (12L), wider
124
  (d=256), modern-norm (RMSNorm/SwiGLU/QK-Norm), and larger-corpus variants all
125
- tie or lose to this 6L Tanno-only champion.
126
  - ~8.6% of Tanno CDR3β sequences also appear in VDJdb — the field-standard
127
  background contamination shared by TCRdist and the published SCEPTR baseline
128
  alike; a CDR3β-clean leak-free benchmark preserves the same small advantage.
 
11
  pipeline_tag: feature-extraction
12
  ---
13
 
14
+ # CLEST-TCRdist
15
+
16
+ **C**ontrastive **LE**arning with **S**oft **T**argets from **TCRdist**.
17
+ Checkpoint `SCEPTR6L_AC_soft_800k_20ep_bs1024`.
18
 
19
  A paired-chain αβ **T-cell receptor (TCR) language model** that maps a TCR to a
20
  128-dimensional vector such that TCRs binding the same epitope land close
21
+ together. It is built on
22
  [SCEPTR](https://github.com/yutanagano/sceptr) (Nagano et al. 2025, *Cell
23
+ Systems*) and changes exactly one term of its training objective: the
24
+ autocontrastive target, which treats every other receptor in the batch as a
25
+ negative, is replaced with a graded target `exp(-d_TCRdist/τ)` computed in
26
+ batch (τ = 50). Architecture, corpus, augmentation and compute are held fixed
27
+ against the autocontrastive control, so the difference between them is
28
+ attributable to that term.
29
+
30
+ ### Two things to read before quoting a number
31
+
32
+ - **This is one training run, not a mean.** The checkpoint predates seeding in
33
+ our pipeline. Three seeded replicates of the identical configuration score
34
+ 0.809 ± 0.006 prototype AUROC@50 and 0.774 ± 0.009 at nearest-neighbour, where
35
+ this run scores 0.842 and 0.794 — it sits above all three in every cell. Treat
36
+ the replicate means as the recipe's expected performance.
37
+ - **What the objective is actually worth.** Seeding both arms (3 soft-label
38
+ replicates, 2 autocontrastive controls, scored in one run on shared reference
39
+ draws) puts the gap over the matched control at **+0.038 AUROC@50** at
40
+ nearest-neighbour — six times the pooled seed spread, positive on five of six
41
+ epitopes — down from the +0.058 the two published runs suggested. The control
42
+ arm reproduces its published run almost exactly (0.7356/0.7366 vs 0.7361), so
43
+ this is a real shrinkage and not environment drift.
44
+ - **It does not beat TCRdist on accuracy, and we do not claim it does.** Over
45
+ seeds the 1-NN margin changes sign. What the objective buys is measured
46
+ against its own autocontrastive control, and a scoring-rule inversion that is
47
+ itself conditional on the evaluation repertoire (see the paper).
48
 
49
  - **Architecture:** 6-layer Pre-LN / NormFormer transformer encoder,
50
  d_model = 128, 8 heads, `CdrSimpleEmbedder` (embedding_dim = 29), CLS-token
 
75
  epitope (the reference set) and it scores **any query TCR** for how strongly
76
  it is specific to that epitope — no epitope sequence and no classifier
77
  retraining required. It is the epitope-free, learned analogue of TCRMatch and
78
+ tcrdist3 meta-clonotypes. On the paper's fourteen-method slate this
79
+ checkpoint ranks first (NN AUROC@50 = 0.801, prototype AUROC@50 = 0.860), but
80
+ that ranking should be read with two controls attached: the checkpoint was
81
+ selected on the same epitope panel it is scored on (moving selection to six
82
+ held-out epitopes costs 0.013), and it is a single unseeded run that sits above
83
+ all three of its seeded replicates. The margin over TCRdist does not survive
84
+ either control, and does not survive giving TCRdist its own published read-out
85
+ either (Dash et al. score the nearest 10% of the reference set, not the nearest
86
+ one; under each method's own best rule the comparison is a draw, +0.010,
87
+ p = 0.42). The durable comparison is against the matched autocontrastive
88
+ control. Note that the prototype-over-nearest-neighbour gain is **not** specific
89
+ to this objective: once the control is reseeded it gains +0.041 from prototype
90
+ scoring against this model's +0.034, so that inversion is a property of the
91
+ evaluation repertoire rather than of the training target.
92
 
93
  ```python
94
  import numpy as np
 
144
 
145
  | Model | AUROC@50 | AUROC@200 |
146
  |---|---|---|
147
+ | **CLEST-TCRdist (this model)** | **0.7945** | **0.8138** |
148
  | TCRdist (reference) | 0.7787 | 0.8020 |
149
  | SCEPTR (base AC, published) | 0.7360 | 0.7459 |
150
  | CDR3 Levenshtein | 0.6989 | — |
 
157
  predicts binding only by retrieval against a labeled reference set.
158
  - The scaling study found this recipe is **plateaued**: deeper (12L), wider
159
  (d=256), modern-norm (RMSNorm/SwiGLU/QK-Norm), and larger-corpus variants all
160
+ tie or lose to this 6L Tanno-only recipe.
161
  - ~8.6% of Tanno CDR3β sequences also appear in VDJdb — the field-standard
162
  background contamination shared by TCRdist and the published SCEPTR baseline
163
  alike; a CDR3β-clean leak-free benchmark preserves the same small advantage.