robbywang commited on
Commit
76e9c2d
·
verified ·
1 Parent(s): c6d7927

Update CYP challenge method report

Browse files
Files changed (1) hide show
  1. README.md +56 -27
README.md CHANGED
@@ -6,44 +6,73 @@ tags:
6
  ---
7
  # yw — OpenADMET CYP direct-inhibition method report
8
 
9
- Updated: 2026-10-17T10:49:06.757590+00:00
10
 
11
  ## Submission
12
 
13
- This report describes the latest best-scored public challenge submission under the
14
- anonymous leaderboard alias **yw**. It uses no proprietary training data.
15
 
16
- Prediction file SHA-256: `1799926f3d09c069ea9b348d024a2abd72efac544a631cfd78d05410d2adc2b0`.
17
- Model revision: `06e1089342f4ec8297830b103935684e73668812`.
18
 
19
- ## Data and method
 
 
 
20
 
21
- Interval-bound LightGBM champion with a CYP2D6 assay-tagged screen nonhit probability correction
22
 
23
- Features: Morgan radius-2 count fingerprints with chirality, RDKit descriptors, a CYP2D6 screen nonhit classifier and separate hit/nonhit L1 response models
24
 
25
- Data sources:
26
- - OpenADMET direct-inhibition training data (champion)
27
- - OpenADMET single-concentration log2 fold change under TDI condition
28
- - Local screen-multitask-features champion predictions
 
 
29
 
30
- Validation: Controller molecule-grouped random, scaffold, test-like direct splits and five-fold screen splits; all screen data from held-out canonical molecules excluded
31
 
32
- Compounds were separated before fitting any model component or generating
33
- out-of-fold assay features. Direct, TDI, and screening readouts were treated
34
- as distinct assay observations. CYP2D6 direct measurements are selected
35
- from screening hits, limiting what random-fold direct validation establishes.
36
 
37
- ## Live leaderboard result
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
- Overall MA-ST-RAE: **0.797**.
40
 
41
- | Enzyme | ST-RAE |
42
- | --- | ---: |
43
- | CYP1A2 | 0.887 |
44
- | CYP2C9 | 0.5562 |
45
- | CYP2D6 | 1.172 |
46
- | CYP3A4 | 0.5728 |
 
 
 
 
 
 
 
 
 
 
 
47
 
48
- The live leaderboard uses a blinded subset. The report will be updated when
49
- the final prediction is selected.
 
 
 
6
  ---
7
  # yw — OpenADMET CYP direct-inhibition method report
8
 
9
+ Updated: 2026-11-03T16:30:42.279580+00:00
10
 
11
  ## Submission
12
 
13
+ This report describes the final public challenge submission under the anonymous leaderboard
14
+ alias **yw**. It uses no proprietary training data and no external assay data. No code is published.
15
 
16
+ Prediction file SHA-256: `80695a19012518a3fbfd5ec3086dd112701b307b27abacba41b9472ced05ed0f`.
 
17
 
18
+ The file combines, per enzyme, prediction columns from earlier model runs (no joint refit and no new
19
+ training). Each enzyme's column was chosen by comparing the public live-leaderboard ST-RAE of earlier
20
+ submissions that contained it. The live leaderboard covers only part of the blinded series, so these
21
+ choices are tuned to that subset.
22
 
23
+ ## Live leaderboard result
24
 
25
+ Overall MA-ST-RAE of this file: expected 0.7404 (mean of the four per-enzyme scores; the board value will be added when scored).
26
 
27
+ | Enzyme | Source run | Live ST-RAE of the column |
28
+ | --- | --- | ---: |
29
+ | CYP1A2 | `20261031T032022-cyp1a2-nested-support-conditioned-family-gate` | 0.772 |
30
+ | CYP2C9 | `20261029T175417-cyp1a2-cyp3a4-fixed-descriptor-graph-ensemble` | 0.5217 |
31
+ | CYP2D6 | `20261017T004159-cyp2d6-assay-tagged-nonhit-calibration` | 1.172 |
32
+ | CYP3A4 | `20261029T175417-cyp1a2-cyp3a4-fixed-descriptor-graph-ensemble` | 0.4958 |
33
 
34
+ ## Data and method of each source run
35
 
36
+ Each description is quoted from that run's own record. Only the enzymes listed for a run are taken
37
+ from it, so statements about its other enzymes do not apply to this file.
 
 
38
 
39
+ ### `20261031T032022-cyp1a2-nested-support-conditioned-family-gate` (used for CYP1A2)
40
+
41
+ Model revision: `c232bdc94f87c336f3253b2e51c52bae87474d32`.
42
+
43
+ Method: CYP1A2 p=L+clip(0.5+Z beta,0,1)*(J-L), fixed eight-column support gate. Deterministic interval-error LP plus .01 L1 beta penalty, training weights constrained [0,1]. Full real-support gate is sole candidate; no control selection. Three other endpoints exactly F.
44
+
45
+ Features: intercept; m_z; d_z; abs_d_z; similarity_z; scaffold; m_z*d_z; similarity_z*d_z
46
+
47
+ Data sources: Challenge direct confidence intervals; Archived L tree and J descriptor-graph nested predictions; separately tagged TDI-condition screening supervision only in base models
48
+
49
+ Validation: Controller canonical random/scaffold five-fold, testlike holdout and archived Butina cluster five-fold. Gate training uses only each population inner OOF archive and permitted fitting labels; blinded fit uses full inner OOF. All 85 inner fits and 17 refits audited before gate training; no global outer OOF input.
50
+
51
+ ### `20261029T175417-cyp1a2-cyp3a4-fixed-descriptor-graph-ensemble` (used for CYP2C9, CYP3A4)
52
+
53
+ Model revision: `911812692f61783085b34085e0fb498683c28dcc`.
54
 
55
+ Method: Fixed (L + GJ)/2 for CYP1A2 and CYP3A4; C CYP2C9/CYP2D6 copied exactly. GJ is the archived equal three-seed epoch-100 graph. No training, calibration, learned weights, checkpoint selection, or control substitution.
56
 
57
+ Features: Chemprop 2.3.1 depth-3 256-dimensional BondMessagePassing, default atom/bond features, fixed half-mean/half-componentwise-max pooling; X_d concatenation to 268-input two-layer eight-output RegressionFFN, hidden256, dropout0.1, no batch normalization; ['MolWt', 'MolLogP', 'TPSA', 'MolMR', 'NumHDonors', 'NumHAcceptors', 'NumRotatableBonds', 'RingCount', 'NumAromaticRings', 'FractionCSP3', 'HeavyAtomCount', 'NumHeteroatoms']; Historical LightGBM: chiral Morgan counts, RDKit descriptors, nested predicted TDI-condition screening responses
58
+
59
+ Data sources: Challenge direct-inhibition intervals; Separate TDI-condition single-concentration screen tasks; Hash-pinned completed L, C, and J archives; no new assay data
60
+
61
+ Validation: Unchanged canonical random/scaffold five-fold and testlike holdout; C fixed Butina cluster diagnostic. Every graph fit/held index and fitting-only descriptor/target normalization verified; all 153 saved inference arrays exactly reproduce seed-average exports. Historical stacking control flow and saved cluster screen memberships audited. TDI/Emax labels were not used in these components. All four predictions finite for every held row. No held assay enters a fitted component.
62
+
63
+ ### `20261017T004159-cyp2d6-assay-tagged-nonhit-calibration` (used for CYP2D6)
64
+
65
+ Model revision: `06e1089342f4ec8297830b103935684e73668812`.
66
+
67
+ Method: Interval-bound LightGBM champion with a CYP2D6 assay-tagged screen nonhit probability correction
68
+
69
+ Features: Morgan radius-2 count fingerprints with chirality, RDKit descriptors, a CYP2D6 screen nonhit classifier and separate hit/nonhit L1 response models
70
+
71
+ Data sources: OpenADMET direct-inhibition training data (champion); OpenADMET single-concentration log2 fold change under TDI condition; Local screen-multitask-features champion predictions
72
+
73
+ Validation: Controller molecule-grouped random, scaffold, test-like direct splits and five-fold screen splits; all screen data from held-out canonical molecules excluded
74
 
75
+ Compounds were separated before fitting any model component or generating out-of-fold assay
76
+ features. Direct, TDI, and screening readouts were treated as distinct assay observations. CYP2D6
77
+ direct measurements are selected from screening hits, limiting what random-fold direct validation
78
+ establishes.