Upload results/ralph26_comprehensive_summary.md with huggingface_hub

results/ralph26_comprehensive_summary.md

@@ -0,0 +1,172 @@
+# Ralph26 Comprehensive Summary: PFAM Module K-Sweep and Target-Aware Feature Selection
+
+**Date:** 2026-02-12
+**Provenance:** Compiled from results of Tasks 1-9, all using real data from TARA-Oceans metagenomic assemblies
+**Scripts:** `/media/drn2/External/TARA-Oceans/03_analyses/WorldModelApp/scripts/ralph26_task{1..9}_*.py`
+
+---
+
+## Executive Summary
+
+**VERDICT: NEGATIVE** — No PFAM genomic representation achieves significant complementary signal for POC prediction beyond environmental features. Across 8 module resolutions, 3 target-aware feature sets, within-fold leakage-free selection, residual prediction, ratio target engineering, and regional subsetting, zero comparisons achieve p < 0.05 for POC. The best representation (k=40 modules, dR2 = +0.040) does not survive permutation testing (empirical p = 0.199). The joint embedding analysis for POC should be reframed as a negative but informative result.
+
+---
+
+## 1. Module K-Sweep (Task 3)
+
+Env-only baseline: R2 = 0.631 +/- 0.239 (9-fold spatial CV, n=1,151 bio-valid samples)
+
+| k  | dR2     | t-pval | Wilcoxon p | Cohen's d | 95% CI              | Folds+ |
+|----|---------|--------|------------|-----------|----------------------|--------|
+| 5  | -0.012  | 0.750  | 1.000      | -0.110    | [-0.097, +0.073]     | 5/9    |
+| 8  | -0.049  | 0.492  | 1.000      | -0.240    | [-0.206, +0.108]     | 6/9    |
+| 10 | +0.013  | 0.644  | 0.496      | +0.160    | [-0.048, +0.073]     | 6/9    |
+| 15 | -0.007  | 0.886  | 0.652      | -0.049    | [-0.115, +0.102]     | 6/9    |
+| 20 | -0.093  | 0.157  | 0.129      | -0.521    | [-0.231, +0.045]     | 2/9    |
+| 30 | -0.530  | 0.364  | 0.820      | -0.321    | [-1.800, +0.740]     | 4/9    |
+| 40 | +0.040  | 0.471  | 0.496      | +0.252    | [-0.081, +0.161]     | 6/9    |
+| 50 | +0.025  | 0.670  | 0.301      | +0.148    | [-0.106, +0.157]     | 6/9    |
+
+**Finding:** No k value achieves significance. Best: k=40 (dR2 = +0.040, p = 0.47). k=20 (the original Ralph23 choice) is among the worst (dR2 = -0.093). k=30 is catastrophic (mean R2 = 0.10 due to one extreme fold).
+
+---
+
+## 2. POC-Correlated Feature Selection (Task 4, with leakage)
+
+| Feature Set | N    | dR2     | t-pval | Wilcoxon p | Cohen's d | Folds+ |
+|-------------|------|---------|--------|------------|-----------|--------|
+| top20       | 20   | -0.075  | 0.434  | 0.734      | -0.275    | 3/9    |
+| top50       | 50   | +0.017  | 0.623  | 0.652      | +0.171    | 4/9    |
+| top100      | 100  | -0.034  | 0.398  | 1.000      | -0.298    | 5/9    |
+
+**Finding:** Even with leaked feature selection (upper bound), no improvement. Best: top50 (dR2 = +0.017, p = 0.62).
+
+---
+
+## 3. Within-Fold Feature Selection (Task 5, leakage-free)
+
+| Config          | N  | dR2     | t-pval | Wilcoxon p | Cohen's d | Folds+ | Jaccard |
+|-----------------|-----|---------|--------|------------|-----------|--------|---------|
+| within-fold 50  | 50  | +0.015  | 0.738  | 0.164      | +0.115    | 7/9    | 0.744   |
+
+**Finding:** Leakage-free result (dR2 = +0.015, p = 0.74) confirms Task 4 finding. High Jaccard similarity (0.744) means the same ~31 core PFAMs are consistently selected across folds, but their signal is already captured by environmental features.
+
+---
+
+## 4. Residual Prediction (Task 6)
+
+| Config       | N  | Mean R2 (residuals) | t-pval (one-sided) | Folds R2>0 |
+|-------------|-----|---------------------|---------------------|------------|
+| modules_k40  | 40  | -11.01              | 0.859               | 1/9        |
+| poc_top50    | 50  | -4.45               | 0.935               | 0/9        |
+
+**Finding:** DEFINITIVELY NEGATIVE. PFAM features cannot predict env-only POC residuals. Mean R2 values are massively negative (worse than predicting mean residual). No complementary signal exists.
+
+---
+
+## 5. Ratio Target Engineering (Task 7)
+
+Target: log2(POC / Chl-a) — the "dark carbon" ratio
+
+| Config          | N  | dR2     | t-pval | Wilcoxon p | Cohen's d | Folds+ |
+|-----------------|-----|---------|--------|------------|-----------|--------|
+| env_modules_k40 | 40  | +0.005  | 0.798  | 0.910      | +0.088    | 5/9    |
+| env_poc_top50   | 50  | +0.015  | 0.499  | 0.652      | +0.236    | 6/9    |
+
+**Finding:** Ratio target does not rescue PFAM signal. The "dark carbon" hypothesis is not supported.
+
+---
+
+## 6. Regional Subsetting (Task 8)
+
+| Region   | Config       | dR2     | t-pval | Cohen's d | Folds+ |
+|----------|-------------|---------|--------|-----------|--------|
+| Atlantic | modules_k40  | +0.002  | 0.987  | +0.011    | 1/3    |
+| Atlantic | poc_top50    | -0.156  | 0.358  | -0.684    | 1/3    |
+| Pacific  | modules_k40  | -0.079  | 0.154  | -1.294    | 0/3    |
+| Pacific  | poc_top50    | -3.590  | 0.426  | -0.573    | 2/3    |
+
+**Finding:** No basin-specific PFAM signal for POC. Atlantic flat, Pacific actively harmful.
+
+---
+
+## 7. Permutation Test on Best Representation (Task 9)
+
+Best: modules_k40 (Task 3, dR2 = +0.040)
+
+| Metric                | Value    |
+|----------------------|----------|
+| Observed dR2 (fast)  | +0.054   |
+| Null mean            | +0.003   |
+| Null std             | 0.106    |
+| N permutations       | 1,000    |
+| N exceeding observed | 198      |
+| **Empirical p-value**| **0.199**|
+| Z-score              | +0.48    |
+
+**Finding:** NOT SIGNIFICANT. 198/1000 random permutations produced dR2 >= observed. The best PFAM representation is indistinguishable from chance.
+
+---
+
+## Comprehensive Verdict
+
+### Evidence Table (POC only, all analyses)
+
+| Task | Analysis                         | dR2     | p-value | Significant? |
+|------|----------------------------------|---------|---------|--------------|
+| 3    | K-sweep: k=5                     | -0.012  | 0.750   | No           |
+| 3    | K-sweep: k=8                     | -0.049  | 0.492   | No           |
+| 3    | K-sweep: k=10                    | +0.013  | 0.644   | No           |
+| 3    | K-sweep: k=15                    | -0.007  | 0.886   | No           |
+| 3    | K-sweep: k=20                    | -0.093  | 0.157   | No           |
+| 3    | K-sweep: k=30                    | -0.530  | 0.364   | No           |
+| 3    | K-sweep: k=40 (best overall)     | +0.040  | 0.471   | No           |
+| 3    | K-sweep: k=50                    | +0.025  | 0.670   | No           |
+| 4    | POC-correlated top20 (leaked)    | -0.075  | 0.434   | No           |
+| 4    | POC-correlated top50 (leaked)    | +0.017  | 0.623   | No           |
+| 4    | POC-correlated top100 (leaked)   | -0.034  | 0.398   | No           |
+| 5    | Within-fold top50 (leakage-free) | +0.015  | 0.738   | No           |
+| 6    | Residual: modules_k40            | R2=-11  | 0.859   | No           |
+| 6    | Residual: poc_top50              | R2=-4.5 | 0.935   | No           |
+| 7    | Ratio target: modules_k40        | +0.005  | 0.798   | No           |
+| 7    | Ratio target: poc_top50          | +0.015  | 0.499   | No           |
+| 8    | Atlantic: modules_k40            | +0.002  | 0.987   | No           |
+| 8    | Atlantic: poc_top50              | -0.156  | 0.358   | No           |
+| 8    | Pacific: modules_k40             | -0.079  | 0.154   | No           |
+| 8    | Pacific: poc_top50               | -3.590  | 0.426   | No           |
+| 9    | Permutation test: modules_k40    | +0.040  | 0.199   | No           |
+
+**Total: 0/21 comparisons significant at p < 0.05.**
+
+### Decision: NEGATIVE
+
+The PFAM genomic signal does not provide complementary information for ocean productivity (POC) prediction beyond what environmental features already capture. This conclusion holds across:
+- 8 module resolutions (k = 5 to 50)
+- 3 pre-selected POC-correlated feature sets (with and without leakage control)
+- Leakage-free within-fold feature selection
+- Residual prediction (env-only unexplained variance)
+- Ratio target engineering (POC:Chl-a)
+- Regional subsetting (Atlantic and Pacific separately)
+- 1000-permutation significance test
+
+### Manuscript Recommendation
+
+**NEGATIVE verdict: reframe the joint embedding section as a well-characterized negative result.**
+
+1. **Results section:** Remove or drastically reduce the joint embedding paragraph for POC. Keep 1-2 sentences stating the k-sweep and target-aware feature selection were performed, with the negative finding. The Chl-a and NFLH results (env-only vs env+PFAM) remain the focus.
+
+2. **Discussion section:** Reframe the POC paragraph as an informative negative:
+   - The 9,064 PFAM domains with measurable POC correlations (140 at |rho| > 0.2) demonstrate genuine biological association
+   - However, this association is fully mediated by shared environmental drivers (SST, MLD, nutrients)
+   - Once environmental features are included, PFAM features add no complementary predictive signal
+   - This was tested exhaustively across 8 module resolutions, target-aware selection, ratio targets, residual prediction, and regional subsetting (0/21 comparisons significant)
+
+3. **Supplement:** Move the comprehensive Ralph26 analysis summary to supplementary material as evidence of thorough investigation.
+
+4. **What NOT to claim:** Do not claim "PFAM domains are unrelated to POC" — they are correlated. The correct framing is that their POC-predictive information is redundant with environmental features.
+
+---
+
+## Key Scientific Insight
+
+The within-fold selection analysis (Task 5) reveals the mechanism of the negative result: 31 core PFAM domains are consistently the most POC-correlated across all spatial folds (Jaccard = 0.744), and all are negatively correlated (higher PFAM abundance = lower POC). This likely reflects oligotrophic communities with diverse protein domains but low productivity — a pattern already captured by SST, MLD, and nutrient concentrations. The PFAM-POC correlation is real but environmentally confounded.