Genome Atlas: expanding Sankey view with annotated / not-annotated bars
Browse filesReworks the Genome Atlas tree into an expanding Sankey diagram.
**Sankey bars.** Each group is a bar split into annotated (green) and not-annotated (gray) assemblies. Two-tone flows carry both parts from a parent into its lineages. Bar and flow heights follow assembly counts within a column, and each column is rescaled to the full height so deep lineages stay legible.
**Expands to the right.** Clicking a group adds its lineages as a new column instead of re-rooting the view, so every ancestor stays visible. Clicking a group in an earlier column replaces the columns after it. The selection is encoded as a path (`2759/33154/33208`, with `~offset` for expanded "Other lineages"), validated server-side, so the view stays stateless. Search and the shortcuts open the full lineage of a taxon. A selected lineage outside a column's top six is pinned into that column. The view auto-scrolls to the newest column.
Other details:
- Event payload key is `route`, because Gradio treats a `path` field as a file reference.
- "← Parent group" is now "← Collapse last group"; breadcrumbs collapse to any ancestor.
- Added `tests/test_taxonomy.py`: column expansion, lineage paths, aggregate expansion, invalid-path rejection. The full suite passes (19 tests).
- Updated the README and FEATURE_BACKLOG descriptions.
Tested locally in sample mode with Gradio 6.6.0.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- FEATURE_BACKLOG.md +1 -1
- README.md +2 -2
- atlas.css +20 -12
- atlas.js +11 -2
- taxonomy.py +193 -132
- tests/test_taxonomy.py +40 -0
|
@@ -40,7 +40,7 @@ Decisions and dependencies for implementation:
|
|
| 40 |
|
| 41 |
## F002 — Taxonomy map and navigation
|
| 42 |
|
| 43 |
-
Current implementation:
|
| 44 |
|
| 45 |
**Goal:** Provide a taxonomy-based view of the available genomes and annotations.
|
| 46 |
|
|
|
|
| 40 |
|
| 41 |
## F002 — Taxonomy map and navigation
|
| 42 |
|
| 43 |
+
Current implementation: a server-rendered SVG Sankey of the taxonomy leads the page, above annotation lookup. It supports click/keyboard expansion into columns that keep every ancestor visible, breadcrumbs, parent navigation, shortcuts, and scientific-name/taxon-ID search. Every route is restricted to Eukaryota. Coverage colors, counts, and percentages use a separate SQLite snapshot; see [refresh instructions](README.md#refresh-annotation-coverage). Linking taxa to annotation accessions remains future work.
|
| 44 |
|
| 45 |
**Goal:** Provide a taxonomy-based view of the available genomes and annotations.
|
| 46 |
|
|
@@ -17,13 +17,13 @@ Future features and open design decisions are tracked in the [feature backlog](F
|
|
| 17 |
|
| 18 |
## GenBank taxonomy
|
| 19 |
|
| 20 |
-
The landing page opens with an interactive **tree of eukaryotic life**, using NCBI's current GenBank assembly summary and taxonomy.
|
| 21 |
|
| 22 |
The viewer is restricted to **Eukaryota (NCBI taxid 2759) and its descendants**, including search, breadcrumbs, and parent navigation. The model predicts eukaryotic annotations; bacteria, archaea, and viruses are outside this display. The current eukaryotic snapshot contains **70,395 assemblies**, of which **17,561 (24.9%)** have published annotations. The underlying SQLite snapshot retains the full NCBI inventory for refresh/provenance, while the interface uses only the eukaryotic subtree and its denominators. Assemblies with unresolved taxonomy cannot be assigned to that subtree and are excluded.
|
| 23 |
|
| 24 |
The inventory accepts `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. These counts describe genome assemblies, not all GenBank nucleotide records. An assembly counts as having annotations when it occurs in the publication inventory; this includes partial assemblies and does not imply full base coverage. Exact accession versions are matched to the NCBI snapshot. Published versions outside that snapshot are excluded from percentages.
|
| 25 |
|
| 26 |
-
Click a group to
|
| 27 |
|
| 28 |
The graphic uses server-rendered SVG/HTML with scoped styles in `atlas.css` and delegated click/keyboard handlers in `atlas.js`; no external graphics library or image assets are required. Include these files when deploying.
|
| 29 |
|
|
|
|
| 17 |
|
| 18 |
## GenBank taxonomy
|
| 19 |
|
| 20 |
+
The landing page opens with an interactive **tree of eukaryotic life**, using NCBI's current GenBank assembly summary and taxonomy. It is drawn as a Sankey diagram: each group is a bar split into annotated (green) and not-annotated (gray) assemblies, and flows carry both parts from a group into its lineages. Counts and percentages appear beside the groups, with details on hover. The default **Genome atlas** tab contains the tree and its lineage controls. A separate **Database** tab holds accession search, segment visualization, and downloads, using the full published-file snapshot. A third **Wet Lab Experiments** tab is ready for experimental results and currently shows a clearly labeled coming-soon state. RefSeq exploration can be added beneath the tree. The three equal-width navigation options span the top of the page.
|
| 21 |
|
| 22 |
The viewer is restricted to **Eukaryota (NCBI taxid 2759) and its descendants**, including search, breadcrumbs, and parent navigation. The model predicts eukaryotic annotations; bacteria, archaea, and viruses are outside this display. The current eukaryotic snapshot contains **70,395 assemblies**, of which **17,561 (24.9%)** have published annotations. The underlying SQLite snapshot retains the full NCBI inventory for refresh/provenance, while the interface uses only the eukaryotic subtree and its denominators. Assemblies with unresolved taxonomy cannot be assigned to that subtree and are excluded.
|
| 23 |
|
| 24 |
The inventory accepts `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. These counts describe genome assemblies, not all GenBank nucleotide records. An assembly counts as having annotations when it occurs in the publication inventory; this includes partial assemblies and does not imply full base coverage. Exact accession versions are matched to the NCBI snapshot. Published versions outside that snapshot are excluded from percentages.
|
| 25 |
|
| 26 |
+
Click a group to expand its direct lineages into a new column on the right; every ancestor column stays visible, so the full path from Eukaryota is always on screen. Clicking a group in an earlier column replaces the columns after it. Use breadcrumbs, “Collapse last group”, or shortcuts to go back. Nodes support Enter/Space keyboard navigation. The opening view expands the actual animal/fungal ancestor, Opisthokonta. Each column shows six direct lineages, an expandable remainder, and the selected lineage if it falls outside the top six. Bar and flow heights follow assembly counts within a column; every column is rescaled to the full height. “Other lineages” is an explicitly labeled display aggregate, not an invented clade; its counts sum the remaining siblings. Repeated expansion reaches all siblings without truncating them. Scientific-name/taxon-ID search also reaches individual taxa. Column spacing does not encode time. The view scrolls horizontally, and jumps to the newest column after each expansion.
|
| 27 |
|
| 28 |
The graphic uses server-rendered SVG/HTML with scoped styles in `atlas.css` and delegated click/keyboard handlers in `atlas.js`; no external graphics library or image assets are required. Include these files when deploying.
|
| 29 |
|
|
@@ -22,23 +22,31 @@
|
|
| 22 |
.atlas-breadcrumb button:hover { text-decoration:underline; }
|
| 23 |
.crumb-divider { opacity:.4; }
|
| 24 |
.tree-scroll { overflow-x:auto; overscroll-behavior-x:contain; margin:0 -5px; }
|
| 25 |
-
.life-tree { display:block; width:100%;
|
| 26 |
-
.
|
| 27 |
-
.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
.tree-node { cursor:pointer; }
|
|
|
|
| 29 |
.tree-hit { fill:transparent; transition:fill .15s; }
|
| 30 |
-
.tree-node:hover .tree-hit,.tree-node:focus .tree-hit { fill:#
|
| 31 |
.tree-node:focus { outline:none; }
|
| 32 |
-
.tree-node:focus .
|
| 33 |
-
.tree-
|
| 34 |
-
.tree-
|
| 35 |
-
.tree-
|
| 36 |
-
.tree-count {
|
| 37 |
-
.tree-
|
|
|
|
|
|
|
| 38 |
.atlas-footer { display:flex; align-items:center; flex-wrap:wrap; gap:15px; justify-content:space-between; border-top:1px solid #dde4d5; padding-top:14px; color:#6c7a66; font-size:11px; line-height:1.6; }
|
| 39 |
.atlas-legend { display:flex; gap:7px; align-items:center; }
|
| 40 |
-
.atlas-
|
| 41 |
-
.atlas-
|
|
|
|
| 42 |
.atlas-date { opacity:.8; }
|
| 43 |
.atlas-footnote { font-size:10px; line-height:1.6; color:#77816e; margin-top:12px; max-width:860px; }
|
| 44 |
.atlas-tooltip { position:absolute; z-index:5; pointer-events:none; max-width:290px; padding:12px 15px; border-radius:12px; color:#f5f8ef; background:#1d4334; font-size:12px; line-height:1.7; box-shadow:0 5px 20px #18372122; }
|
|
|
|
| 22 |
.atlas-breadcrumb button:hover { text-decoration:underline; }
|
| 23 |
.crumb-divider { opacity:.4; }
|
| 24 |
.tree-scroll { overflow-x:auto; overscroll-behavior-x:contain; margin:0 -5px; }
|
| 25 |
+
.life-tree { display:block; width:100%; height:auto; overflow:visible; }
|
| 26 |
+
.flow { stroke:none; transition:opacity .15s; }
|
| 27 |
+
.flow-annotated { fill:#40916f; opacity:.42; }
|
| 28 |
+
.flow-missing { fill:#cfd5c9; opacity:.6; }
|
| 29 |
+
.bar-annotated { fill:#287557; }
|
| 30 |
+
.bar-missing { fill:#bcc4b5; }
|
| 31 |
+
.bar-outline { fill:none; stroke:#f7f8f0; stroke-width:1; }
|
| 32 |
.tree-node { cursor:pointer; }
|
| 33 |
+
.tree-node.is-direct { cursor:default; }
|
| 34 |
.tree-hit { fill:transparent; transition:fill .15s; }
|
| 35 |
+
.tree-node:hover .tree-hit,.tree-node:focus .tree-hit { fill:#e5ecdd99; }
|
| 36 |
.tree-node:focus { outline:none; }
|
| 37 |
+
.tree-node:focus .bar-outline { stroke:#173c30; stroke-width:2; }
|
| 38 |
+
.tree-node.is-selected .bar-outline { stroke:#173c30; stroke-width:1.5; }
|
| 39 |
+
.tree-node.is-sibling { opacity:.62; transition:opacity .15s; }
|
| 40 |
+
.tree-node.is-sibling:hover,.tree-node.is-sibling:focus { opacity:1; }
|
| 41 |
+
.tree-label,.tree-count { paint-order:stroke; stroke:#f7f8f0; stroke-width:4px; stroke-linejoin:round; }
|
| 42 |
+
.tree-label { font:600 13px Arial,Helvetica,sans-serif; fill:#254735; }
|
| 43 |
+
.tree-node.is-selected .tree-label { font-weight:700; fill:#173c30; }
|
| 44 |
+
.tree-count { font:11px Arial,Helvetica,sans-serif; fill:#697966; }
|
| 45 |
.atlas-footer { display:flex; align-items:center; flex-wrap:wrap; gap:15px; justify-content:space-between; border-top:1px solid #dde4d5; padding-top:14px; color:#6c7a66; font-size:11px; line-height:1.6; }
|
| 46 |
.atlas-legend { display:flex; gap:7px; align-items:center; }
|
| 47 |
+
.atlas-swatch { width:10px; height:10px; border-radius:2px; }
|
| 48 |
+
.atlas-annotated { background:#287557; }
|
| 49 |
+
.atlas-missing { background:#bcc4b5; }
|
| 50 |
.atlas-date { opacity:.8; }
|
| 51 |
.atlas-footnote { font-size:10px; line-height:1.6; color:#77816e; margin-top:12px; max-width:860px; }
|
| 52 |
.atlas-tooltip { position:absolute; z-index:5; pointer-events:none; max-width:290px; padding:12px 15px; border-radius:12px; color:#f5f8ef; background:#1d4334; font-size:12px; line-height:1.7; box-shadow:0 5px 20px #18372122; }
|
|
@@ -1,12 +1,12 @@
|
|
| 1 |
// Delegate to the component root so navigation still works after HTML updates.
|
| 2 |
function activate(event) {
|
| 3 |
-
const target = event.target.closest('[data-
|
| 4 |
if (!target || !element.contains(target)) return;
|
| 5 |
if (event.type === 'keydown' && !['Enter', ' '].includes(event.key)) return;
|
| 6 |
// Native buttons already synthesize a click on keyboard activation.
|
| 7 |
if (event.type === 'keydown' && target.tagName.toLowerCase() === 'button') return;
|
| 8 |
event.preventDefault();
|
| 9 |
-
trigger('click', {
|
| 10 |
}
|
| 11 |
element.addEventListener('click', activate);
|
| 12 |
element.addEventListener('keydown', activate);
|
|
@@ -26,3 +26,12 @@ element.addEventListener('pointerleave', () => {
|
|
| 26 |
const tooltip = element.querySelector('.atlas-tooltip');
|
| 27 |
if (tooltip) tooltip.hidden = true;
|
| 28 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
// Delegate to the component root so navigation still works after HTML updates.
|
| 2 |
function activate(event) {
|
| 3 |
+
const target = event.target.closest('[data-path]');
|
| 4 |
if (!target || !element.contains(target)) return;
|
| 5 |
if (event.type === 'keydown' && !['Enter', ' '].includes(event.key)) return;
|
| 6 |
// Native buttons already synthesize a click on keyboard activation.
|
| 7 |
if (event.type === 'keydown' && target.tagName.toLowerCase() === 'button') return;
|
| 8 |
event.preventDefault();
|
| 9 |
+
trigger('click', {route: target.dataset.path});
|
| 10 |
}
|
| 11 |
element.addEventListener('click', activate);
|
| 12 |
element.addEventListener('keydown', activate);
|
|
|
|
| 26 |
const tooltip = element.querySelector('.atlas-tooltip');
|
| 27 |
if (tooltip) tooltip.hidden = true;
|
| 28 |
});
|
| 29 |
+
// Newly expanded columns appear on the right; keep them in view after each update.
|
| 30 |
+
let shownRoute = element.querySelector('.life-tree')?.dataset.route;
|
| 31 |
+
new MutationObserver(() => {
|
| 32 |
+
const route = element.querySelector('.life-tree')?.dataset.route;
|
| 33 |
+
if (!route || route === shownRoute) return;
|
| 34 |
+
shownRoute = route;
|
| 35 |
+
const scroller = element.querySelector('.tree-scroll');
|
| 36 |
+
if (scroller) requestAnimationFrame(() => { scroller.scrollLeft = scroller.scrollWidth; });
|
| 37 |
+
}).observe(element, {childList: true, subtree: true, attributes: true, attributeFilter: ['data-route']});
|
|
@@ -20,15 +20,14 @@ DISPLAY_NAMES = {2759: "Eukaryotes", 33208: "Animals", 33090: "Green plants",
|
|
| 20 |
2763: "Red algae", 3041: "Green algae"}
|
| 21 |
|
| 22 |
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
return "#0f4e3b"
|
| 32 |
|
| 33 |
|
| 34 |
class Taxonomy:
|
|
@@ -57,132 +56,194 @@ class Taxonomy:
|
|
| 57 |
rows = conn.execute(EUK_TREE + "SELECT taxa.* FROM taxa JOIN euk USING(taxid) WHERE name LIKE ? ESCAPE '\\' ORDER BY total_count DESC LIMIT 100", (pattern,)).fetchall()
|
| 58 |
return [(f"{r['name']} · {r['total_count']:,} assemblies · taxid {r['taxid']}", str(r['taxid'])) for r in rows]
|
| 59 |
|
| 60 |
-
def
|
| 61 |
-
|
| 62 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
raise ValueError("Choose a eukaryotic taxon from this snapshot.")
|
| 64 |
with closing(self.connect()) as conn:
|
| 65 |
-
def
|
| 66 |
-
return
|
| 67 |
|
| 68 |
-
def node(row):
|
| 69 |
-
return dict(taxid=row['taxid'], offset=
|
| 70 |
label=DISPLAY_NAMES.get(row['taxid'], row['name']),
|
| 71 |
-
total=row['total_count'], covered=row.get('covered_total', 0),
|
| 72 |
|
| 73 |
def remainder(rows, parent, start):
|
| 74 |
return dict(taxid=parent, offset=start, name=f"Other lineages · {len(rows):,} groups (display aggregate)",
|
| 75 |
label=f"Other lineages ({len(rows):,})", total=sum(r['total_count'] for r in rows),
|
| 76 |
-
covered=sum(r.get('covered_total', 0) for r in rows),
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
if
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 172 |
percent = 100 * covered / total if self.coverage and total else 0
|
| 173 |
-
selected_name =
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
|
|
|
|
|
|
| 178 |
stamp = (self.coverage or self.metadata)['created_at'][:10]
|
| 179 |
if self.coverage:
|
| 180 |
summary = (f'<div class="atlas-kicker">{escape(selected_name)} · ASSEMBLY COVERAGE</div>'
|
| 181 |
f'<div class="atlas-summary"><strong>{covered:,}</strong><span>of {total:,}<br>assemblies annotated</span>'
|
| 182 |
f'<b class="atlas-badge">{percent:.1f}%</b></div>')
|
|
|
|
|
|
|
| 183 |
else:
|
| 184 |
summary = f'<div class="atlas-kicker">{escape(selected_name)}</div><div class="atlas-summary"><strong>{total:,}</strong><span>GenBank assemblies<br>Coverage unavailable</span></div>'
|
| 185 |
-
|
|
|
|
| 186 |
return f'''<section class="atlas" aria-label="Tree of eukaryotic life">
|
| 187 |
<header class="atlas-header">
|
| 188 |
<div><div class="atlas-kicker">HUGGING FACE BIO <span> / </span> GENOME ATLAS</div>
|
|
@@ -191,19 +252,18 @@ class Taxonomy:
|
|
| 191 |
<div class="atlas-summary-block">{summary}</div>
|
| 192 |
</header>
|
| 193 |
<nav class="atlas-nav" aria-label="Explore major lineages">
|
| 194 |
-
<div class="atlas-shortcuts"><button data-
|
| 195 |
-
<button data-
|
| 196 |
-
<button data-
|
| 197 |
-
<button class="atlas-back" data-
|
| 198 |
</nav>
|
| 199 |
<div class="atlas-breadcrumb">{breadcrumbs}</div>
|
| 200 |
-
<div class="tree-scroll" tabindex="0" aria-label="Interactive tree. Scroll horizontally
|
| 201 |
{leaf_note}<div class="atlas-tooltip" role="tooltip" hidden></div>
|
| 202 |
-
<footer class="atlas-footer">
|
| 203 |
-
<
|
| 204 |
-
<span>Click a group to unfold its lineages <span class="atlas-mobile-hint">· Swipe to explore</span></span>
|
| 205 |
<span class="atlas-date">Updated {stamp}</span></footer>
|
| 206 |
-
<div class="atlas-footnote">
|
| 207 |
</section>'''
|
| 208 |
|
| 209 |
|
|
@@ -219,7 +279,7 @@ def build_taxonomy_tab():
|
|
| 219 |
|
| 220 |
def explore_tree(evt: gr.EventData):
|
| 221 |
try:
|
| 222 |
-
return taxonomy.view(evt.
|
| 223 |
except (ValueError, TypeError, AttributeError) as exc:
|
| 224 |
raise gr.Error("Choose a eukaryotic group in this snapshot.") from exc
|
| 225 |
|
|
@@ -236,7 +296,7 @@ def build_taxonomy_tab():
|
|
| 236 |
return gr.Dropdown(choices=choices, value=None), f"{len(choices)} results (up to 100, largest first)." if choices else "No matching eukaryotic taxa in this snapshot."
|
| 237 |
|
| 238 |
def taxonomy_view(taxid):
|
| 239 |
-
return taxonomy.
|
| 240 |
|
| 241 |
matches.input(taxonomy_view, matches, tree)
|
| 242 |
find.click(taxonomy_search, query, [matches, status])
|
|
@@ -244,8 +304,9 @@ def build_taxonomy_tab():
|
|
| 244 |
with gr.Accordion("About the tree and coverage", open=False, elem_classes="atlas-disclosure"):
|
| 245 |
gr.Markdown("This tree follows **NCBI Taxonomy within Eukaryota**. Animals and fungi share the Opisthokonta branch; "
|
| 246 |
"green plants include green algae. ‘Other lineages’ combines the remaining siblings for display and can be expanded. "
|
| 247 |
-
"Each
|
| 248 |
-
"The
|
|
|
|
| 249 |
"Coverage means the fraction of current GenBank assembly versions with published annotations, including partial assemblies. "
|
| 250 |
"It does not measure the fraction of bases annotated. Parent counts include descendants once each. "
|
| 251 |
"Exact accession versions are matched. The Database tab searches a dated snapshot of all published annotation files.\n\n"
|
|
|
|
| 20 |
2763: "Red algae", 3041: "Green algae"}
|
| 21 |
|
| 22 |
|
| 23 |
+
COLUMN_LIMIT = 6
|
| 24 |
+
MAX_DEPTH = 80
|
| 25 |
+
DEFAULT_PATH = "2759/33154"
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def encode_path(steps):
|
| 29 |
+
"""Serialize selected (taxid, offset) pairs; an offset marks an expanded "Other lineages" aggregate."""
|
| 30 |
+
return "/".join(f"{t}~{o}" if o else str(t) for t, o in steps)
|
|
|
|
| 31 |
|
| 32 |
|
| 33 |
class Taxonomy:
|
|
|
|
| 56 |
rows = conn.execute(EUK_TREE + "SELECT taxa.* FROM taxa JOIN euk USING(taxid) WHERE name LIKE ? ESCAPE '\\' ORDER BY total_count DESC LIMIT 100", (pattern,)).fetchall()
|
| 57 |
return [(f"{r['name']} · {r['total_count']:,} assemblies · taxid {r['taxid']}", str(r['taxid'])) for r in rows]
|
| 58 |
|
| 59 |
+
def lineage(self, conn, taxid):
|
| 60 |
+
rows = [dict(conn.execute("SELECT * FROM taxa WHERE taxid=?", (taxid,)).fetchone())]
|
| 61 |
+
while rows[-1]['taxid'] != EUKARYOTA:
|
| 62 |
+
rows.append(dict(conn.execute("SELECT * FROM taxa WHERE taxid=?", (rows[-1]['parent_id'],)).fetchone()))
|
| 63 |
+
return rows[::-1]
|
| 64 |
+
|
| 65 |
+
def taxon_path(self, taxid):
|
| 66 |
+
"""Path that expands every ancestor of a taxon, for search results and shortcuts."""
|
| 67 |
+
taxid = int(taxid)
|
| 68 |
+
if taxid not in self.eukaryote_ids:
|
| 69 |
+
raise ValueError("Choose a eukaryotic taxon from this snapshot.")
|
| 70 |
+
with closing(self.connect()) as conn:
|
| 71 |
+
return encode_path((r['taxid'], 0) for r in self.lineage(conn, taxid))
|
| 72 |
+
|
| 73 |
+
def view_taxon(self, taxid):
|
| 74 |
+
return self.view(self.taxon_path(taxid))
|
| 75 |
+
|
| 76 |
+
def view(self, path=DEFAULT_PATH):
|
| 77 |
+
"""Render one column per expanded group. A path lists the selected node of each column."""
|
| 78 |
+
try:
|
| 79 |
+
steps = [(int(t), int(o or 0)) for t, _, o in (part.partition("~") for part in str(path).split("/"))]
|
| 80 |
+
except ValueError as exc:
|
| 81 |
+
raise ValueError("Choose a eukaryotic taxon from this snapshot.") from exc
|
| 82 |
+
if steps[0] != (EUKARYOTA, 0) or len(steps) > MAX_DEPTH or any(
|
| 83 |
+
t not in self.eukaryote_ids or o < 0 for t, o in steps):
|
| 84 |
raise ValueError("Choose a eukaryotic taxon from this snapshot.")
|
| 85 |
with closing(self.connect()) as conn:
|
| 86 |
+
def taxon(t):
|
| 87 |
+
return dict(conn.execute("SELECT * FROM taxa WHERE taxid=?", (t,)).fetchone())
|
| 88 |
|
| 89 |
+
def node(row, offset=0, **extra):
|
| 90 |
+
return dict(taxid=row['taxid'], offset=offset, name=row['name'],
|
| 91 |
label=DISPLAY_NAMES.get(row['taxid'], row['name']),
|
| 92 |
+
total=row['total_count'], covered=row.get('covered_total', 0), **extra)
|
| 93 |
|
| 94 |
def remainder(rows, parent, start):
|
| 95 |
return dict(taxid=parent, offset=start, name=f"Other lineages · {len(rows):,} groups (display aggregate)",
|
| 96 |
label=f"Other lineages ({len(rows):,})", total=sum(r['total_count'] for r in rows),
|
| 97 |
+
covered=sum(r.get('covered_total', 0) for r in rows), aggregate=True)
|
| 98 |
+
|
| 99 |
+
def column(step, selected):
|
| 100 |
+
"""Direct lineages of a selected node, keeping the next selection visible."""
|
| 101 |
+
parent, offset = step
|
| 102 |
+
rows = [dict(r) for r in conn.execute("SELECT * FROM taxa WHERE parent_id=? AND taxid!=? ORDER BY total_count DESC, taxid", (parent, parent))]
|
| 103 |
+
if offset and offset >= len(rows):
|
| 104 |
+
raise ValueError("This group has no further lineages.")
|
| 105 |
+
shown, rest = rows[offset:offset + COLUMN_LIMIT], rows[offset + COLUMN_LIMIT:]
|
| 106 |
+
pinned = [r for r in rest if selected and not selected[1] and r['taxid'] == selected[0]]
|
| 107 |
+
shown, rest = shown + pinned, [r for r in rest if r not in pinned]
|
| 108 |
+
nodes = [node(r) for r in shown]
|
| 109 |
+
if rest:
|
| 110 |
+
nodes.append(remainder(rest, parent, offset + COLUMN_LIMIT))
|
| 111 |
+
row = taxon(parent)
|
| 112 |
+
if not offset and row['direct_count'] and rows:
|
| 113 |
+
nodes.append(dict(taxid=parent, offset=0, name="Assemblies assigned directly to this taxon",
|
| 114 |
+
label="Direct assignments", total=row['direct_count'],
|
| 115 |
+
covered=row.get('covered_direct', 0), direct=True))
|
| 116 |
+
return nodes
|
| 117 |
+
|
| 118 |
+
columns = [[node(taxon(EUKARYOTA))]]
|
| 119 |
+
for depth, step in enumerate(steps[1:], 1):
|
| 120 |
+
if step[1] and (step[0] != steps[depth - 1][0] or step[1] <= steps[depth - 1][1]) or \
|
| 121 |
+
not step[1] and taxon(step[0])['parent_id'] != steps[depth - 1][0]:
|
| 122 |
+
raise ValueError("Choose a eukaryotic taxon from this snapshot.")
|
| 123 |
+
columns.append(column(steps[depth - 1], step))
|
| 124 |
+
tail = column(steps[-1], None)
|
| 125 |
+
if tail:
|
| 126 |
+
columns.append(tail)
|
| 127 |
+
for depth, nodes in enumerate(columns):
|
| 128 |
+
for item in nodes:
|
| 129 |
+
item['path'] = encode_path(steps[:depth] + [(item['taxid'], item['offset'])])
|
| 130 |
+
item['selected'] = depth < len(steps) and not item.get('direct') and \
|
| 131 |
+
(item['taxid'], item['offset']) == steps[depth]
|
| 132 |
+
if depth < len(steps) and not any(item['selected'] for item in nodes):
|
| 133 |
+
raise ValueError("Choose a eukaryotic taxon from this snapshot.")
|
| 134 |
+
lineage = [taxon(t) for t, _ in steps]
|
| 135 |
+
shortcuts = {t: encode_path((r['taxid'], 0) for r in self.lineage(conn, t)) for t in (33208, 33090, 4751)}
|
| 136 |
+
return self.render(columns, steps, lineage, shortcuts, has_children=bool(tail))
|
| 137 |
+
|
| 138 |
+
def render(self, columns, steps, lineage, shortcuts, has_children):
|
| 139 |
+
top, chart, spacing, bar = 24, 460, 250, 14
|
| 140 |
+
width = max(830, 40 + (len(columns) - 1) * spacing + bar + 250)
|
| 141 |
+
height = top * 2 + chart
|
| 142 |
+
|
| 143 |
+
def stack(nodes):
|
| 144 |
+
"""Sankey layout: bar heights follow assembly counts, with room for every label."""
|
| 145 |
+
gap, slot = (12 if len(nodes) > 1 else 0), 36
|
| 146 |
+
fixed, scale = set(), 0
|
| 147 |
+
for _ in range(len(nodes) + 1):
|
| 148 |
+
free = chart - gap * (len(nodes) - 1) - slot * len(fixed)
|
| 149 |
+
flexible = sum(n['total'] for k, n in enumerate(nodes) if k not in fixed)
|
| 150 |
+
scale = free / flexible if flexible else 0
|
| 151 |
+
small = fixed | {k for k, n in enumerate(nodes) if n['total'] * scale < slot}
|
| 152 |
+
if small == fixed:
|
| 153 |
+
break
|
| 154 |
+
fixed = small
|
| 155 |
+
y = top
|
| 156 |
+
for k, n in enumerate(nodes):
|
| 157 |
+
room = slot if k in fixed else n['total'] * scale
|
| 158 |
+
n['h'] = max(3, n['total'] * scale)
|
| 159 |
+
n['y'] = y + (room - n['h']) / 2
|
| 160 |
+
y += room + gap
|
| 161 |
+
|
| 162 |
+
def share(n):
|
| 163 |
+
return n['covered'] / n['total'] if self.coverage and n['total'] else 0
|
| 164 |
+
|
| 165 |
+
def ribbon(x0, a0, a1, x1, b0, b1, kind):
|
| 166 |
+
bend = (x0 + x1) / 2
|
| 167 |
+
return (f'<path d="M{x0},{a0:.1f} C{bend},{a0:.1f} {bend},{b0:.1f} {x1},{b0:.1f} '
|
| 168 |
+
f'L{x1},{b1:.1f} C{bend},{b1:.1f} {bend},{a1:.1f} {x0},{a1:.1f} Z" class="flow flow-{kind}"/>')
|
| 169 |
+
|
| 170 |
+
flows, nodes = [], []
|
| 171 |
+
for depth, column in enumerate(columns):
|
| 172 |
+
stack(column)
|
| 173 |
+
x = 40 + depth * spacing
|
| 174 |
+
for n in column:
|
| 175 |
+
n['x'] = x
|
| 176 |
+
for depth, column in enumerate(columns[:-1]):
|
| 177 |
+
parent = next((n for n in column if n['selected']), None)
|
| 178 |
+
children = columns[depth + 1]
|
| 179 |
+
if not parent:
|
| 180 |
+
continue
|
| 181 |
+
total = sum(c['total'] for c in children) or 1
|
| 182 |
+
cursor = parent['y']
|
| 183 |
+
for child in children:
|
| 184 |
+
a0 = cursor
|
| 185 |
+
a1 = cursor + parent['h'] * child['total'] / total
|
| 186 |
+
cursor = a1
|
| 187 |
+
f = share(child)
|
| 188 |
+
am, bm = a0 + f * (a1 - a0), child['y'] + f * child['h']
|
| 189 |
+
x0, x1 = parent['x'] + bar, child['x']
|
| 190 |
+
if f > 0:
|
| 191 |
+
flows.append(ribbon(x0, a0, am, x1, child['y'], bm, "annotated"))
|
| 192 |
+
if f < 1:
|
| 193 |
+
flows.append(ribbon(x0, am, a1, x1, bm, child['y'] + child['h'], "missing"))
|
| 194 |
+
|
| 195 |
+
for depth, column in enumerate(columns):
|
| 196 |
+
on_path = depth < len(steps)
|
| 197 |
+
for n in column:
|
| 198 |
+
x, y, h = n['x'], n['y'], n['h']
|
| 199 |
+
f = share(n)
|
| 200 |
+
percent = 100 * f
|
| 201 |
+
detail = (f"{n['name']} · {n['covered']:,} of {n['total']:,} assemblies have published annotations ({percent:.2f}%). Includes partial assemblies."
|
| 202 |
+
if self.coverage else f"{n['name']} · {n['total']:,} assemblies. Coverage inventory unavailable.")
|
| 203 |
+
if n.get('aggregate'):
|
| 204 |
+
detail += " Display aggregate, not a taxonomic clade. Activate to see its lineages."
|
| 205 |
+
elif not n.get('direct') and not n['selected']:
|
| 206 |
+
detail += " Activate to expand this group to the right."
|
| 207 |
+
action = "" if n.get('direct') else f'data-path="{n["path"]}" role="button" tabindex="0"'
|
| 208 |
+
classes = "tree-node" + (" is-selected" if n['selected'] else " is-sibling" if on_path else "") + \
|
| 209 |
+
(" is-direct" if n.get('direct') else "")
|
| 210 |
+
nodes.append(f'<g class="{classes}" {action} aria-label="{escape(detail, quote=True)}" data-detail="{escape(detail, quote=True)}"><title>{escape(detail)}</title>')
|
| 211 |
+
cy = y + h / 2
|
| 212 |
+
nodes.append(f'<rect x="{x - 6}" y="{min(y, cy - 18):.1f}" width="{spacing - 40}" height="{max(h, 36):.1f}" rx="10" class="tree-hit"/>')
|
| 213 |
+
green = h * f
|
| 214 |
+
if green > 0:
|
| 215 |
+
nodes.append(f'<rect x="{x}" y="{y:.1f}" width="{bar}" height="{green:.1f}" class="bar bar-annotated"/>')
|
| 216 |
+
if green < h:
|
| 217 |
+
nodes.append(f'<rect x="{x}" y="{y + green:.1f}" width="{bar}" height="{h - green:.1f}" class="bar bar-missing"/>')
|
| 218 |
+
nodes.append(f'<rect x="{x}" y="{y:.1f}" width="{bar}" height="{h:.1f}" class="bar-outline"/>')
|
| 219 |
+
label = n['label'] if len(n['label']) <= 24 else n['label'][:22] + "…"
|
| 220 |
+
stats = f"{percent:.1f}% · {n['covered']:,} / {n['total']:,}" if self.coverage else f"{n['total']:,} assemblies"
|
| 221 |
+
nodes.append(f'<text x="{x + bar + 8}" y="{cy - 3:.1f}" class="tree-label">{escape(label)}</text>'
|
| 222 |
+
f'<text x="{x + bar + 8}" y="{cy + 13:.1f}" class="tree-count">{stats}</text></g>')
|
| 223 |
+
svg = (f'<svg viewBox="0 0 {width} {height}" class="life-tree" data-route="{encode_path(steps)}" style="min-width:{width if width > 1040 else 760}px" '
|
| 224 |
+
f'aria-label="Eukaryotic taxonomy and annotation coverage">' + ''.join(flows) + ''.join(nodes) + '</svg>')
|
| 225 |
+
|
| 226 |
+
focus = next(n for n in columns[len(steps) - 1] if n['selected'])
|
| 227 |
+
total, covered = focus['total'], focus['covered']
|
| 228 |
percent = 100 * covered / total if self.coverage and total else 0
|
| 229 |
+
selected_name = f"Other lineages of {lineage[-1]['name']}" if focus.get('aggregate') else focus['label']
|
| 230 |
+
crumbs = []
|
| 231 |
+
for depth, (row, (_, offset)) in enumerate(zip(lineage, steps)):
|
| 232 |
+
name = "Other lineages" if offset else DISPLAY_NAMES.get(row['taxid'], row['name'])
|
| 233 |
+
crumbs.append(f'<button data-path="{encode_path(steps[:depth + 1])}">{escape(name)}</button>')
|
| 234 |
+
breadcrumbs = ' <span class="crumb-divider">/</span> '.join(crumbs)
|
| 235 |
+
parent = encode_path(steps[:-1] or steps)
|
| 236 |
stamp = (self.coverage or self.metadata)['created_at'][:10]
|
| 237 |
if self.coverage:
|
| 238 |
summary = (f'<div class="atlas-kicker">{escape(selected_name)} · ASSEMBLY COVERAGE</div>'
|
| 239 |
f'<div class="atlas-summary"><strong>{covered:,}</strong><span>of {total:,}<br>assemblies annotated</span>'
|
| 240 |
f'<b class="atlas-badge">{percent:.1f}%</b></div>')
|
| 241 |
+
legend = ('<div class="atlas-legend"><i class="atlas-swatch atlas-annotated"></i><span>Annotated</span>'
|
| 242 |
+
'<i class="atlas-swatch atlas-missing"></i><span>Not annotated</span></div>')
|
| 243 |
else:
|
| 244 |
summary = f'<div class="atlas-kicker">{escape(selected_name)}</div><div class="atlas-summary"><strong>{total:,}</strong><span>GenBank assemblies<br>Coverage unavailable</span></div>'
|
| 245 |
+
legend = '<div class="atlas-legend"><i class="atlas-swatch atlas-missing"></i><span>Assemblies · coverage unavailable</span></div>'
|
| 246 |
+
leaf_note = '<p class="atlas-leaf-note">This is a terminal taxon in the assembly snapshot. Use the breadcrumb to explore its relatives.</p>' if not has_children else ''
|
| 247 |
return f'''<section class="atlas" aria-label="Tree of eukaryotic life">
|
| 248 |
<header class="atlas-header">
|
| 249 |
<div><div class="atlas-kicker">HUGGING FACE BIO <span> / </span> GENOME ATLAS</div>
|
|
|
|
| 252 |
<div class="atlas-summary-block">{summary}</div>
|
| 253 |
</header>
|
| 254 |
<nav class="atlas-nav" aria-label="Explore major lineages">
|
| 255 |
+
<div class="atlas-shortcuts"><button data-path="{DEFAULT_PATH}">All eukaryotes</button>
|
| 256 |
+
<button data-path="{shortcuts[33208]}">Animals</button><button data-path="{shortcuts[33090]}">Green plants</button>
|
| 257 |
+
<button data-path="{shortcuts[4751]}">Fungi</button></div>
|
| 258 |
+
<button class="atlas-back" data-path="{parent}">← Collapse last group</button>
|
| 259 |
</nav>
|
| 260 |
<div class="atlas-breadcrumb">{breadcrumbs}</div>
|
| 261 |
+
<div class="tree-scroll" tabindex="0" aria-label="Interactive tree. Scroll horizontally to see deeper lineages.">{svg}</div>
|
| 262 |
{leaf_note}<div class="atlas-tooltip" role="tooltip" hidden></div>
|
| 263 |
+
<footer class="atlas-footer">{legend}
|
| 264 |
+
<span>Click a group to expand it to the right <span class="atlas-mobile-hint">· Swipe to explore</span></span>
|
|
|
|
| 265 |
<span class="atlas-date">Updated {stamp}</span></footer>
|
| 266 |
+
<div class="atlas-footnote">Bar and flow heights are proportional to assembly counts within each column; each column is rescaled to fill the height, so compare sizes within a column. Coverage counts assemblies with published annotations, including partial assemblies.</div>
|
| 267 |
</section>'''
|
| 268 |
|
| 269 |
|
|
|
|
| 279 |
|
| 280 |
def explore_tree(evt: gr.EventData):
|
| 281 |
try:
|
| 282 |
+
return taxonomy.view(evt.route)
|
| 283 |
except (ValueError, TypeError, AttributeError) as exc:
|
| 284 |
raise gr.Error("Choose a eukaryotic group in this snapshot.") from exc
|
| 285 |
|
|
|
|
| 296 |
return gr.Dropdown(choices=choices, value=None), f"{len(choices)} results (up to 100, largest first)." if choices else "No matching eukaryotic taxa in this snapshot."
|
| 297 |
|
| 298 |
def taxonomy_view(taxid):
|
| 299 |
+
return taxonomy.view_taxon(taxid) if taxid else taxonomy.view()
|
| 300 |
|
| 301 |
matches.input(taxonomy_view, matches, tree)
|
| 302 |
find.click(taxonomy_search, query, [matches, status])
|
|
|
|
| 304 |
with gr.Accordion("About the tree and coverage", open=False, elem_classes="atlas-disclosure"):
|
| 305 |
gr.Markdown("This tree follows **NCBI Taxonomy within Eukaryota**. Animals and fungi share the Opisthokonta branch; "
|
| 306 |
"green plants include green algae. ‘Other lineages’ combines the remaining siblings for display and can be expanded. "
|
| 307 |
+
"Each expanded group adds a column on the right with up to six direct lineages plus any remainder, so every ancestor stays visible. "
|
| 308 |
+
"The opening view also expands the animal/fungal branch. Bars and flows are drawn like a Sankey diagram: "
|
| 309 |
+
"heights follow **actual assembly counts** within each column, split into annotated and not-annotated assemblies.\n\n"
|
| 310 |
"Coverage means the fraction of current GenBank assembly versions with published annotations, including partial assemblies. "
|
| 311 |
"It does not measure the fraction of bases annotated. Parent counts include descendants once each. "
|
| 312 |
"Exact accession versions are matched. The Database tab searches a dated snapshot of all published annotation files.\n\n"
|
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import re
|
| 2 |
+
import unittest
|
| 3 |
+
|
| 4 |
+
from taxonomy import DATABASE, DEFAULT_PATH, Taxonomy
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
@unittest.skipUnless(DATABASE.exists(), "taxonomy snapshot unavailable")
|
| 8 |
+
class TaxonomyTests(unittest.TestCase):
|
| 9 |
+
@classmethod
|
| 10 |
+
def setUpClass(cls):
|
| 11 |
+
cls.taxonomy = Taxonomy()
|
| 12 |
+
|
| 13 |
+
def columns(self, html):
|
| 14 |
+
return sorted({float(x) for x in re.findall(r'<rect x="([\d.]+)" y="[\d.]+" width="14"', html)})
|
| 15 |
+
|
| 16 |
+
def test_expansion_keeps_every_ancestor_column(self):
|
| 17 |
+
self.assertEqual(len(self.columns(self.taxonomy.view("2759"))), 2)
|
| 18 |
+
self.assertEqual(len(self.columns(self.taxonomy.view(DEFAULT_PATH))), 3)
|
| 19 |
+
html = self.taxonomy.view("2759/33154/33208")
|
| 20 |
+
self.assertEqual(len(self.columns(html)), 4)
|
| 21 |
+
self.assertEqual(html.count('class="tree-node is-selected"'), 3)
|
| 22 |
+
|
| 23 |
+
def test_search_path_expands_lineage(self):
|
| 24 |
+
path = self.taxonomy.taxon_path(9606)
|
| 25 |
+
self.assertTrue(path.startswith("2759/33154/33208/"))
|
| 26 |
+
self.assertTrue(path.endswith("/9606"))
|
| 27 |
+
self.assertIn(f'data-route="{path}"', self.taxonomy.view_taxon(9606))
|
| 28 |
+
|
| 29 |
+
def test_other_lineages_expand_as_columns(self):
|
| 30 |
+
html = self.taxonomy.view("2759/2759~6")
|
| 31 |
+
self.assertIn('data-path="2759/2759~6/2759~12"', html)
|
| 32 |
+
|
| 33 |
+
def test_rejects_paths_outside_the_tree(self):
|
| 34 |
+
for path in ("1", "33208", "2759/33208", "2759/2759~3", "2759/9606", "abc"):
|
| 35 |
+
with self.assertRaises(ValueError, msg=path):
|
| 36 |
+
self.taxonomy.view(path)
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
if __name__ == "__main__":
|
| 40 |
+
unittest.main()
|