Fold the views into one panel, right-size the controls, and open Database from a group
#7
by lvwerra HF Staff - opened
- FEATURE_BACKLOG.md +1 -1
- README.md +4 -2
- app.css +23 -10
- app.py +12 -2
- atlas.css +2 -2
- data/taxonomy.sqlite +2 -2
- refresh_assemblies.py +96 -0
- taxonomy.py +127 -28
- tests/test_taxonomy.py +26 -0
FEATURE_BACKLOG.md
CHANGED
|
@@ -55,7 +55,7 @@ Decisions and dependencies for implementation:
|
|
| 55 |
|
| 56 |
- Implemented: eukaryotic tree with bounded subtrees and exact, expandable “Other lineages” aggregates; NCBI taxonomy with source checksums and dates; merged IDs resolved. Unresolved taxonomy is excluded from the eukaryotic view.
|
| 57 |
- Implemented: English names beside every scientific name, from NCBI Taxonomy common names plus a curated gloss table (`refresh_common_names.py`).
|
| 58 |
-
-
|
| 59 |
- Implemented: clade silhouettes from PhyloPic, filtered to CC0/Public Domain Mark/CC BY at build time (`refresh_icons.py`), with every CC BY artist credited in the interface.
|
| 60 |
|
| 61 |
## F003 — Comparison with RefSeq annotations
|
|
|
|
| 55 |
|
| 56 |
- Implemented: eukaryotic tree with bounded subtrees and exact, expandable “Other lineages” aggregates; NCBI taxonomy with source checksums and dates; merged IDs resolved. Unresolved taxonomy is excluded from the eukaryotic view.
|
| 57 |
- Implemented: English names beside every scientific name, from NCBI Taxonomy common names plus a curated gloss table (`refresh_common_names.py`).
|
| 58 |
+
- Implemented: a **Show database results** jump from the selected group into the Database tab, backed by the `assemblies` table from `refresh_assemblies.py`. It hands over one annotated accession; listing every assembly in a group, and distinguishing partial from complete base coverage, remain open.
|
| 59 |
- Implemented: clade silhouettes from PhyloPic, filtered to CC0/Public Domain Mark/CC BY at build time (`refresh_icons.py`), with every CC BY artist credited in the interface.
|
| 60 |
|
| 61 |
## F003 — Comparison with RefSeq annotations
|
README.md
CHANGED
|
@@ -17,13 +17,13 @@ Future features and open design decisions are tracked in the [feature backlog](F
|
|
| 17 |
|
| 18 |
## GenBank taxonomy
|
| 19 |
|
| 20 |
-
The landing page opens with an interactive **tree of eukaryotic life**, using NCBI's current GenBank assembly summary and taxonomy. It is drawn as a Sankey diagram running **down the page**: each group is a horizontal bar split into annotated (green) and not-annotated (gray) assemblies, and flows carry both parts from a group into its lineages on the level below. Expanding a group adds a level underneath rather than a column to the right, so a full lineage — eukaryotes down to humans is about thirty levels — is one ordinary page scroll instead of an endless sideways one. The chart itself only scrolls on screens too narrow to keep the labels legible. Shortcut buttons above the tree jump straight to a lineage: animals, mammals, humans, birds, ray-finned fish, insects, jellyfish, green plants, flowering plants and fungi.
|
| 21 |
|
| 22 |
The viewer is restricted to **Eukaryota (NCBI taxid 2759) and its descendants**, including search, breadcrumbs, and parent navigation. The model predicts eukaryotic annotations; bacteria, archaea, and viruses are outside this display. The current eukaryotic snapshot contains **70,395 assemblies**, of which **17,561 (24.9%)** have published annotations. The underlying SQLite snapshot retains the full NCBI inventory for refresh/provenance, while the interface uses only the eukaryotic subtree and its denominators. Assemblies with unresolved taxonomy cannot be assigned to that subtree and are excluded.
|
| 23 |
|
| 24 |
The inventory accepts `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. These counts describe genome assemblies, not all GenBank nucleotide records. An assembly counts as having annotations when it occurs in the publication inventory; this includes partial assemblies and does not imply full base coverage. Exact accession versions are matched to the NCBI snapshot. Published versions outside that snapshot are excluded from percentages.
|
| 25 |
|
| 26 |
-
Click a group to
|
| 27 |
|
| 28 |
English names come from `data/common_names.json`, built by `refresh_common_names.py` from NCBI Taxonomy's `genbank common name` and `common name` entries (about 9,200 taxa) plus a curated table of roughly 190 short glosses for the large unranked clades NCBI leaves unnamed. Glosses are plain-language summaries of the living members of a clade, not formal synonyms; the scientific name stays the primary label and a gloss that merely repeats it is dropped at build time. Rebuild it with `python refresh_common_names.py` once `.cache/coverage/taxdump.tar.gz` is present; the build fails if a curated taxid no longer matches its scientific name in the snapshot.
|
| 29 |
|
|
@@ -33,6 +33,8 @@ The graphic uses server-rendered SVG/HTML with scoped styles in `atlas.css` and
|
|
| 33 |
|
| 34 |
The annotation workspace shares the atlas palette and typography through `style.py` and `app.css`: warm panels, serif section headings, green controls, and matching tables, downloads, and disclosures. CDS plots use a solid green positive strand and a dashed warm-brown negative strand; binary labels use green. The fixed light palette stays consistent with the landing graphic under either OS color preference. Include both style files when deploying; Gradio 6 receives the theme and stylesheet in `launch()`.
|
| 35 |
|
|
|
|
|
|
|
| 36 |
### Refresh the taxonomy snapshot
|
| 37 |
|
| 38 |
```bash
|
|
|
|
| 17 |
|
| 18 |
## GenBank taxonomy
|
| 19 |
|
| 20 |
+
The landing page opens with an interactive **tree of eukaryotic life**, using NCBI's current GenBank assembly summary and taxonomy. It is drawn as a Sankey diagram running **down the page**: each group is a horizontal bar split into annotated (green) and not-annotated (gray) assemblies, and flows carry both parts from a group into its lineages on the level below. Expanding a group adds a level underneath rather than a column to the right, so a full lineage — eukaryotes down to humans is about thirty levels — is one ordinary page scroll instead of an endless sideways one. The chart itself only scrolls on screens too narrow to keep the labels legible. Shortcut buttons above the tree jump straight to a lineage: animals, mammals, humans, birds, ray-finned fish, insects, jellyfish, green plants, flowering plants and fungi. Each group shows a silhouette, its scientific name and its English name; counts and percentages live in the tooltip, which appears over a group or over the flux arriving at it. The flux feeding the current selection stays highlighted, so the trunk you have walked reads against the lineages you passed over. Each group is labelled with its scientific name and, where one exists, a plain-English name underneath, so the Latin backbone of the tree (Opisthokonta, Eumetazoa, Ecdysozoa) stays readable without prior knowledge. The two views sit inside one curved panel, switched by a full-width toggle at the top: the **Genome Atlas** holds the tree and its lineage controls, and **Database** holds accession search, segment visualization, and downloads over the full published-file snapshot. A **Show database results** button carries the current group across, handing the Database tab one annotated assembly from the selected lineage; it says so plainly when a group has none. RefSeq exploration can be added beneath the tree.
|
| 21 |
|
| 22 |
The viewer is restricted to **Eukaryota (NCBI taxid 2759) and its descendants**, including search, breadcrumbs, and parent navigation. The model predicts eukaryotic annotations; bacteria, archaea, and viruses are outside this display. The current eukaryotic snapshot contains **70,395 assemblies**, of which **17,561 (24.9%)** have published annotations. The underlying SQLite snapshot retains the full NCBI inventory for refresh/provenance, while the interface uses only the eukaryotic subtree and its denominators. Assemblies with unresolved taxonomy cannot be assigned to that subtree and are excluded.
|
| 23 |
|
| 24 |
The inventory accepts `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. These counts describe genome assemblies, not all GenBank nucleotide records. An assembly counts as having annotations when it occurs in the publication inventory; this includes partial assemblies and does not imply full base coverage. Exact accession versions are matched to the NCBI snapshot. Published versions outside that snapshot are excluded from percentages.
|
| 25 |
|
| 26 |
+
Click a group to open its direct lineages as a new level below; every ancestor level stays visible, so the full path from Eukaryota is always on screen. Clicking a group in an earlier level replaces the levels after it. Use breadcrumbs, “Collapse last group”, or shortcuts to go back. Nodes support Enter/Space keyboard navigation. The opening view expands the actual animal/fungal ancestor, Opisthokonta. Each level shows six direct lineages, an expandable remainder, and the selected lineage if it falls outside the top six. Bar and flow widths follow assembly counts within a level; every level is rescaled to the full width. “Other lineages” is an explicitly labeled display aggregate, not an invented clade; its counts sum the remaining siblings. Repeated expansion reaches all siblings without truncating them. Search reaches individual taxa by scientific name, taxon ID, or English name, so "sponges", "jellyfish" or "house mouse" all work; matches appear as a list with the closest first and already selected, and Enter opens it. Level spacing does not encode time. Expanding nudges the page down only when the new level has fallen past the viewport.
|
| 27 |
|
| 28 |
English names come from `data/common_names.json`, built by `refresh_common_names.py` from NCBI Taxonomy's `genbank common name` and `common name` entries (about 9,200 taxa) plus a curated table of roughly 190 short glosses for the large unranked clades NCBI leaves unnamed. Glosses are plain-language summaries of the living members of a clade, not formal synonyms; the scientific name stays the primary label and a gloss that merely repeats it is dropped at build time. Rebuild it with `python refresh_common_names.py` once `.cache/coverage/taxdump.tar.gz` is present; the build fails if a curated taxid no longer matches its scientific name in the snapshot.
|
| 29 |
|
|
|
|
| 33 |
|
| 34 |
The annotation workspace shares the atlas palette and typography through `style.py` and `app.css`: warm panels, serif section headings, green controls, and matching tables, downloads, and disclosures. CDS plots use a solid green positive strand and a dashed warm-brown negative strand; binary labels use green. The fixed light palette stays consistent with the landing graphic under either OS color preference. Include both style files when deploying; Gradio 6 receives the theme and stylesheet in `launch()`.
|
| 35 |
|
| 36 |
+
The **Show database results** jump needs a taxon-to-accession table, which `refresh_assemblies.py` writes into `data/taxonomy.sqlite` as an `assemblies` table: one row per annotated eukaryotic assembly, about 33,600 of them. It reads the NCBI assembly summary and the existing `data/coverage.json`, and deliberately leaves the counts alone — rebuilding those here would re-read a newer summary and shift every total in the interface away from the published snapshot. Only annotated assemblies are stored, so the jump never offers an accession with nothing behind it.
|
| 37 |
+
|
| 38 |
### Refresh the taxonomy snapshot
|
| 39 |
|
| 40 |
```bash
|
app.css
CHANGED
|
@@ -1,7 +1,8 @@
|
|
| 1 |
/* Global component styling, paired with the landing graphic's scoped atlas.css. */
|
| 2 |
.gradio-container { max-width:1280px!important; margin:auto!important; padding:24px 20px 40px!important; color-scheme:light; }
|
| 3 |
#eukaryotic-atlas { padding:0!important; border:0!important; background:transparent!important; }
|
| 4 |
-
.gradio-container .atlas-panel { background:
|
|
|
|
| 5 |
.workspace-heading { display:flex; align-items:flex-start; gap:14px; padding:0 0 4px; }
|
| 6 |
.workspace-step { width:32px; height:32px; border:1px solid #cdddc8; background:#eaf1e2; border-radius:50%; display:flex; align-items:center; justify-content:center; flex-shrink:0; font:500 11px Arial,sans-serif; color:#527261; margin-top:4px; }
|
| 7 |
.workspace-heading h2 { margin:0 0 8px; font:400 32px/1.15 Georgia,'Times New Roman',serif; letter-spacing:-.7px; color:#173c30; }
|
|
@@ -17,8 +18,17 @@
|
|
| 17 |
.gradio-container button:focus-visible,.gradio-container a:focus-visible { outline:2px solid #327555; outline-offset:3px; }
|
| 18 |
.gradio-container .prose a { text-underline-offset:3px; }
|
| 19 |
.gradio-container .prose code { color:#315641; }
|
| 20 |
-
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
#annotation-plot { border:1px solid #e0e6d8; background:#fffefa; border-radius:16px; overflow:hidden; }
|
| 23 |
#annotation-plot .plot-container { background:#fffefa; }
|
| 24 |
#annotation-examples { font-size:12px; }
|
|
@@ -29,7 +39,8 @@
|
|
| 29 |
.workspace-footer a:hover { text-decoration:underline; }
|
| 30 |
@media(max-width:700px) {
|
| 31 |
.gradio-container { padding:12px 10px 24px!important; }
|
| 32 |
-
.gradio-container .atlas-panel {
|
|
|
|
| 33 |
.workspace-heading h2 { font-size:28px; }
|
| 34 |
.workspace-heading { gap:10px; }
|
| 35 |
.gradio-container .action-button { width:100%; }
|
|
@@ -38,12 +49,14 @@
|
|
| 38 |
|
| 39 |
/* Full-width navigation: one equal segment for each view. */
|
| 40 |
.gradio-container { width:100%!important; min-width:0!important; box-sizing:border-box; }
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
#atlas-navigation
|
| 44 |
-
#atlas-navigation > .tab-wrapper
|
| 45 |
-
#atlas-navigation > .tab-wrapper > .tab-container
|
| 46 |
-
#atlas-navigation > .tab-wrapper > .tab-container > button
|
|
|
|
|
|
|
| 47 |
#atlas-navigation > .tab-wrapper::before, #atlas-navigation > .tab-wrapper::after,
|
| 48 |
#atlas-navigation > .tab-wrapper > .tab-container::before, #atlas-navigation > .tab-wrapper > .tab-container::after,
|
| 49 |
#atlas-navigation > .tab-wrapper > .tab-container > button::before, #atlas-navigation > .tab-wrapper > .tab-container > button::after { content:none!important; display:none!important; }
|
|
|
|
| 1 |
/* Global component styling, paired with the landing graphic's scoped atlas.css. */
|
| 2 |
.gradio-container { max-width:1280px!important; margin:auto!important; padding:24px 20px 40px!important; color-scheme:light; }
|
| 3 |
#eukaryotic-atlas { padding:0!important; border:0!important; background:transparent!important; }
|
| 4 |
+
.gradio-container .atlas-panel { background:transparent; border:0; border-radius:0; padding:22px 20px; box-shadow:none; gap:16px; margin-top:0; }
|
| 5 |
+
.gradio-container .atlas-panel + .atlas-panel { border-top:1px solid #e3e9dc; }
|
| 6 |
.workspace-heading { display:flex; align-items:flex-start; gap:14px; padding:0 0 4px; }
|
| 7 |
.workspace-step { width:32px; height:32px; border:1px solid #cdddc8; background:#eaf1e2; border-radius:50%; display:flex; align-items:center; justify-content:center; flex-shrink:0; font:500 11px Arial,sans-serif; color:#527261; margin-top:4px; }
|
| 8 |
.workspace-heading h2 { margin:0 0 8px; font:400 32px/1.15 Georgia,'Times New Roman',serif; letter-spacing:-.7px; color:#173c30; }
|
|
|
|
| 18 |
.gradio-container button:focus-visible,.gradio-container a:focus-visible { outline:2px solid #327555; outline-offset:3px; }
|
| 19 |
.gradio-container .prose a { text-underline-offset:3px; }
|
| 20 |
.gradio-container .prose code { color:#315641; }
|
| 21 |
+
/* Gradio's block padding sat around the whole disclosure, so a one-line title
|
| 22 |
+
occupied 76px. The frame is the button; the body carries its own padding. */
|
| 23 |
+
.gradio-container .atlas-disclosure { border:1px solid #e3e9dc; border-radius:12px; box-shadow:none; margin-top:8px; padding:0; background:#fbfcf7; }
|
| 24 |
+
.gradio-container .atlas-disclosure > button.label-wrap { padding:11px 14px; min-height:0; }
|
| 25 |
+
.gradio-container .atlas-disclosure > button.label-wrap > span:first-child { font:600 14px/1.4 Arial,Helvetica,sans-serif; color:#2a4a38; }
|
| 26 |
+
.gradio-container .atlas-disclosure > button.label-wrap > span.icon { font-size:9px; line-height:1; color:#8c9a86; }
|
| 27 |
+
.gradio-container .atlas-disclosure > button.label-wrap + * { padding:0 14px 14px; }
|
| 28 |
+
.lineage-query textarea, .lineage-query input { font-size:14px; }
|
| 29 |
+
.lineage-matches .wrap { display:flex; flex-direction:column; gap:2px; }
|
| 30 |
+
.lineage-matches label { border-radius:8px; padding:7px 10px; font-size:13px; }
|
| 31 |
+
.lineage-matches label:hover { background:#eef3e9; }
|
| 32 |
#annotation-plot { border:1px solid #e0e6d8; background:#fffefa; border-radius:16px; overflow:hidden; }
|
| 33 |
#annotation-plot .plot-container { background:#fffefa; }
|
| 34 |
#annotation-examples { font-size:12px; }
|
|
|
|
| 39 |
.workspace-footer a:hover { text-decoration:underline; }
|
| 40 |
@media(max-width:700px) {
|
| 41 |
.gradio-container { padding:12px 10px 24px!important; }
|
| 42 |
+
.gradio-container .atlas-panel { padding:18px 12px; }
|
| 43 |
+
#atlas-navigation { border-radius:18px; padding:8px 8px 0; }
|
| 44 |
.workspace-heading h2 { font-size:28px; }
|
| 45 |
.workspace-heading { gap:10px; }
|
| 46 |
.gradio-container .action-button { width:100%; }
|
|
|
|
| 49 |
|
| 50 |
/* Full-width navigation: one equal segment for each view. */
|
| 51 |
.gradio-container { width:100%!important; min-width:0!important; box-sizing:border-box; }
|
| 52 |
+
/* The navigation is the card: the two views sit inside one curved boundary and
|
| 53 |
+
the control reads as a toggle rather than a pair of tabs. */
|
| 54 |
+
#atlas-navigation { border:1px solid #e0e6d8; background:#f7f8f0; border-radius:24px; padding:12px 12px 0; min-width:0; max-width:100%; box-shadow:0 12px 50px #234b2910; }
|
| 55 |
+
#atlas-navigation > .tab-wrapper { width:100%; height:auto!important; min-height:0; border:0!important; box-shadow:none!important; margin:0; padding:0; }
|
| 56 |
+
#atlas-navigation > .tab-wrapper > .tab-container { display:grid!important; grid-template-columns:1fr 1fr; gap:4px; width:100%!important; height:auto!important; box-sizing:border-box; border:0!important; border-radius:14px; padding:4px; margin:0; background:#e7eee0; box-shadow:none!important; }
|
| 57 |
+
#atlas-navigation > .tab-wrapper > .tab-container > button { display:flex!important; align-items:center; justify-content:center; width:100%; min-width:0; min-height:40px; height:auto!important; box-sizing:border-box; padding:9px 12px; margin:0!important; border:0!important; border-radius:11px; box-shadow:none!important; background:transparent; color:#4c6450; font:600 14px/1.3 Arial,Helvetica,sans-serif; white-space:nowrap; text-align:center; text-decoration:none!important; transition:background-color .15s, color .15s; }
|
| 58 |
+
#atlas-navigation > .tab-wrapper > .tab-container > button:hover { background:#dce6d4; color:#173c30; }
|
| 59 |
+
#atlas-navigation > .tab-wrapper > .tab-container > button[aria-selected="true"] { background:#1f5740; color:#f6faf3; }
|
| 60 |
#atlas-navigation > .tab-wrapper::before, #atlas-navigation > .tab-wrapper::after,
|
| 61 |
#atlas-navigation > .tab-wrapper > .tab-container::before, #atlas-navigation > .tab-wrapper > .tab-container::after,
|
| 62 |
#atlas-navigation > .tab-wrapper > .tab-container > button::before, #atlas-navigation > .tab-wrapper > .tab-container > button::after { content:none!important; display:none!important; }
|
app.py
CHANGED
|
@@ -129,9 +129,9 @@ def build_app(catalog=None):
|
|
| 129 |
return str(target)
|
| 130 |
|
| 131 |
with gr.Blocks(title="GenBank Annotation Explorer", delete_cache=(3600, 3600)) as demo:
|
| 132 |
-
with gr.Tabs(selected="atlas", elem_id="atlas-navigation"):
|
| 133 |
with gr.Tab("Genome Atlas", id="atlas", elem_id="atlas-overview"):
|
| 134 |
-
build_taxonomy_tab()
|
| 135 |
with gr.Tab("Database", id="database", elem_id="atlas-database"):
|
| 136 |
gr.HTML('<header class="database-heading"><p class="database-eyebrow">THE ANNOTATION COLLECTION</p>'
|
| 137 |
'<h1>Explore the database</h1><p>Find an accession, explore its coding landscape, and download the annotations.</p></header>',
|
|
@@ -196,6 +196,16 @@ def build_app(catalog=None):
|
|
| 196 |
gr.HTML('<footer class="workspace-footer"><span>Hugging Face Bio · Genome Atlas</span>'
|
| 197 |
'<a href="https://huggingface.co/buckets/HuggingFaceBio/genbank-annotations" target="_blank" rel="noopener noreferrer">Explore the annotation collection ↗</a></footer>', apply_default_css=False)
|
| 198 |
outputs = [status, results, segment, file]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 199 |
for event in (search_button.click, accession.submit):
|
| 200 |
event(search, accession, outputs).then(select_segment, [segment, mode, threshold], [metadata, start, end, plot, note, file])
|
| 201 |
segment.input(select_segment, [segment, mode, threshold], [metadata, start, end, plot, note, file])
|
|
|
|
| 129 |
return str(target)
|
| 130 |
|
| 131 |
with gr.Blocks(title="GenBank Annotation Explorer", delete_cache=(3600, 3600)) as demo:
|
| 132 |
+
with gr.Tabs(selected="atlas", elem_id="atlas-navigation") as navigation:
|
| 133 |
with gr.Tab("Genome Atlas", id="atlas", elem_id="atlas-overview"):
|
| 134 |
+
atlas = build_taxonomy_tab()
|
| 135 |
with gr.Tab("Database", id="database", elem_id="atlas-database"):
|
| 136 |
gr.HTML('<header class="database-heading"><p class="database-eyebrow">THE ANNOTATION COLLECTION</p>'
|
| 137 |
'<h1>Explore the database</h1><p>Find an accession, explore its coding landscape, and download the annotations.</p></header>',
|
|
|
|
| 196 |
gr.HTML('<footer class="workspace-footer"><span>Hugging Face Bio · Genome Atlas</span>'
|
| 197 |
'<a href="https://huggingface.co/buckets/HuggingFaceBio/genbank-annotations" target="_blank" rel="noopener noreferrer">Explore the annotation collection ↗</a></footer>', apply_default_css=False)
|
| 198 |
outputs = [status, results, segment, file]
|
| 199 |
+
if atlas:
|
| 200 |
+
# The atlas names a group; the Database tab searches accessions. The
|
| 201 |
+
# jump hands over one annotated assembly from the selected group and
|
| 202 |
+
# runs the ordinary search with it.
|
| 203 |
+
def open_database(path):
|
| 204 |
+
return gr.Tabs(selected="database"), atlas["accession_for"](path)
|
| 205 |
+
|
| 206 |
+
atlas["button"].click(open_database, atlas["route"], [navigation, accession]) \
|
| 207 |
+
.then(search, accession, outputs) \
|
| 208 |
+
.then(select_segment, [segment, mode, threshold], [metadata, start, end, plot, note, file])
|
| 209 |
for event in (search_button.click, accession.submit):
|
| 210 |
event(search, accession, outputs).then(select_segment, [segment, mode, threshold], [metadata, start, end, plot, note, file])
|
| 211 |
segment.input(select_segment, [segment, mode, threshold], [metadata, start, end, plot, note, file])
|
atlas.css
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
.atlas { --ink:#173c30; --muted:#647467; position:relative; overflow:hidden; color:var(--ink); background:
|
| 2 |
.atlas * { box-sizing:border-box; }
|
| 3 |
.atlas-header { display:flex; justify-content:space-between; align-items:center; gap:28px; }
|
| 4 |
.atlas-kicker { font-size:10px; font-weight:700; letter-spacing:1.5px; line-height:1.6; text-transform:uppercase; color:#527261; }
|
|
@@ -72,7 +72,7 @@
|
|
| 72 |
.atlas button:focus-visible { outline:2px solid #327555; outline-offset:3px; }
|
| 73 |
.atlas-leaf-note { color:#647467; font-size:13px; text-align:center; margin:0 0 20px; }
|
| 74 |
@media(max-width:800px) {
|
| 75 |
-
.atlas { padding:
|
| 76 |
/* Levels fill the page width, so on a phone the only way to keep the labels
|
| 77 |
legible is to let the chart itself scroll sideways again. */
|
| 78 |
.tree-scroll { overflow-x:auto; overscroll-behavior-x:contain; }
|
|
|
|
| 1 |
+
.atlas { --ink:#173c30; --muted:#647467; position:relative; overflow:hidden; color:var(--ink); background:transparent; border:0; border-radius:0; padding:24px 20px 14px; font-family:Arial,Helvetica,sans-serif; }
|
| 2 |
.atlas * { box-sizing:border-box; }
|
| 3 |
.atlas-header { display:flex; justify-content:space-between; align-items:center; gap:28px; }
|
| 4 |
.atlas-kicker { font-size:10px; font-weight:700; letter-spacing:1.5px; line-height:1.6; text-transform:uppercase; color:#527261; }
|
|
|
|
| 72 |
.atlas button:focus-visible { outline:2px solid #327555; outline-offset:3px; }
|
| 73 |
.atlas-leaf-note { color:#647467; font-size:13px; text-align:center; margin:0 0 20px; }
|
| 74 |
@media(max-width:800px) {
|
| 75 |
+
.atlas { padding:18px 10px 14px; }
|
| 76 |
/* Levels fill the page width, so on a phone the only way to keep the labels
|
| 77 |
legible is to let the chart itself scroll sideways again. */
|
| 78 |
.tree-scroll { overflow-x:auto; overscroll-behavior-x:contain; }
|
data/taxonomy.sqlite
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:88e8b10a82026dfe21a3022ae788e50d82a4ca521f948a29f4a219e111b185b1
|
| 3 |
+
size 26984448
|
refresh_assemblies.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Add the taxon-to-accession table the atlas needs to open Database results.
|
| 2 |
+
|
| 3 |
+
The taxonomy snapshot stores counts, not accessions, so a group in the atlas had
|
| 4 |
+
no way to name a single assembly the Database tab could look up. This writes an
|
| 5 |
+
``assemblies`` table into the existing ``data/taxonomy.sqlite`` holding one row
|
| 6 |
+
per *annotated* eukaryotic assembly: the accession the Database tab searches and
|
| 7 |
+
the taxon it belongs to.
|
| 8 |
+
|
| 9 |
+
Only annotated assemblies are kept. The atlas offers this as a jump into
|
| 10 |
+
published annotations, so an accession with nothing behind it would be a dead
|
| 11 |
+
end, and the restriction keeps the table at roughly 33,000 rows.
|
| 12 |
+
|
| 13 |
+
Counts are deliberately untouched. Rebuilding them here would re-read a newer
|
| 14 |
+
assembly summary and silently shift every total in the interface away from the
|
| 15 |
+
published snapshot; see refresh_taxonomy.py for that, which is a separate
|
| 16 |
+
decision.
|
| 17 |
+
"""
|
| 18 |
+
import argparse
|
| 19 |
+
from datetime import datetime, timezone
|
| 20 |
+
import json
|
| 21 |
+
from pathlib import Path
|
| 22 |
+
import sqlite3
|
| 23 |
+
import tarfile
|
| 24 |
+
|
| 25 |
+
ROOT = Path(__file__).resolve().parent
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def merged_taxids(taxdump):
|
| 29 |
+
with tarfile.open(taxdump, "r:gz") as archive, archive.extractfile("merged.dmp") as stream:
|
| 30 |
+
return {int(parts[0]): int(parts[1]) for parts in
|
| 31 |
+
([p.strip() for p in line.decode().split("|")] for line in stream)}
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def build(summary, taxdump, database, coverage):
|
| 35 |
+
annotated = set(json.loads(Path(coverage).read_text())["accessions"])
|
| 36 |
+
merged = merged_taxids(taxdump)
|
| 37 |
+
with sqlite3.connect(database) as conn:
|
| 38 |
+
known = {row[0] for row in conn.execute("SELECT taxid FROM taxa")}
|
| 39 |
+
|
| 40 |
+
rows, skipped, unresolved = [], 0, 0
|
| 41 |
+
with open(summary) as stream:
|
| 42 |
+
columns = None
|
| 43 |
+
for line in stream:
|
| 44 |
+
if line.startswith("##"):
|
| 45 |
+
continue
|
| 46 |
+
if line.startswith("#"):
|
| 47 |
+
header = line.lstrip("# ").rstrip("\n").split("\t")
|
| 48 |
+
columns = {name: header.index(name) for name in ("assembly_accession", "taxid", "version_status")}
|
| 49 |
+
continue
|
| 50 |
+
row = line.rstrip("\n").split("\t")
|
| 51 |
+
accession = row[columns["assembly_accession"]]
|
| 52 |
+
if accession not in annotated or row[columns["version_status"]] != "latest":
|
| 53 |
+
skipped += 1
|
| 54 |
+
continue
|
| 55 |
+
try:
|
| 56 |
+
taxid = int(row[columns["taxid"]])
|
| 57 |
+
except ValueError:
|
| 58 |
+
unresolved += 1
|
| 59 |
+
continue
|
| 60 |
+
seen = set()
|
| 61 |
+
while taxid in merged and taxid not in seen:
|
| 62 |
+
seen.add(taxid)
|
| 63 |
+
taxid = merged[taxid]
|
| 64 |
+
if taxid not in known:
|
| 65 |
+
unresolved += 1
|
| 66 |
+
continue
|
| 67 |
+
rows.append((accession, taxid))
|
| 68 |
+
|
| 69 |
+
if not rows:
|
| 70 |
+
raise ValueError("No annotated assemblies matched the summary; check the inputs.")
|
| 71 |
+
with sqlite3.connect(database) as conn:
|
| 72 |
+
conn.executescript("DROP TABLE IF EXISTS assemblies;"
|
| 73 |
+
"CREATE TABLE assemblies (accession TEXT PRIMARY KEY, taxid INTEGER);")
|
| 74 |
+
conn.executemany("INSERT OR REPLACE INTO assemblies VALUES (?,?)", rows)
|
| 75 |
+
conn.execute("CREATE INDEX assembly_taxa ON assemblies(taxid)")
|
| 76 |
+
metadata = json.loads(conn.execute("SELECT value FROM metadata").fetchone()[0])
|
| 77 |
+
metadata["assemblies"] = {"created_at": datetime.now(timezone.utc).isoformat(),
|
| 78 |
+
"rows": len(rows), "scope": "annotated eukaryotic assemblies only",
|
| 79 |
+
"source": "https://ftp.ncbi.nlm.nih.gov/genomes/ASSEMBLY_REPORTS/assembly_summary_genbank.txt"}
|
| 80 |
+
conn.execute("UPDATE metadata SET value=?", (json.dumps(metadata),))
|
| 81 |
+
print(json.dumps({"rows": len(rows), "annotated_accessions": len(annotated),
|
| 82 |
+
"unresolved_taxids": unresolved, "rows_skipped": skipped}, indent=2))
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
def main():
|
| 86 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 87 |
+
parser.add_argument("--summary", type=Path, default=ROOT / ".cache/coverage/assembly_summary_genbank.txt")
|
| 88 |
+
parser.add_argument("--taxdump", type=Path, default=ROOT / ".cache/coverage/taxdump.tar.gz")
|
| 89 |
+
parser.add_argument("--database", type=Path, default=ROOT / "data/taxonomy.sqlite")
|
| 90 |
+
parser.add_argument("--coverage", type=Path, default=ROOT / "data/coverage.json")
|
| 91 |
+
args = parser.parse_args()
|
| 92 |
+
build(args.summary, args.taxdump, args.database, args.coverage)
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
if __name__ == "__main__":
|
| 96 |
+
main()
|
taxonomy.py
CHANGED
|
@@ -39,6 +39,7 @@ SHORTCUTS = ((33208, "Animals"), (40674, "Mammals"), (9606, "Humans"), (8782, "B
|
|
| 39 |
|
| 40 |
|
| 41 |
COLUMN_LIMIT = 6
|
|
|
|
| 42 |
MAX_DEPTH = 80
|
| 43 |
DEFAULT_PATH = "2759/33154"
|
| 44 |
|
|
@@ -75,17 +76,51 @@ class Taxonomy:
|
|
| 75 |
conn.row_factory = sqlite3.Row
|
| 76 |
return conn
|
| 77 |
|
| 78 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
query = str(query or "").strip()
|
| 80 |
-
|
| 81 |
-
|
|
|
|
| 82 |
with closing(self.connect()) as conn:
|
| 83 |
if query.lstrip("-").isdigit():
|
| 84 |
-
|
|
|
|
| 85 |
else:
|
| 86 |
pattern = "%" + query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_") + "%"
|
| 87 |
-
|
| 88 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
def icon_for(self, conn, taxid):
|
| 91 |
"""Nearest icon at or above this taxon, following the lineage to Eukaryota.
|
|
@@ -116,6 +151,34 @@ class Taxonomy:
|
|
| 116 |
rows.append(dict(conn.execute("SELECT * FROM taxa WHERE taxid=?", (rows[-1]['parent_id'],)).fetchone()))
|
| 117 |
return rows[::-1]
|
| 118 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
def taxon_path(self, taxid):
|
| 120 |
"""Path that expands every ancestor of a taxon, for search results and shortcuts."""
|
| 121 |
taxid = int(taxid)
|
|
@@ -351,8 +414,7 @@ class Taxonomy:
|
|
| 351 |
leaf_note = '<p class="atlas-leaf-note">This is a terminal taxon in the assembly snapshot. Use the breadcrumb to explore its relatives.</p>' if not has_children else ''
|
| 352 |
return f'''<section class="atlas" aria-label="Tree of eukaryotic life">
|
| 353 |
<header class="atlas-header">
|
| 354 |
-
<div><
|
| 355 |
-
<h1>The tree of eukaryotic life<span>.</span></h1>
|
| 356 |
<p>Follow the branches. Discover where we’ve annotated.</p></div>
|
| 357 |
<div class="atlas-summary-block">{summary}</div>
|
| 358 |
</header>
|
|
@@ -389,10 +451,10 @@ def credit_lines(taxonomy):
|
|
| 389 |
|
| 390 |
|
| 391 |
def build_taxonomy_tab():
|
| 392 |
-
"""Render the tree and its controls
|
| 393 |
if not DATABASE.exists():
|
| 394 |
gr.Markdown("The eukaryotic tree will appear when the taxonomy snapshot is available.")
|
| 395 |
-
return
|
| 396 |
taxonomy = Taxonomy()
|
| 397 |
if taxonomy.icons:
|
| 398 |
# The chart references each silhouette by URL, so the browser caches it
|
|
@@ -402,35 +464,61 @@ def build_taxonomy_tab():
|
|
| 402 |
js_on_load=(ROOT / "atlas.js").read_text(), apply_default_css=False,
|
| 403 |
elem_id="eukaryotic-atlas")
|
| 404 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 405 |
def explore_tree(evt: gr.EventData):
|
| 406 |
try:
|
| 407 |
-
return
|
| 408 |
except (ValueError, TypeError, AttributeError) as exc:
|
| 409 |
raise gr.Error("Choose a eukaryotic group in this snapshot.") from exc
|
| 410 |
|
| 411 |
-
tree.click(explore_tree, outputs=tree, show_progress="minimal", concurrency_limit=1)
|
| 412 |
with gr.Accordion("Find a lineage", open=False, elem_classes="atlas-disclosure"):
|
| 413 |
-
|
| 414 |
-
|
| 415 |
-
|
| 416 |
-
matches = gr.
|
| 417 |
-
|
| 418 |
|
| 419 |
def taxonomy_search(text):
|
| 420 |
-
choices = taxonomy.search(text)
|
| 421 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 422 |
|
| 423 |
def taxonomy_view(taxid):
|
| 424 |
-
return taxonomy.
|
| 425 |
|
| 426 |
-
|
| 427 |
-
|
| 428 |
-
|
| 429 |
-
|
| 430 |
-
|
| 431 |
-
|
| 432 |
-
|
| 433 |
-
|
|
|
|
| 434 |
with gr.Accordion("About the tree and coverage", open=False, elem_classes="atlas-disclosure"):
|
| 435 |
gr.Markdown("This tree follows **NCBI Taxonomy within Eukaryota**. Animals and fungi share the Opisthokonta branch; "
|
| 436 |
"green plants include green algae. ‘Other lineages’ combines the remaining siblings for display and can be expanded. "
|
|
@@ -460,3 +548,14 @@ def build_taxonomy_tab():
|
|
| 460 |
"inventory_method": coverage.get('method'),
|
| 461 |
"published_versions_outside_current_GenBank_snapshot": coverage.get('unmatched_assembly_versions'),
|
| 462 |
"sources": taxonomy.metadata['sources']}, label="Snapshot provenance")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
|
| 41 |
COLUMN_LIMIT = 6
|
| 42 |
+
SEARCH_LIMIT = 12
|
| 43 |
MAX_DEPTH = 80
|
| 44 |
DEFAULT_PATH = "2759/33154"
|
| 45 |
|
|
|
|
| 76 |
conn.row_factory = sqlite3.Row
|
| 77 |
return conn
|
| 78 |
|
| 79 |
+
def closeness(self, row, needle):
|
| 80 |
+
"""Whole-name matches first, then word starts: 'thale' should find thale cress
|
| 81 |
+
before it finds Magnaporthales."""
|
| 82 |
+
names = [row['name'].casefold(), self.english.get(row['taxid'], "").casefold()]
|
| 83 |
+
if any(name == needle for name in names):
|
| 84 |
+
return 0
|
| 85 |
+
if any(name.startswith(needle) for name in names):
|
| 86 |
+
return 1
|
| 87 |
+
if any(word.startswith(needle) for name in names for word in name.split()):
|
| 88 |
+
return 2
|
| 89 |
+
return 3
|
| 90 |
+
|
| 91 |
+
def label(self, row):
|
| 92 |
+
english = self.english.get(row['taxid'], "")
|
| 93 |
+
return (f"{row['name']}" + (f" · {english}" if english else "") +
|
| 94 |
+
f" · {row['total_count']:,} assemblies · taxid {row['taxid']}")
|
| 95 |
+
|
| 96 |
+
def search(self, query, limit=SEARCH_LIMIT):
|
| 97 |
+
"""Match a taxon id, a scientific name, or one of the English names.
|
| 98 |
+
|
| 99 |
+
The eukaryote set is already in memory, so the filtering happens here
|
| 100 |
+
rather than in a recursive CTE the query would otherwise re-walk on every
|
| 101 |
+
keystroke.
|
| 102 |
+
"""
|
| 103 |
query = str(query or "").strip()
|
| 104 |
+
if len(query) < 2 and not query.isdigit():
|
| 105 |
+
return []
|
| 106 |
+
needle = query.casefold()
|
| 107 |
with closing(self.connect()) as conn:
|
| 108 |
if query.lstrip("-").isdigit():
|
| 109 |
+
row = conn.execute("SELECT * FROM taxa WHERE taxid=?", (int(query),)).fetchone()
|
| 110 |
+
rows = [row] if row and row['taxid'] in self.eukaryote_ids else []
|
| 111 |
else:
|
| 112 |
pattern = "%" + query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_") + "%"
|
| 113 |
+
found = {r['taxid']: r for r in conn.execute(
|
| 114 |
+
"SELECT * FROM taxa WHERE name LIKE ? ESCAPE '\\' ORDER BY total_count DESC LIMIT ?",
|
| 115 |
+
(pattern, limit * 4)) if r['taxid'] in self.eukaryote_ids}
|
| 116 |
+
english = [t for t, name in self.english.items()
|
| 117 |
+
if needle in name.casefold() and t not in found and t in self.eukaryote_ids]
|
| 118 |
+
for chunk in (english[i:i + 400] for i in range(0, len(english), 400)):
|
| 119 |
+
marks = ",".join("?" * len(chunk))
|
| 120 |
+
found.update({r['taxid']: r for r in conn.execute(
|
| 121 |
+
f"SELECT * FROM taxa WHERE taxid IN ({marks})", chunk)})
|
| 122 |
+
rows = sorted(found.values(), key=lambda r: (self.closeness(r, needle), -r['total_count']))[:limit]
|
| 123 |
+
return [(self.label(r), str(r['taxid'])) for r in rows]
|
| 124 |
|
| 125 |
def icon_for(self, conn, taxid):
|
| 126 |
"""Nearest icon at or above this taxon, following the lineage to Eukaryota.
|
|
|
|
| 151 |
rows.append(dict(conn.execute("SELECT * FROM taxa WHERE taxid=?", (rows[-1]['parent_id'],)).fetchone()))
|
| 152 |
return rows[::-1]
|
| 153 |
|
| 154 |
+
def assemblies_for(self, path, limit=1):
|
| 155 |
+
"""Annotated accessions under the taxon a route selects, largest group first.
|
| 156 |
+
|
| 157 |
+
The recursive walk starts at the selected taxon rather than at Eukaryota,
|
| 158 |
+
so it touches only that subtree.
|
| 159 |
+
"""
|
| 160 |
+
try:
|
| 161 |
+
taxid = int(str(path).split("/")[-1].partition("~")[0])
|
| 162 |
+
except (ValueError, AttributeError):
|
| 163 |
+
return []
|
| 164 |
+
if taxid not in self.eukaryote_ids:
|
| 165 |
+
return []
|
| 166 |
+
with closing(self.connect()) as conn:
|
| 167 |
+
if not conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='assemblies'").fetchone():
|
| 168 |
+
return []
|
| 169 |
+
return [r[0] for r in conn.execute(
|
| 170 |
+
"""WITH RECURSIVE sub(taxid) AS (
|
| 171 |
+
SELECT ? UNION ALL
|
| 172 |
+
SELECT t.taxid FROM taxa t JOIN sub ON t.parent_id=sub.taxid WHERE t.taxid!=t.parent_id
|
| 173 |
+
) SELECT accession FROM assemblies JOIN sub USING(taxid) ORDER BY accession LIMIT ?""",
|
| 174 |
+
(taxid, limit))]
|
| 175 |
+
|
| 176 |
+
def selected_name(self, path):
|
| 177 |
+
taxid = int(str(path).split("/")[-1].partition("~")[0])
|
| 178 |
+
with closing(self.connect()) as conn:
|
| 179 |
+
row = conn.execute("SELECT name FROM taxa WHERE taxid=?", (taxid,)).fetchone()
|
| 180 |
+
return row['name'] if row else str(taxid)
|
| 181 |
+
|
| 182 |
def taxon_path(self, taxid):
|
| 183 |
"""Path that expands every ancestor of a taxon, for search results and shortcuts."""
|
| 184 |
taxid = int(taxid)
|
|
|
|
| 414 |
leaf_note = '<p class="atlas-leaf-note">This is a terminal taxon in the assembly snapshot. Use the breadcrumb to explore its relatives.</p>' if not has_children else ''
|
| 415 |
return f'''<section class="atlas" aria-label="Tree of eukaryotic life">
|
| 416 |
<header class="atlas-header">
|
| 417 |
+
<div><h1>The tree of eukaryotic life<span>.</span></h1>
|
|
|
|
| 418 |
<p>Follow the branches. Discover where we’ve annotated.</p></div>
|
| 419 |
<div class="atlas-summary-block">{summary}</div>
|
| 420 |
</header>
|
|
|
|
| 451 |
|
| 452 |
|
| 453 |
def build_taxonomy_tab():
|
| 454 |
+
"""Render the tree and its controls, and hand back the jump into the Database tab."""
|
| 455 |
if not DATABASE.exists():
|
| 456 |
gr.Markdown("The eukaryotic tree will appear when the taxonomy snapshot is available.")
|
| 457 |
+
return None
|
| 458 |
taxonomy = Taxonomy()
|
| 459 |
if taxonomy.icons:
|
| 460 |
# The chart references each silhouette by URL, so the browser caches it
|
|
|
|
| 464 |
js_on_load=(ROOT / "atlas.js").read_text(), apply_default_css=False,
|
| 465 |
elem_id="eukaryotic-atlas")
|
| 466 |
|
| 467 |
+
route = gr.State(DEFAULT_PATH)
|
| 468 |
+
|
| 469 |
+
def jump_text(path):
|
| 470 |
+
name = taxonomy.selected_name(path)
|
| 471 |
+
return (f"Show database results for {name}" if taxonomy.assemblies_for(path)
|
| 472 |
+
else f"No annotated assemblies in {name}")
|
| 473 |
+
|
| 474 |
+
def jump_label(path):
|
| 475 |
+
return gr.Button(jump_text(path), interactive=bool(taxonomy.assemblies_for(path)))
|
| 476 |
+
|
| 477 |
+
# The jump into the Database tab needs a real accession, so the button says
|
| 478 |
+
# up front whether the current group has any annotated assembly behind it.
|
| 479 |
+
open_database = gr.Button(jump_text(DEFAULT_PATH), elem_classes="action-button",
|
| 480 |
+
elem_id="atlas-open-database")
|
| 481 |
+
|
| 482 |
+
def show(path):
|
| 483 |
+
return taxonomy.view(path), path, jump_label(path)
|
| 484 |
+
|
| 485 |
def explore_tree(evt: gr.EventData):
|
| 486 |
try:
|
| 487 |
+
return show(evt.route)
|
| 488 |
except (ValueError, TypeError, AttributeError) as exc:
|
| 489 |
raise gr.Error("Choose a eukaryotic group in this snapshot.") from exc
|
| 490 |
|
| 491 |
+
tree.click(explore_tree, outputs=[tree, route, open_database], show_progress="minimal", concurrency_limit=1)
|
| 492 |
with gr.Accordion("Find a lineage", open=False, elem_classes="atlas-disclosure"):
|
| 493 |
+
query = gr.Textbox(label="Find a lineage", show_label=False, elem_classes="lineage-query",
|
| 494 |
+
placeholder="Scientific name, English name such as sponges or jellyfish, or a taxon ID")
|
| 495 |
+
status = gr.Markdown("Start typing to see matching lineages.", elem_classes="quiet-note")
|
| 496 |
+
matches = gr.Radio(choices=[], label="Matching lineages", show_label=False,
|
| 497 |
+
interactive=True, elem_classes="lineage-matches")
|
| 498 |
|
| 499 |
def taxonomy_search(text):
|
| 500 |
+
choices = taxonomy.search(text)
|
| 501 |
+
if not choices:
|
| 502 |
+
typed = str(text or "").strip()
|
| 503 |
+
return gr.Radio(choices=[], value=None), (
|
| 504 |
+
"No matching eukaryotic taxa in this snapshot." if len(typed) >= 2
|
| 505 |
+
else "Start typing to see matching lineages.")
|
| 506 |
+
best = choices[0][0].split(" · ")[0]
|
| 507 |
+
return (gr.Radio(choices=choices, value=choices[0][1]),
|
| 508 |
+
f"{len(choices)} matches, closest first. Press Enter for **{best}**, or pick another.")
|
| 509 |
|
| 510 |
def taxonomy_view(taxid):
|
| 511 |
+
return show(taxonomy.taxon_path(taxid) if taxid else DEFAULT_PATH)
|
| 512 |
|
| 513 |
+
def taxonomy_best(text):
|
| 514 |
+
choices = taxonomy.search(text)
|
| 515 |
+
return taxonomy_view(choices[0][1] if choices else None)
|
| 516 |
+
|
| 517 |
+
# No separate search button: the list follows what is typed, Enter takes
|
| 518 |
+
# the closest match, and picking any row opens it.
|
| 519 |
+
query.change(taxonomy_search, query, [matches, status], show_progress="hidden")
|
| 520 |
+
query.submit(taxonomy_best, query, [tree, route, open_database])
|
| 521 |
+
matches.input(taxonomy_view, matches, [tree, route, open_database])
|
| 522 |
with gr.Accordion("About the tree and coverage", open=False, elem_classes="atlas-disclosure"):
|
| 523 |
gr.Markdown("This tree follows **NCBI Taxonomy within Eukaryota**. Animals and fungi share the Opisthokonta branch; "
|
| 524 |
"green plants include green algae. ‘Other lineages’ combines the remaining siblings for display and can be expanded. "
|
|
|
|
| 548 |
"inventory_method": coverage.get('method'),
|
| 549 |
"published_versions_outside_current_GenBank_snapshot": coverage.get('unmatched_assembly_versions'),
|
| 550 |
"sources": taxonomy.metadata['sources']}, label="Snapshot provenance")
|
| 551 |
+
if taxonomy.icons:
|
| 552 |
+
with gr.Accordion("Silhouette credits", open=False, elem_classes="atlas-disclosure"):
|
| 553 |
+
gr.Markdown("\n".join(["Silhouettes from [PhyloPic](https://www.phylopic.org), used unmodified. "
|
| 554 |
+
"Public-domain images need no credit; the CC BY ones do, and every artist is listed here. "
|
| 555 |
+
"`data/icons.json` maps each taxon to its image.", ""] + credit_lines(taxonomy)))
|
| 556 |
+
|
| 557 |
+
def first_accession(path):
|
| 558 |
+
found = taxonomy.assemblies_for(path)
|
| 559 |
+
return found[0] if found else ""
|
| 560 |
+
|
| 561 |
+
return {"route": route, "button": open_database, "accession_for": first_accession}
|
tests/test_taxonomy.py
CHANGED
|
@@ -177,6 +177,32 @@ class TaxonomyTests(unittest.TestCase):
|
|
| 177 |
# A trunk ribbon is still an ordinary ribbon, hoverable like any other.
|
| 178 |
self.assertNotIn('class="flow flow-annotated is-path" data-node="0-0"', chart)
|
| 179 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
def test_rejects_paths_outside_the_tree(self):
|
| 181 |
for path in ("1", "33208", "2759/33208", "2759/2759~3", "2759/9606", "abc"):
|
| 182 |
with self.assertRaises(ValueError, msg=path):
|
|
|
|
| 177 |
# A trunk ribbon is still an ordinary ribbon, hoverable like any other.
|
| 178 |
self.assertNotIn('class="flow flow-annotated is-path" data-node="0-0"', chart)
|
| 179 |
|
| 180 |
+
def test_search_matches_english_names(self):
|
| 181 |
+
for text, expected in (("sponges", "Porifera"), ("jellyfish", "Scyphozoa"),
|
| 182 |
+
("birds", "Aves"), ("human", "Homo sapiens")):
|
| 183 |
+
found = self.taxonomy.search(text)
|
| 184 |
+
self.assertTrue(found, text)
|
| 185 |
+
# The closest match leads, so it can be preselected in the picker.
|
| 186 |
+
self.assertTrue(found[0][0].startswith(expected + " · "), f"{text} -> {found[0][0]}")
|
| 187 |
+
self.assertEqual(self.taxonomy.search("9606")[0][1], "9606")
|
| 188 |
+
self.assertEqual(self.taxonomy.search("x"), [])
|
| 189 |
+
|
| 190 |
+
def test_database_jump_offers_an_annotated_accession(self):
|
| 191 |
+
with sqlite3.connect(f"{DATABASE.resolve().as_uri()}?mode=ro", uri=True) as conn:
|
| 192 |
+
if not conn.execute("SELECT name FROM sqlite_master WHERE name='assemblies'").fetchone():
|
| 193 |
+
self.skipTest("assembly table unavailable")
|
| 194 |
+
annotated = set(json.loads((DATABASE.parent / "coverage.json").read_text())["accessions"])
|
| 195 |
+
for path in (DEFAULT_PATH, "2759/33154/33208", self.taxonomy.taxon_path(40674)):
|
| 196 |
+
found = self.taxonomy.assemblies_for(path, 3)
|
| 197 |
+
self.assertTrue(found, path)
|
| 198 |
+
# Every offer is a published annotation, never a dead end.
|
| 199 |
+
for accession in found:
|
| 200 |
+
self.assertIn(accession, annotated, path)
|
| 201 |
+
# Mammal accessions must sit inside the mammal subtree, not above it.
|
| 202 |
+
mammals = set(self.taxonomy.assemblies_for(self.taxonomy.taxon_path(40674), 50))
|
| 203 |
+
birds = set(self.taxonomy.assemblies_for(self.taxonomy.taxon_path(8782), 50))
|
| 204 |
+
self.assertFalse(mammals & birds)
|
| 205 |
+
|
| 206 |
def test_rejects_paths_outside_the_tree(self):
|
| 207 |
for path in ("1", "33208", "2759/33208", "2759/2759~3", "2759/9606", "abc"):
|
| 208 |
with self.assertRaises(ValueError, msg=path):
|