Color GenBank taxonomy by published annotation coverage

FEATURE_BACKLOG.md

@@ -8,8 +8,8 @@ Last updated: 2026-09-30.
 
 | ID | Feature | Status |
 | --- | --- | --- |
-| F001 | GenBank inference coverage map | Planned; not implemented |
-| F002 | Taxonomy map and navigation | GenBank assembly taxonomy implemented; annotation links and coverage pending |
+| F001 | GenBank inference coverage map | Assembly-presence colors implemented; base completeness and history pending |
+| F002 | Taxonomy map and navigation | Taxonomy and coverage colors implemented; annotation links pending |
 | F003 | Comparison with RefSeq annotations | Planned; not implemented |
 | F004 | Download annotations by accession, individually or in bulk | Partial; individual indexed segment downloads exist |
 | F005 | Fast accession lookup and retrieval across the full dataset | Prototype implemented for three remote objects; full coverage pending |
@@ -17,6 +17,10 @@ Last updated: 2026-09-30.
 
 ## F001 — GenBank inference coverage map
 
+Current implementation: the taxonomy sunburst colors each group by the fraction of current GenBank assembly versions with published annotations. Hover shows counts and percentages; partial assemblies count as present, and this is explicitly distinguished from base coverage. The inventory includes completed published shards plus metadata from unmarked/legacy outputs, deduplicates assembly versions, and reports versions outside the current NCBI snapshot. `refresh_coverage.py` caches metadata by content fingerprints; `refresh_taxonomy.py` rolls coverage into SQLite. Refreshes are manual. See [coverage refresh and evidence limitations](README.md#refresh-annotation-coverage).
+
+Remaining: partial versus complete assembly/base coverage, automatic updates and history, and navigation to available annotations. The existing completion markers do not bind input content hashes; stronger publication manifests would make the inventory independently auditable.
+
 **Goal:** Show how much of GenBank we currently have annotations for, and how that coverage grows as more inference completes.
 
 Desired behavior:
@@ -36,7 +40,7 @@ Decisions and dependencies for implementation:
 
 ## F002 — Taxonomy map and navigation
 
-Current implementation: the first tab shows an NCBI taxonomy sunburst sized by current GenBank assembly counts. It supports branch expansion, child/parent navigation, and scientific-name/taxon-ID search. A separate SQLite snapshot stores aggregated counts and provenance; `python refresh_taxonomy.py --download` refreshes it. See [refresh instructions](README.md#refresh-the-taxonomy-snapshot). The chart includes all assembly groups; it does not show inference coverage. Linking taxa to annotation accessions and coverage coloring remain future work. F001 is deferred for now.
+Current implementation: the first tab shows an NCBI taxonomy sunburst sized by current GenBank assembly counts and colored by published annotation presence. It supports branch expansion, child/parent navigation, and scientific-name/taxon-ID search. A separate SQLite snapshot stores aggregated counts and provenance; see [refresh instructions](README.md#refresh-annotation-coverage). The chart includes all assembly groups. Linking taxa to annotation accessions remains future work.
 
 **Goal:** Provide a taxonomy-based view of the available genomes and annotations.
 
@@ -50,7 +54,7 @@ Desired behavior:
 Decisions and dependencies for implementation:
 
 - Implemented: sunburst with bounded subtrees and exact “Other taxa” counts; NCBI taxonomy with source checksums and dates; merged IDs resolved and unresolved IDs explicitly counted.
-- Remaining: connect the taxonomy inventory to inference coverage and accession-level annotation results.
+- Remaining: connect the taxonomy inventory to accession-level annotation results and distinguish partial/complete base coverage.
 
 ## F003 — Comparison with RefSeq annotations
 

README.md

@@ -17,9 +17,9 @@ Future features and open design decisions are tracked in the [feature backlog](F
 
 ## GenBank taxonomy
 
-The first tab shows an interactive sunburst of **GenBank genome assemblies**, using NCBI's current GenBank assembly summary and taxonomy. Sector area represents assembly count. The second tab provides annotation lookup over the existing small index.
+The first tab shows an interactive sunburst of **GenBank genome assemblies**, using NCBI's current GenBank assembly summary and taxonomy. Sector area represents assembly count. Color represents the fraction of assemblies with published annotations: gray for none found in the inventory, increasingly dark teal for higher percentages. Hover shows the numerator, denominator, and percentage. The second tab provides annotation lookup over the existing small index, which is independent of the broader coverage inventory.
 
-The taxonomy includes all taxonomic groups and assembly levels in the current summary, accepting `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. Merged taxon IDs are resolved; unresolved IDs are counted in a separate root branch. These counts describe genome assemblies, not all GenBank nucleotide records or annotation coverage.
+The taxonomy includes all taxonomic groups and assembly levels in the current summary, accepting `GCA_` accessions with `version_status=latest`. Each assembly contributes once to its taxon and each ancestor. Merged taxon IDs are resolved; unresolved IDs are counted in a separate root branch. These counts describe genome assemblies, not all GenBank nucleotide records. An assembly counts as having annotations when it occurs in the publication inventory; this includes partial assemblies and does not imply full base coverage. Exact accession versions are matched to the NCBI snapshot. Published versions outside that snapshot are reported separately and excluded from chart percentages.
 
 Click sectors to expand the loaded chart. Use the child-group menu or scientific-name/taxon-ID search to load another subtree, and the parent/home buttons to return. Each chart loads three descendant levels and the eight largest children per node; smaller children are combined into an accurately counted “Other taxa” sector. Search can reach those taxa individually. Hover percentages refer to the selected group; the group heading also shows its share of the full snapshot.
 
@@ -31,7 +31,22 @@ python refresh_taxonomy.py --download
 
 This downloads roughly 2 GB of **metadata only** to `.cache/coverage/`, then atomically rebuilds `data/taxonomy.sqlite`. To rebuild from already downloaded metadata, omit `--download`. No sequence or probability data is downloaded. The builder uses the Python standard library. The SQLite file contains taxon relationships, names, counts, and source URLs/checksums/dates; it is independent of `data/catalog.sqlite`.
 
-Upload the new `data/taxonomy.sqlite` to the Space and restart/redeploy to activate it. The app reads the bundled snapshot and does not download NCBI metadata at startup. Refreshes are manual; the displayed date is the snapshot build date. Annotation coverage coloring and accession navigation from taxonomy remain in the backlog.
+Upload the new `data/taxonomy.sqlite` to the Space and restart/redeploy to activate it. The app reads the bundled snapshot and does not download NCBI metadata at startup. Refreshes are manual; the displayed dates distinguish the taxonomy build from the annotation inventory. Accession navigation from taxonomy and base-level completeness remain in the backlog.
+
+### Refresh annotation coverage
+
+```bash
+python refresh_coverage.py
+python refresh_taxonomy.py
+```
+
+Use a Hugging Face token with read access to both `HuggingFaceBio/genbank-annotations` and `HuggingFaceBio/GENERanno-packed-inference-input`. The first command lists current published objects, identifies assemblies, and writes `data/coverage.json`. It reads Parquet footers and assembly-accession columns only; sequences and probabilities stay in the buckets. By default 12 workers process shards; change this with `--workers`.
+
+For completed shards, the inventory joins `_SUCCESS.json` to assembly IDs in the corresponding packed `sources.parquet`. This relies on the pipeline's strict unpack contract: the publication script runs with `--strict`, checks part counts, and writes its completion marker after publishing annotations. It also assumes the input shard namespace still identifies the input used by that marker; the older markers do not contain input content hashes. This is assembly-presence evidence, not an audit of every output base. Shards without completion markers, including older single-file outputs, are inventoried directly from annotation metadata.
+
+Assembly IDs are deduplicated across contigs, segments, and shards. The inventory records source/marker hashes and a fingerprint of published objects. Metadata results cached under `.cache/coverage/shards/` are reused only when those fingerprints match. A failed shard aborts publication of the new inventory; rerunning resumes using cached successes. Bucket listing is a dated observation, not an atomic bucket transaction.
+
+The second command rebuilds taxonomy counts using that inventory and the cached NCBI files. Use `--download` to also refresh NCBI metadata. Upload both `data/coverage.json` (inventory and provenance) and `data/taxonomy.sqlite` (runtime counts) to the Space and redeploy. The refresh does not expand the accession lookup index.
 
 ## Run locally
 

data/taxonomy.sqlite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:69388d96ee05e254b8eb8530a62a71d3d9e4f89e397427c1b7f99c140d488b55
-size 24084480
+oid sha256:3bc92845313c3bfc3f406a9ded21d32ebc3dcc336d44a1eb791ea2281094e62c
+size 24797184

refresh_coverage.py

@@ -0,0 +1,175 @@
+"""Inventory published annotation assemblies using completion markers and metadata.
+
+Completed shards use assembly IDs from their packed sources, under the pipeline's
+strict unpack/publication contract. Shards without markers read annotation IDs
+directly. No sequences or probability columns are read. Cached results are tied
+to source and publication content hashes, so reruns reuse unchanged shards.
+"""
+import argparse
+from collections import defaultdict
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from datetime import datetime, timezone
+import hashlib
+import json
+from pathlib import Path
+import re
+import time
+
+from huggingface_hub import HfApi, HfFileSystem, get_token
+import pyarrow.parquet as pq
+
+from remote_catalog import MeasuredFile, source_info
+
+ROOT = Path(__file__).resolve().parent
+ANNOTATIONS = "HuggingFaceBio/genbank-annotations"
+INPUTS = "HuggingFaceBio/GENERanno-packed-inference-input"
+
+
+def fingerprint(value):
+    return hashlib.sha256(json.dumps(value, sort_keys=True).encode()).hexdigest()
+
+
+def read_accessions(fs, bucket, path):
+    accessions = set()
+    with MeasuredFile(fs, f"buckets/{bucket}/{path}") as stream:
+        pf = pq.ParquetFile(stream)
+        column = pf.schema.names.index("assembly_accession")
+        for group in range(pf.num_row_groups):
+            stats = pf.metadata.row_group(group).column(column).statistics
+            if stats and stats.has_min_max and stats.min == stats.max and stats.null_count == 0:
+                accessions.add(stats.min)
+            else:
+                accessions.update(pf.read_row_group(group, columns=["assembly_accession"], use_threads=False)["assembly_accession"].to_pylist())
+        fetched = stream.bytes_read
+    if any(not isinstance(a, str) or not re.fullmatch(r"GCA_\d+\.\d+", a) for a in accessions):
+        raise ValueError(f"Invalid assembly accession in {path}")
+    return accessions, fetched
+
+
+def collect(cache_dir, output, workers):
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    token = get_token()
+    api = HfApi(token=token)
+    started = datetime.now(timezone.utc).isoformat()
+    shards = defaultdict(lambda: {"objects": [], "marker": None})
+    print("Listing published annotation objects…", flush=True)
+    for item in api.list_bucket_tree(ANNOTATIONS, "annotations", recursive=True):
+        if item.type != "file":
+            continue
+        parts = item.path.split("/")
+        if len(parts) == 3 and parts[-1].endswith(".parquet"):
+            # Older publishers wrote one file per shard, without a shard folder.
+            shards["/".join(parts[1:])]["objects"].append((item.path, item.xet_hash, item.size))
+            continue
+        if len(parts) != 4:
+            if item.path.endswith(".parquet"):
+                raise ValueError(f"Unrecognized annotation layout: {item.path}")
+            continue
+        shard = "/".join(parts[1:3])
+        if parts[-1].endswith(".parquet"):
+            shards[shard]["objects"].append((item.path, item.xet_hash, item.size))
+        elif parts[-1] == "_SUCCESS.json":
+            shards[shard]["marker"] = (item.path, item.xet_hash)
+    shards = {key: val for key, val in shards.items() if val["objects"]}
+    if not shards:
+        raise ValueError("No annotation objects found")
+    print(f"Found {len(shards):,} published shards, {sum(len(v['objects']) for v in shards.values()):,} objects", flush=True)
+
+    def scan(shard, entry):
+        fs, worker_api = HfFileSystem(token=token), HfApi(token=token)
+        source = None
+        if entry["marker"]:
+            path = f"final_overlapped_pack/packed_blocks_by_division/{shard}/sources.parquet"
+            info = source_info(worker_api, INPUTS, path)
+            if info is None:
+                raise ValueError(f"Missing packed source metadata for {shard}")
+            source = (path, info.xet_hash)
+        identity = {"objects": sorted(entry["objects"]), "marker": entry["marker"], "source": source, "version": 1}
+        digest = fingerprint(identity)
+        target = cache_dir / (shard.replace("/", "__") + ".json")
+        if target.exists():
+            previous = json.loads(target.read_text())
+            if previous["fingerprint"] == digest:
+                return previous
+        accessions, fetched = set(), 0
+        marker = None
+        if source:
+            marker = json.loads(fs.cat(f"buckets/{ANNOTATIONS}/{entry['marker'][0]}"))
+            division, shard_id = shard.split("/")
+            if (marker.get("division") != division or marker.get("shard") != shard_id
+                    or int(marker.get("expected_parts", 0)) <= 0
+                    or int(marker.get("pack_count", 0)) < int(marker["expected_parts"])
+                    or int(marker.get("prob_count", 0)) < int(marker["expected_parts"])):
+                raise ValueError(f"Invalid completion marker for {shard}")
+            accessions, fetched = read_accessions(fs, INPUTS, source[0])
+            after = source_info(worker_api, INPUTS, source[0])
+            if after is None or after.xet_hash != source[1]:
+                raise ValueError(f"Source changed while reading {shard}; retry")
+            after_marker = source_info(worker_api, ANNOTATIONS, entry['marker'][0])
+            if after_marker is None or after_marker.xet_hash != entry['marker'][1]:
+                raise ValueError(f"Completion marker changed while reading {shard}; retry")
+        else:
+            for path, content_hash, _ in entry["objects"]:
+                ids, size = read_accessions(fs, ANNOTATIONS, path)
+                accessions.update(ids)
+                fetched += size
+                after = source_info(worker_api, ANNOTATIONS, path)
+                if after is None or after.xet_hash != content_hash:
+                    raise ValueError(f"Annotation changed while reading {path}; retry")
+        result = {"shard": shard, "fingerprint": digest, "accessions": sorted(accessions),
+                  "annotation_objects": len(entry["objects"]), "metadata_bytes_read": fetched,
+                  "method": "strict_completion_marker_and_packed_sources" if source else "annotation_metadata",
+                  "source": source, "marker": entry["marker"], "completion": marker}
+        temporary = target.with_suffix(".tmp")
+        temporary.write_text(json.dumps(result))
+        temporary.replace(target)
+        return result
+
+    def retry_scan(shard, entry):
+        for attempt in range(3):
+            try:
+                return scan(shard, entry)
+            except Exception:
+                if attempt == 2:
+                    raise
+                time.sleep(2 ** attempt)
+
+    results, errors = [], []
+    with ThreadPoolExecutor(max_workers=workers) as pool:
+        futures = {pool.submit(retry_scan, shard, entry): shard for shard, entry in shards.items()}
+        for future in as_completed(futures):
+            try:
+                results.append(future.result())
+            except Exception as exc:
+                errors.append(f"{futures[future]}: {type(exc).__name__}: {exc}")
+                print(errors[-1], flush=True)
+            if (len(results) + len(errors)) % 20 == 0:
+                print(f"Inventoried {len(results):,}/{len(shards):,} shards; {len(errors)} errors", flush=True)
+    if errors:
+        raise RuntimeError(f"Coverage snapshot not replaced: {len(errors)} shards failed. Retry to reuse cached successes.\n" + "\n".join(errors))
+    accessions = sorted({a for r in results for a in r["accessions"]})
+    inventory = {"created_at": datetime.now(timezone.utc).isoformat(), "listing_started_at": started,
+                 "annotation_bucket": ANNOTATIONS, "input_bucket": INPUTS,
+                 "scope": "Assemblies with published annotations; includes partial assemblies. Exact accession versions; no base completeness claim.",
+                 "method": "Completed shard assembly IDs from packed sources under strict unpack publication contract; unmarked shards read from annotation metadata. Source namespace is assumed to remain associated with its published completion marker.",
+                 "annotation_object_count": sum(r["annotation_objects"] for r in results),
+                 "shard_count": len(results), "metadata_bytes_read": sum(r["metadata_bytes_read"] for r in results),
+                 "inventory_fingerprint": fingerprint(sorted(r["fingerprint"] for r in results)),
+                 "accessions": accessions,
+                 "shards": [{k: v for k, v in r.items() if k != "accessions"} for r in sorted(results, key=lambda r: r['shard'])]}
+    output.parent.mkdir(parents=True, exist_ok=True)
+    temporary = output.with_suffix(".json.tmp")
+    temporary.write_text(json.dumps(inventory))
+    temporary.replace(output)
+    print(f"Saved {len(accessions):,} distinct assembly versions to {output}", flush=True)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--cache-dir", type=Path, default=ROOT / ".cache/coverage/shards")
+    parser.add_argument("--output", type=Path, default=ROOT / "data/coverage.json")
+    parser.add_argument("--workers", type=int, default=12)
+    args = parser.parse_args()
+    if not 1 <= args.workers <= 32:
+        parser.error("workers must be between 1 and 32")
+    collect(args.cache_dir, args.output, args.workers)

refresh_taxonomy.py

@@ -22,8 +22,12 @@ def dump_rows(archive, name):
             yield [part.strip() for part in line.decode("utf-8").split("|")]
 
 
-def build(summary, taxdump, output):
+def build(summary, taxdump, output, coverage_path=None):
     counts = Counter()
+    covered_counts = Counter()
+    inventory = json.loads(coverage_path.read_text()) if coverage_path and coverage_path.exists() else None
+    covered = set(inventory["accessions"]) if inventory else set()
+    matched = set()
     skipped = 0
     with open(summary) as stream:
         for line in stream:
@@ -42,6 +46,12 @@ def build(summary, taxdump, output):
             except ValueError:
                 taxid = -1
             counts[taxid] += 1
+            accession = row[columns["assembly_accession"]]
+            if accession in covered:
+                if accession in matched:
+                    raise ValueError(f"Duplicate current assembly accession: {accession}")
+                matched.add(accession)
+                covered_counts[taxid] += 1
     print(f"Read {sum(counts.values()):,} current assemblies", flush=True)
     parents, ranks, merged = {}, {}, {}
     with tarfile.open(taxdump, "r:gz") as archive:
@@ -51,21 +61,27 @@ def build(summary, taxdump, output):
         for row in dump_rows(archive, "merged.dmp"):
             merged[int(row[0])] = int(row[1])
         direct = Counter()
+        covered_direct = Counter()
         for taxid, count in counts.items():
+            original_taxid = taxid
             seen = set()
             while taxid in merged and taxid not in seen:
                 seen.add(taxid)
                 taxid = merged[taxid]
             direct[taxid if taxid in parents else -1] += count
+            covered_direct[taxid if taxid in parents else -1] += covered_counts[original_taxid]
         parents[-1], ranks[-1] = 1, "unresolved"
         totals = Counter()
+        covered_totals = Counter()
         for taxid, count in direct.items():
+            covered_count = covered_direct[taxid]
             seen = set()
             while True:
                 if taxid in seen:
                     raise ValueError("Taxonomy cycle detected")
                 seen.add(taxid)
                 totals[taxid] += count
+                covered_totals[taxid] += covered_count
                 if taxid == 1:
                     break
                 taxid = parents[taxid]
@@ -76,6 +92,8 @@ def build(summary, taxdump, output):
                 names[taxid] = row[1]
     if not totals[1] or totals[1] != sum(counts.values()):
         raise ValueError("Invalid root assembly count")
+    if covered_totals[1] != len(matched) or any(covered_totals[t] > totals[t] for t in totals):
+        raise ValueError("Invalid covered assembly count")
     provenance = {}
     for path in (summary, taxdump):
         digest = hashlib.sha256()
@@ -91,18 +109,24 @@ def build(summary, taxdump, output):
                 "assembly_count": totals[1], "unresolved_assemblies": direct[-1],
                 "taxa_with_assemblies": len(totals), "skipped_rows": skipped,
                 "scope": "Current GCA assemblies in NCBI assembly_summary_genbank.txt; version_status=latest. Counts include all assembly levels and taxonomic groups. Not all GenBank nucleotide records."}
+    if inventory:
+        metadata["coverage"] = {k: v for k, v in inventory.items() if k not in ("accessions", "shards")}
+        metadata["coverage"].update(assembly_versions_in_inventory=len(covered),
+                                    matched_current_assemblies=len(matched),
+                                    unmatched_assembly_versions=len(covered - matched),
+                                    unmatched_accession_examples=sorted(covered - matched)[:25])
     output.parent.mkdir(parents=True, exist_ok=True)
     temporary = output.with_suffix(".sqlite.tmp")
     temporary.unlink(missing_ok=True)
     try:
         with sqlite3.connect(temporary) as conn:
-            conn.executescript("CREATE TABLE taxa (taxid INTEGER PRIMARY KEY, parent_id INTEGER, name TEXT, rank TEXT, direct_count INTEGER, total_count INTEGER); CREATE INDEX parents ON taxa(parent_id, total_count DESC); CREATE TABLE metadata (value TEXT);")
-            conn.executemany("INSERT INTO taxa VALUES (?,?,?,?,?,?)", ((t, parents[t], names.get(t, str(t)), ranks[t], direct[t], n) for t, n in totals.items()))
+            conn.executescript("CREATE TABLE taxa (taxid INTEGER PRIMARY KEY, parent_id INTEGER, name TEXT, rank TEXT, direct_count INTEGER, total_count INTEGER, covered_direct INTEGER, covered_total INTEGER); CREATE INDEX parents ON taxa(parent_id, total_count DESC); CREATE TABLE metadata (value TEXT);")
+            conn.executemany("INSERT INTO taxa VALUES (?,?,?,?,?,?,?,?)", ((t, parents[t], names.get(t, str(t)), ranks[t], direct[t], n, covered_direct[t], covered_totals[t]) for t, n in totals.items()))
             conn.execute("INSERT INTO metadata VALUES (?)", (json.dumps(metadata),))
         temporary.replace(output)
     finally:
         temporary.unlink(missing_ok=True)
-    print(json.dumps(metadata, indent=2), flush=True)
+    print(json.dumps({k: v for k, v in metadata.items() if k != "sources"}, indent=2), flush=True)
 
 
 def main():
@@ -110,6 +134,7 @@ def main():
     parser.add_argument("--cache-dir", type=Path, default=ROOT / ".cache/coverage")
     parser.add_argument("--output", type=Path, default=ROOT / "data/taxonomy.sqlite")
     parser.add_argument("--download", action="store_true", help="Download fresh NCBI metadata before rebuilding (about 2 GB).")
+    parser.add_argument("--coverage", type=Path, default=ROOT / "data/coverage.json", help="Published annotation inventory from refresh_coverage.py, used when present.")
     args = parser.parse_args()
     args.cache_dir.mkdir(parents=True, exist_ok=True)
     if args.download:
@@ -122,7 +147,7 @@ def main():
                     out.write(chunk)
             temporary.replace(target)
             target.with_suffix(target.suffix + ".source.json").write_text(json.dumps(source))
-    build(args.cache_dir / "assembly_summary_genbank.txt", args.cache_dir / "taxdump.tar.gz", args.output)
+    build(args.cache_dir / "assembly_summary_genbank.txt", args.cache_dir / "taxdump.tar.gz", args.output, args.coverage)
 
 
 if __name__ == "__main__":

taxonomy.py

@@ -15,6 +15,7 @@ class Taxonomy:
         self.path = Path(path)
         with closing(self.connect()) as conn:
             self.metadata = json.loads(conn.execute("SELECT value FROM metadata").fetchone()[0])
+        self.coverage = self.metadata.get("coverage")
 
     def connect(self):
         conn = sqlite3.connect(f"{self.path.resolve().as_uri()}?mode=ro", uri=True)
@@ -36,16 +37,18 @@ class Taxonomy:
             root = conn.execute("SELECT * FROM taxa WHERE taxid=?", (int(taxid),)).fetchone()
             if root is None:
                 raise ValueError("Taxon is absent from this assembly snapshot")
-            ids, labels, parents, values, details = [], [], [], [], []
+            ids, labels, parents, values, details, colors = [], [], [], [], [], []
 
-            def add(key, label, parent, count, rank):
+            def add(key, label, parent, count, rank, covered):
                 ids.append(str(key)); labels.append(label); parents.append(str(parent))
                 values.append(count)
-                details.append([rank, count / root['total_count'] * 100])
+                percent = 100 * covered / count if count else 0
+                colors.append(percent)
+                details.append([rank, count / root['total_count'] * 100, covered, percent])
 
             def visit(row, parent="", depth=0):
                 key = row['taxid']
-                add(key, row['name'], parent, row['total_count'], row['rank'])
+                add(key, row['name'], parent, row['total_count'], row['rank'], row['covered_total'] if self.coverage else 0)
                 if depth >= 3:
                     return
                 children = conn.execute("SELECT * FROM taxa WHERE parent_id=? AND taxid!=? ORDER BY total_count DESC", (key, key)).fetchall()
@@ -54,9 +57,10 @@ class Taxonomy:
                     visit(child, key, depth + 1)
                 other = sum(c['total_count'] for c in children[8:])
                 if other:
-                    add(f"other-{key}", f"Other taxa ({len(children) - 8:,})", key, other, "Grouped for display; search to explore")
+                    other_covered = sum(c['covered_total'] for c in children[8:]) if self.coverage else 0
+                    add(f"other-{key}", f"Other taxa ({len(children) - 8:,})", key, other, "Grouped for display; search to explore", other_covered)
                 if row['direct_count'] and children:
-                    add(f"direct-{key}", "Assigned directly to this taxon", key, row['direct_count'], "direct assignments")
+                    add(f"direct-{key}", "Assigned directly to this taxon", key, row['direct_count'], "direct assignments", row['covered_direct'] if self.coverage else 0)
 
             visit(root)
             children = conn.execute("SELECT * FROM taxa WHERE parent_id=? AND taxid!=? ORDER BY total_count DESC LIMIT 200", (root['taxid'], root['taxid'])).fetchall()
@@ -66,12 +70,21 @@ class Taxonomy:
                 if current['taxid'] == 1:
                     break
                 current = conn.execute("SELECT * FROM taxa WHERE taxid=?", (current['parent_id'],)).fetchone()
+        marker = dict(colors=colors, cmin=0, cmax=100, showscale=True,
+                      colorscale=[[0, "#e2e8f0"], [0.000001, "#ccece6"], [0.1, "#80cdc1"], [0.5, "#279e91"], [1, "#005a50"]],
+                      colorbar=dict(title="Assemblies with<br>annotations (%)", tickvals=[0, 25, 50, 75, 100]),
+                      line=dict(color="white", width=1)) if self.coverage else {}
+        hover = "<b>%{label}</b><br>%{customdata[0]}<br>%{value:,} assemblies<br>%{customdata[1]:.2f}% of selected group"
+        if self.coverage:
+            hover += "<br><b>%{customdata[2]:,} with published annotations</b><br>%{customdata[3]:.2f}% of assemblies in this group"
         figure = go.Figure(go.Sunburst(ids=ids, labels=labels, parents=parents, values=values,
                                       branchvalues="total", customdata=details, sort=False,
-                                      hovertemplate="<b>%{label}</b><br>%{customdata[0]}<br>%{value:,} assemblies<br>%{customdata[1]:.2f}% of selected group<extra></extra>"))
+                                      marker=marker, hovertemplate=hover + "<extra></extra>"))
         figure.update_layout(height=620, margin=dict(t=10, b=10, l=10, r=10), template="plotly_white")
         count = root['total_count']
         note = f"### {root['name']}\n**{count:,} assemblies** · {count / self.metadata['assembly_count']:.2%} of this GenBank assembly snapshot\n\n" + " → ".join(reversed(lineage))
+        if self.coverage:
+            note += f"\n\n**{root['covered_total']:,} / {count:,} assemblies with published annotations ({root['covered_total'] / count:.2%})**. Assemblies may be partially annotated."
         choices = [(f"{r['name']} · {r['total_count']:,}", str(r['taxid'])) for r in children]
         return figure, note, choices, str(root['parent_id'])
 
@@ -84,7 +97,20 @@ def build_taxonomy_tab():
     meta = taxonomy.metadata
     gr.Markdown(f"## GenBank genome assemblies\n**{meta['assembly_count']:,} assemblies** · Snapshot built **{meta['created_at'][:10]}**\n\n"
                 "Explore the NCBI taxonomy by assembly count. This inventory covers current GenBank genome assemblies; "
-                "it does not count every nucleotide submission or indicate which genomes we have annotated.")
+                "it does not count every nucleotide submission.")
+    if taxonomy.coverage:
+        coverage = taxonomy.coverage
+        gr.Markdown(f"**Annotation inventory updated {coverage['created_at'][:10]}** · "
+                    f"**{coverage['matched_current_assemblies']:,} assemblies** have published annotations\n\n"
+                    "**Area = assembly count. Color = percentage of assemblies with published annotations.** "
+                    "Gray means no published annotations found in this inventory; darker teal means a higher fraction. "
+                    "An assembly counts once if it has annotations, including partial assemblies. This is not the percentage of bases annotated. "
+                    "The Annotation lookup tab currently searches a smaller subset.")
+        if coverage['unmatched_assembly_versions']:
+            gr.Markdown(f"{coverage['unmatched_assembly_versions']:,} published assembly versions are outside the current GenBank snapshot "
+                        "and are excluded from the chart's percentages. Matching uses exact accession versions.")
+    else:
+        gr.Markdown("Annotation coverage inventory is unavailable; colors distinguish taxonomic branches only.")
     initial, initial_note, initial_choices, _ = taxonomy.view("1")
     selected = gr.State("1")
     parent = gr.State("1")
@@ -102,7 +128,7 @@ def build_taxonomy_tab():
     gr.Markdown("Click chart sectors to expand the displayed branches; click the center to zoom out. "
                 "Use **Explore a child group** or **Find a taxon** to load deeper branches. "
                 "Each view includes three descendant levels and the eight largest children per node. "
-                "**Other taxa** preserves the remaining assembly counts. Colors distinguish branches; they do not show annotation coverage.")
+                "**Other taxa** preserves the remaining assembly counts and combines their annotation counts before calculating its percentage.")
     with gr.Accordion("Snapshot sources and scope", open=False):
         gr.Markdown("Source: [NCBI GenBank assembly summary](https://ftp.ncbi.nlm.nih.gov/genomes/ASSEMBLY_REPORTS/assembly_summary_genbank.txt) "
                     "and [NCBI Taxonomy](https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz). "