Buckets:

1.12 TB
339,840 files
Updated 4 days ago
Name
Size
_internal
catalog
cytearc_docs
cytearc_networks
cytearc_tests
datasets
notebooks
README.md7.66 kB
xet
README.md

Cytebase

Analysis-ready single-cell RNA data, streamed straight into your Python session.

Cytebase is a catalog of public single-cell datasets, each converted into a Scarf DataStore and hosted in this bucket. Instead of downloading multi-gigabyte H5AD files, you search the catalog, open a dataset, and read only the cells, genes and metadata you need over the network. Computation runs where your Python session runs: a laptop, a notebook server or cloud compute.

  • 1,369 datasets from 373 studies, about 177 million cells, from CZ CELLxGENE Discover, selected from a snapshot taken on 29 September 2026
  • 10 species, including 1,006 human and 329 mouse datasets
  • Raw counts as published by the original authors, with cell annotations and, where valid, the authors' UMAP coordinates
  • No credentials needed: the bucket is public and the SDK connects to it by default

Quick start

Requires Python 3.12 or newer.

uv pip install --prerelease allow 'scarf[cytebase]' jupyterlab
from scarf import cytebase

catalog = cytebase.Catalog()   # connects to this public bucket

# Search by words (author, tissue, disease, ...)
matches = catalog.search("wilson kidney cortex")
entry = catalog.dataset(matches[0]["cytebase_id"])
entry                          # title, citation, links to the original study

# Or filter by exact ontology labels
catalog.list_terms("tissue")
catalog.find_datasets(organism="Homo sapiens", tissue="kidney")

# Open the dataset; counts stay remote and are fetched on demand
ds = catalog.open_datastore(entry.id)
meta = ds.cells.to_pandas_dataframe(["ids", "cell_type", "donor_id"], key="I")

# Plot the authors' UMAP
umap = cytebase.embedding(ds, "X_umap")
ds.plots.embedding(layout=umap, color_by="cell_type", figsize=(10, 7))

The catalog is a small DuckDB database cached on your machine and verified against its published checksum. You can also query it with SQL:

catalog.query(
    "SELECT cytebase_id, first_author, year, cell_count FROM datasets "
    "WHERE status = ? ORDER BY cell_count DESC LIMIT 10",
    parameters=["ready"],
)

Opened stores are read-only. To save your own results (clusterings, embeddings, markers), mount a local location as described in the Cytebase tutorial.

Example notebooks

The Hugging Face file viewer does not display notebooks of this size, so the links below open rendered copies on nbviewer. To run one, download it from the notebooks folder.

Notebook What it shows Runtime
Catalog tour The collection at a glance by organism, assay, tissue, disease and year; search by text, exact labels and SQL across studies; one dataset's UMAP ~2 min
COVID-19 case study COVID-19 and healthy blood (Wilk et al., 2020): study design, composition per donor, a marker dot plot, and an interferon response compared between donors ~5 min
UMAP gallery Published UMAPs across tissues and species, styling options, 1.1 million Tabula Sapiens cells, and a custom matplotlib figure ~15 min
curl -LO https://huggingface.co/buckets/Nygen/cytebase/resolve/notebooks/cytebase_01_catalog_tour.ipynb

Most of the runtime is spent reading from the network. The saved outputs reflect the catalog when the notebooks were executed.

Rate limits

Anonymous Hugging Face access allows 500 API requests per 5 minutes per IP address, and opening a store uses a share of them. If a cell stops with 429 Too Many Requests, wait five minutes and run it again, or sign in with hf auth login: Catalog() then reads with your token, which has a higher limit.

What is included

Datasets are selected from CELLxGENE Discover by these rules:

  • Primary cells only. Datasets made up entirely of cells published elsewhere (secondary copies, such as integrated atlases) are left out, so cells are not duplicated.
  • RNA assays. Only datasets whose assays are reviewed RNA assays are included: droplet, plate-based and spatial transcriptomics. ATAC-only and DNA methylation datasets are excluded.
  • Raw counts. Conversion uses the counts matrix that CELLxGENE identifies (usually raw.X), with its values stored exactly. Normalized matrices are not imported.

Each dataset carries:

Content Description
RNA counts Cell-major and gene-major copies for fast access by cell or by gene
Cell metadata All CELLxGENE annotations (cell type, tissue, disease, donor, assay, ...) plus Scarf QC columns such as RNA_nCounts
Gene metadata Gene IDs, names and source feature columns
UMAP The authors' X_umap, when present and valid; datasets whose source UMAP contains missing coordinates are included without it
Provenance The CELLxGENE collection and dataset records, source checksum and build receipt

Metadata column names that contain / or \ are stored with _ in their place (for example Baseline eGFR (ml/min/1.73m2) (Binned) becomes Baseline eGFR (ml_min_1.73m2) (Binned)).

Bucket layout

catalog/
  cytebase.duckdb            searchable catalog of all datasets
  cytebase.duckdb.sha256     checksum used to verify the catalog
datasets/{cytebase_id}/
  README.md                  dataset summary and citation
  dataset.json               registration and processing record
  cellxgene/                 original CELLxGENE collection and dataset metadata
  data.zarr/                 the Scarf DataStore
  scarf_ingest.json          conversion and verification receipt
notebooks/                   executed example notebooks

The SDK is the supported way to read the data. The files can also be addressed directly as hf://buckets/Nygen/cytebase/....

Citing and licensing

Every dataset belongs to its original authors. When you use a dataset, cite the publication shown in its catalog entry and README.md, and acknowledge CZ CELLxGENE Discover:

CZI Cell Science Program et al. CZ CELLxGENE Discover: a single-cell data platform for scalable exploration, analysis and modeling of aggregated data. Nucleic Acids Research 53, D886-D900 (2025). https://doi.org/10.1093/nar/gkae1142

Source datasets keep the license under which they were published on CELLxGENE Discover.

If Cytebase or Scarf helps your work, please also cite:

Dhapola et al. Scarf enables a highly memory-efficient analysis of large-scale single-cell genomics data. Nature Communications 13, 4616 (2022). https://doi.org/10.1038/s41467-022-32097-3

Scarf is open source under the BSD 3-Clause License.

About Nygen

Cytebase is built and maintained by Nygen, the team behind the open-source Scarf toolkit.

Total size
1.12 TB
Files
339,840
Last updated
Nov 14
Pre-warmed CDN
US EU US EU

Contributors