VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics
Paper • 2610.28266 • Published
Request access to VGL-stGen 0.5 (Lung)
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
VGL-stGen 0.5 (Lung) is released for non-commercial research use only. The weights are a Model Derivative of google/medgemma-1.5-4b-it and are governed by the Health AI Developer Foundations (HAI-DEF) Terms of Use (https://developers.google.com/health-ai-developer-foundations/terms), which you must accept and pass on to anyone you redistribute to. Portrai's contribution (the adaptation, the SDK code and the documentation) is additionally licensed under CC BY-NC 4.0. The model is not a medical device and is not authorized for clinical use. Access requests are reviewed manually by Portrai; for commercial use, contact Portrai, Inc. (https://portrai.io).
Log in or Sign Up to review the conditions and access this model content.
VGL-stGen 0.5 (Lung)
VGL-stGen adapts MedGemma-1.5-4B to connect lung histology, gene expression and cellular identity. One checkpoint supports thirteen tasks from combinations of H&E patches, ranked gene lists, cell types and discrete expression levels through a typed Python API.
Paper: VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics.
Haenara Shin, Dongjoo Lee, Sungwoo Bae, Kwon Joong Na, Daeseung Lee and Hongyoon Choi (2026).
Access: The model card is public. Repository files, SDK and weights require approval through the access form on this page. After approval, authenticate your Hugging Face account before downloading. See installation and download instructions.
Model Overview
VGL-stGen adapts the language component of MedGemma while keeping its vision tower frozen. Measured genes provide teacher labels for image-to-cell-type training. For conditional expression completion, the interface fixes the queried gene identities and predicts their levels using observed expression and an image or cell type. Training combines 69 paired ST–H&E samples from PortraiATLAS (47 Xenium and 22 Visium), single-cell references and 30,965 unique cancer patches. The sample count is not an independent-patient count. The diagram summarises training and supported inputs; not every input combination was directly supervised.
| Component | Configuration |
|---|---|
| Backbone | google/medgemma-1.5-4b-it |
| Adaptation | Language-side QLoRA; frozen vision tower |
| Final adapter | Six rank-16 updates averaged through rank concatenation at rank 96 |
| Inputs | H&E patches, ranked marker genes, cell-type labels and discrete expression levels |
| Outputs | Cell or tissue labels, gene lists and masked expression levels |
| Evaluated scope | Lung; nine tasks measured on external cohorts and four expression tasks measured internally |
Overview PDF · Task map and evaluation · Detailed implementation
Quick example
import vgl_gen as vgl
from PIL import Image
m = vgl.load(tier="internal")
m.predict({"image": Image.open("patch.png")}, want="cancer_binary").value
# 'cancer'
m.predict({"genes": ["SFTPC", "SFTPB", "NAPSA"]}, want="celltype").value
# 'epithelial_cell'
Each answer records its task, evaluation provenance and weight variant. Read the reported metric together with its evaluation condition; performance varies across tasks and input combinations.
The examples below illustrate the API; displayed labels and confidence values are examples, not additional benchmark measurements.
Lung only. Research use. Not a diagnostic device. See MODEL_CARD.md for what
was measured, how well each route does, and where it fails.
Contents
- Install
- Get the weights
- Quickstart
- Recipes
- What comes back
- Picking a route
- Free-form questions
- Prior correction
- API reference
- Performance at a glance
- Rules worth following
- Troubleshooting
1. Install
| Python | 3.10 or newer |
| GPU | one CUDA device, roughly 10 GB free |
| Disk | about 9 GB for weights |
| Git | with Git LFS, to fetch the weights |
python3.10 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
# CUDA 12.1 build; match this to your driver
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install "transformers==4.50.0" "peft==0.17.1" pillow numpy
Pin rather than float: newer transformers releases change generation internals.
Keep torchvision installed: the recorded image-task configuration uses the fast
image processor (use_fast=True). A slow processor can change gene-list outputs.
Check it:
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
2. Get the weights
Request access on the model page and wait for approval. Log in with a Hugging Face token belonging to the approved account; a token alone does not grant access. Repository files are gated, including the detailed card and downloadable figures. The paper is publicly available.
python -c "from huggingface_hub import login; login(add_to_git_credential=True)"
Weights ship under Git LFS. Cloned without LFS, they are small text pointers rather than tensors. Run these commands in the environment created above:
git lfs install
git clone https://huggingface.co/Portraier/VGL-stGen-0.5-Lung VGL-Gen && cd VGL-Gen
git lfs pull # only if you cloned before installing LFS
du -sh weights/* # merged_bf16 ~8.1G, adapter_post6 ~751M
pip install -e . --no-deps # preserve the pinned environment
From the cloned repository, optional checks are:
pip install pytest pandas scikit-learn pyarrow
pytest -m "not gpu" -q # no GPU or weights required
pytest -m gpu -q # loads the checkpoint
| Path | Size | Use it when |
|---|---|---|
weights/merged_bf16/ |
8.1 GB | Default. No separate base-model download; access to this repository is required. |
weights/adapter_post6/ |
751 MB | You want the light artefact, or you need ask() |
m = vgl.load(tier="internal") # merged, the default
m = vgl.load(tier="internal", weights="adapter") # the LoRA variant
m = vgl.load(tier="internal", weights_dir="/mnt/ckpt") # mounted elsewhere
export VGL_GEN_WEIGHTS_ROOT=/mnt/checkpoints # applies to every variant
The adapter path loads the gated base
google/medgemma-1.5-4b-itfrom Hugging Face and needs the corresponding access and a token. The merged path uses the weights in this repository. Adapter and merged results can differ: recorded matched-subset score differences reach 0.0049 in either direction.
The SDK defaults to merged BF16 weights. The paper's matched multimodal benchmark uses merged BF16 for binary tumour prediction (task 4) and the NF4-base adapter for the other tasks. API examples demonstrate usage; reproducing a published score also requires the corresponding task's processor, precision, decoding and batching settings.
3. Quickstart
import vgl_gen as vgl
from PIL import Image
m = vgl.load(tier="internal")
img = Image.open("patch.png")
genes = ["SFTPC", "SFTPB", "NAPSA", "EPCAM", "KRT7"] # descending expression order
r = m.predict({"image": img}, want="cancer_binary")
r.value # 'cancer'
r.confidence # 0.82
r.certified # True — this route was measured on an external cohort
r.metric # {'cohort': 'TCGA-LUAD', 'balanced_acc': 0.7532, 'chance': 0.5, ...}
print(r.to_json(indent=2))
Gene order matters. Pass genes in descending expression order. In a recorded 23,115-item control, shuffling changes balanced accuracy from 0.6293 to 0.5607. That subset is separate from the full-cohort scoring passes. Only the first twenty input genes are read.
4. Recipes
Each block is complete once m, img and genes exist.
Is this patch cancer?
r = m.predict({"image": img}, want="cancer_binary")
r.value # 'cancer' | 'non-cancer'
r.confidence # 0.82
What cell type is this?
Give it genes if you have them; they are worth far more than the image here.
m.predict({"genes": genes}, want="celltype") # best
m.predict({"image": img, "genes": genes}, want="celltype")
m.predict({"image": img}, route="image->celltype") # weakest; name it explicitly
Which tissue category is this patch?
Nine classes. Use prior correction — raw output collapses onto one class.
rs = m.predict_batch(items, want="cancer_multiclass", calibration="prior_corrected")
rs[0].value # one of TUMOR STROMA MIXED IMMUNE VESSEL BRONCHI NECROSIS LUNG BACKGROUND
Which genes would this express?
m.predict({"image": img}, want="top_genes").value
# ['MUC1', 'EPCAM', 'VEGFA', ...] — requested length: 20; inspect the returned list
m.predict({"celltype": "t_cell"}, want="top_genes").value
# ['CCL5', 'CD52', 'CXCR4', 'CD3D', 'TRAC', ...]
Fill in the genes I am missing
Pass what you know as partial_genes. The requested output length is
20 - len(partial_genes); inspect the returned list when using generated genes.
known = genes[:15]
m.predict({"image": img, "partial_genes": known}, want="top_genes") # requests 20 - len(known) genes
m.predict({"celltype": "t_cell", "partial_genes": known}, want="top_genes")
…and if you have neither an image nor a cell type:
r = m.complete_genes(known)
r.value # the remaining genes
r.via_celltype # 'epithelial_cell' — the step it went through
r.via_celltype_confidence # 0.964
complete_genes infers the cell type first, then completes against it. No single
route does this, because no such task was trained.
A whole slide
items = [{"image": tile, "slide_id": sid} for tile, sid in tiles]
results = m.predict_batch(items, want="cancer_binary")
[r.value for r in results]
Metadata keys (slide_id, cell_id, sample_name, id) ride along untouched and
are what group_by uses for prior correction.
Anything else
m_adapter = vgl.load(tier="internal", weights="adapter")
m_adapter.ask("What does high EPCAM expression indicate in lung tissue?").value
See §7 — it does not run on the fine-tuned weights, and it says so.
Your own prompt
r = m.predict({"image": img}, want="cancer_binary",
prompt="Describe whether this lung tissue patch contains tumour.")
r.certified # False
A custom prompt gives up certification automatically: the measured number was obtained with the canonical wording and no longer describes what ran.
5. What comes back
Every result carries the same provenance fields and serialises with .to_dict() /
.to_json(), discriminated by output_type.
{
"output_type": "CellType",
"value": "epithelial_cell",
"confidence": 0.91,
"certified": true,
"metric": {"cohort": "251013_Lung_External", "balanced_acc": 0.6297, "chance": 0.083},
"route": "genes->celltype",
"calibration": "raw",
"weights_path": "merged_bf16",
"warnings": []
}
output_type |
From | value |
confidence |
Extra |
|---|---|---|---|---|
CancerBinary CancerMulticlass CellType |
closed-label routes | one label | step-0 softmax | probs, label_space |
GeneList |
the four gene routes | list of symbols | None |
— |
GeneListFromChain |
complete_genes() |
list of symbols | None |
via_celltype, via_celltype_confidence |
FreeText |
ask() |
prose | None |
answered_by, degenerate, suggested_route |
Closed-label routes cannot return anything outside their label space — a logits
processor masks every token that cannot continue a permitted label, so value is
always exactly one of them and confidence is the softmax over the permitted first
tokens. Open-generation routes skip that and report no confidence.
Read each score with its metric and evaluation condition. Binary cancer uses balanced accuracy because tumour prevalence is 0.1509; an always-negative answer scores 0.849 raw accuracy. The internal levels-to-cell-type task retains its recorded raw accuracy. Nominal chance references do not establish clinical value.
6. Picking a route
predict takes what you have and what you want, and picks the route that uses the
most of it.
m.predict({"genes": genes}, want="celltype")
m.predict({"image": img, "genes": genes}, want="celltype")
Inputs: image (PIL), genes, partial_genes, celltype (label-space
spelling, e.g. t_cell), expr_bins, partial_expr_bins.
want: cancer_binary, cancer_multiclass, celltype, top_genes,
very_high_genes, expr_bins.
expr_bins is a finished level list ("GENE(Level)"); partial_expr_bins is the
same list with some levels written (__). They are different inputs and pick
different routes.
When two routes tie on how much of your input they consume, predict raises rather
than choose, because the candidates differ in measured quality:
m.predict({"image": img, "celltype": "t_cell", "partial_genes": known},
want="top_genes")
# AmbiguousRouteError: image+partial_genes->top_genes vs celltype+partial_genes->top_genes
m.predict({"image": img, "celltype": "t_cell", "partial_genes": known},
route="image+partial_genes->top_genes") # name it
If nothing can serve your inputs you get MissingInputError naming what each
candidate needs. For a partial gene list with no image and no cell type, that is
expected — use complete_genes().
List what is available:
m.routes() # all thirteen, with the inputs each needs
m.routes(certified_only=True) # the nine measured on an external cohort
Expression levels
Four tasks use expression levels. Their scores come from internal evaluations;
the SDK marks these separately from the external-cohort tasks. The NF4 adapter
requires bitsandbytes in addition to the basic installation.
import warnings, vgl_gen as vgl
# Match the recorded expression-task configuration: NF4 adapter and eager attention
m = vgl.load(tier="internal", weights="adapter_4bit", attn_implementation="eager")
partial = ["VEGFA(__)", "SFTPC(Low)", "HPGD(__)", "A2M(Very High)", ...] # 100 entries
r = m.predict({"image": img, "partial_expr_bins": partial, "scope": "REGION"},
want="expr_bins")
# UncertifiedRouteWarning: ... held-out measurement only: pair_f1 0.6596 ...
r.value # ['VEGFA(Low)', 'HPGD(Med)', ...] — one entry per (__) slot
r.certified # False
r.internal # {'pair_f1': 0.6596, 'baselines': {...}, 'split': ..., 'n': 1000}
The gene names are read out of your prompt, not predicted: the SDK parses the
(__) entries and asks the model only which of the five levels fills each slot.
In the recorded comparison, unconstrained generation scores pair-F1 0.0361,
whereas this fixed-name readout scores 0.6596 for the image-conditioned task.
Decoding uses one model forward pass per masked slot. Runtime depends on the hardware, input length and inference configuration.
These four tasks have internal measurements carried in r.internal. Expression-task
train/test sample and recorded cell/spot identifiers are disjoint. Earlier gene-list
training includes all seven masked-image benchmark samples and 692 of its 1,000
cells or spots with matching identifiers and source images; this benchmark does
not establish patient independence across all training stages. An external test
matching the gene panel and level definitions has not been performed.
Very-high-gene evaluation supplies the true answer length. Levels-to-cell-type evaluation uses the first ten sampled gene–level pairs. Frequency-only baselines do not receive the visible per-cell expression context. The paper also reports a separate controlled adaptation study; its additional-training checkpoints do not replace the weights released here. See the paper or detailed evaluation conditions.
Silence it once you have read the numbers:
warnings.filterwarnings("ignore", category=vgl.UncertifiedRouteWarning)
7. Free-form questions
m = vgl.load(tier="internal", weights="adapter") # note: adapter, not merged
r = m.ask("What does high EPCAM expression indicate in lung tissue?")
r.value # prose
r.certified # False, always
r.answered_by # 'base_model'
r = m.ask("What cell type is this?", inputs={"genes": genes})
r.suggested_route # 'genes->celltype' — use that instead
ask() answers from the base model, with the adapter switched off. Recorded
free-form tests of the adapted weights often repeated cell-type labels. The SDK
therefore uses the base model for this helper; the thirteen-task evaluation does
not describe its free-form answers.
That is also why it needs weights="adapter" — merged weights have the LoRA baked
in with nothing to switch off, and ask() raises AskUnavailableError rather than
return something useless.
What it does for you: a scoped system prompt that declines clinical advice, typed
grounding through inputs=, a pointer to a measured route when one fits your
inputs, and a degenerate flag if the answer collapses. For anything you intend to
act on, use a measured route.
8. Prior correction
Prior correction divides each class probability by its mean over a target group, then renormalises. The effect depends on the task and grouping; the SDK applies it only when explicitly requested.
m.predict_batch(items, want="cancer_multiclass",
calibration="prior_corrected", group_by="slide")
| Route | Use | Why |
|---|---|---|
image->cancer_multiclass |
exploratory prior_corrected |
The same adapter probability dump changes from macro-F1 0.1200 to 0.2801 using the full 39-slide pool |
genes->celltype |
raw |
Correction buys recall with precision — balanced 0.627 → 0.688 but macro-F1 0.602 → 0.445 |
image->cancer_binary |
not supported | No change in the recorded binary diagnostic; this is not a general invariance theorem |
It is transductive: the prior comes from the batch you pass, so predict
refuses it and predict_batch supports it. Pass group_by="slide" to make a
slide's answers independent of how you batched.
The recorded 0.2801 uses one pool over 39 slides, not separate per-slide pools. Correction was selected after labelled evaluation and is exploratory. The per-slide SDK example above is a different operating condition.
9. API reference
| Call | Returns |
|---|---|
vgl.load(tier, weights, device, batch_size, seed, weights_dir) |
VglGen |
m.predict(inputs, want, route, calibration, allow_uncertified, q, prompt) |
one result |
m.predict_batch(items, want, route, calibration, group_by, allow_uncertified, q, prompt) |
list of results |
m.complete_genes(partial_genes, n_genes, calibration) |
GeneListFromChain |
m.ask(question, inputs, max_new_tokens) |
FreeText |
m.routes(certified_only) |
list of dicts: route, certified, inputs, output_type, task |
m.capabilities() |
the whole manifest |
Only inputs, items, question and partial_genes are positional in practice.
Defaults that matter: tier="internal", weights="merged", calibration="raw",
n_genes=20, max_new_tokens=320, allow_uncertified=False.
Errors
All inherit VglGenError; vgl.ALL_ERRORS is the tuple.
| Error | Fires when |
|---|---|
MissingInputError |
your inputs cannot reach the requested output |
AmbiguousRouteError |
two routes consume the same amount of your input |
UnknownRouteError |
the route or want string is not one this checkpoint serves |
UncertifiedRouteError |
a route with no measurement at all was called (currently unreachable) |
UncertifiedRouteWarning |
(warning) an internally-measured route ran; read r.internal |
AttentionMismatchWarning |
(warning) an expression route ran under non-eager attention |
AskUnavailableError |
ask() on merged weights, whose adapter cannot be disabled |
CalibrationNotSupportedError |
prior correction where it is meaningless |
CalibrationNeedsGroupError |
prior correction without a grouping to estimate it on |
WeightsNotFoundError |
the checkpoint or label-space file is not where the manifest says |
ManifestError |
the manifest is malformed |
Reading the manifest
from vgl_gen import manifest as mf
man = m.capabilities()
mf.certified_routes(man) # the nine
mf.metric_summary(man, "genes->celltype") # headline numbers for a route
mf.composition_metric(man, "genes->top_genes") # …for complete_genes
10. Performance at a glance
The table below preserves the SDK's recorded evaluation summaries. The public paper and ancillary results provide protocols, comparators and uncertainty estimates; MODEL_CARD.md contains the detailed card after access approval.
| Route | Headline | Chance |
|---|---|---|
image->cancer_binary |
balanced 0.7532, AUROC 0.8266 | 0.5 |
genes->celltype |
balanced 0.6297 | 0.083 |
image+genes->celltype |
balanced 0.5394 | 0.083 |
image->cancer_multiclass |
macro-F1 0.1200 → 0.2801, same adapter probability dump, 39-slide pooled correction | nominal 0.111 |
image->celltype |
balanced 0.2012 | 0.083 |
image->top_genes |
gene-set F1 0.1173 | 0.044 |
celltype->top_genes |
gene-set F1 0.0965 | 0.043 |
image+partial_genes->top_genes |
gene-set F1 0.0604 | 0.013 |
celltype+partial_genes->top_genes |
gene-set F1 0.0484 | 0.010 |
complete_genes() (composition) |
gene-set F1 0.0472 | 0.010 |
The SDK's certified field denotes a recorded external-cohort measurement.
Classification references are silver labels: ARA-CNN segmentations for cancer and
atlas transfers for cell type. Four image-conditioned tasks have partial spatial-tissue exposure: three of the
four evaluated sections supplied cancer-training H&E, while the Visium squamous
cell carcinoma section did not. The TCGA set includes its development subset. These scores
measure agreement under the conditions described in MODEL_CARD.md.
The paper's main gene-only comparison reports balanced accuracy 0.6293 and macro-F1 0.6023. A separate constrained-generation evaluation on the same 224,210 items gives 0.6297 and 0.6029; the SDK summary retains that evaluation's 0.6297. These are separate scoring records and should not be combined into a new metric pair.
Comparison with unadapted multimodal models
The paper compares the released checkpoint with ten unadapted medical or general-purpose vision-language models across all thirteen tasks. Inputs and task protocols are matched while each model retains its native image processor and chat template. Task 4 uses BF16; the other tasks use NF4. Depending on the task, prediction uses allowed-label decoding, generated gene lists with parsing, or fixed-name autoregressive expression-level choices.
The figure reproduces the paper's Figure 5. Printed values are original scores; colours are column-wise z-scores across eleven models. T5 raw and pooled-corrected scores are shown separately. T1–3 predict cell type from genes, image + genes, or image; T4–5 predict binary tumour status or nine tissue classes from images; T6–7 predict genes from an image or cell type; T8–9 complete partial gene lists using an image or cell type; T10 predicts very-high-expression genes; T11–12 complete queried expression levels from observed levels plus an image or cell type; T13 predicts cell type from gene–level pairs.
VGL-stGen reaches gene-based cell-type balanced accuracy 0.6293, compared with 0.5035 for Gemma4-31B. Binary tumour balanced accuracy in this matched comparison is 0.7524 (slide-bootstrap mean; observed score 0.7526), compared with 0.6986 for Gemma4-12B. The historical encoder–probe comparison above retains 0.7532. Gemma4-31B has higher point estimates on the two cell-type-conditioned gene-list tasks. On nine-class tissue prediction, VGL-stGen's raw macro-F1 0.1200 is below several comparators; pooled correction yields 0.2801 under the separate, transductive condition described in §8.
The task map reproduces the paper's Figure 9. Filled cells denote externally sourced evaluation data and outlined cells denote internal evaluation. Data origin does not imply independence from all training or development stages. Task 2 has no direct training objective. Expression completion supplies 75 observed levels and queries 25 named genes. See MODEL_CARD.md or the paper for the full task definitions and evaluation conditions.
11. Rules worth following
- Gene order is signal. Descending expression rank, never sorted. Only the first 20 are read.
- Match the task's inference configuration. The SDK default batch size is 16. The paper's multimodal benchmark uses 16 for tasks 1–3, 32 for tasks 4–5, 8 for tasks 6–10 and 13, and one autoregressive completion row at a time for tasks 11–12. Also retain the task's generation budget, attention and precision settings.
- Match tissue scale as well as pixel dimensions. Spatial-transcriptomic and pathologist-derived training patches use a nominal 112-µm field of view at 224 × 224 pixels. LubLung training crops and TCGA classification use approximately 87 µm; TCGA application examples use 128 µm. Resizing does not equalise the amount of tissue shown. The paper does not establish invariance to these scale changes.
- Retain canonical prompts for score comparisons. A custom prompt changes the
evaluation condition;
prompt=drops the recorded certification flag. - Pass cell types in label-space spelling (
t_cell,cdc); the SDK converts to the form the prompt needs.
12. Troubleshooting
weights/merged_bf16/*.safetensors is a few hundred bytes.
Git LFS was not installed when you cloned. git lfs install && git lfs pull.
401, or "is not a local folder and is not a valid model identifier", on the
adapter path.
The base model is gated. Accept the terms on Hugging Face and export a token, or set
HF_HUB_OFFLINE=1 if you already have it cached, or use the merged path, which
does not download the base model separately. Access to this repository is still
required to obtain its checkpoints.
CUDA out of memory. The merged bf16 weights need roughly 10 GB. Free the device, or use a larger one.
AskUnavailableError from ask().
You loaded the merged weights. ask() needs an adapter it can switch off — load
with weights="adapter". The nine measured routes work on either path.
MissingInputError when I only have a partial gene list.
Expected: no route completes a gene list without an image or a cell type to anchor
it. Use complete_genes().
A gene route returned fewer genes than I expected.
The requested count is 20 - len(partial_genes). Passing 18 known genes asks for 2;
no partial list requests 20. Generation and parsing may return fewer unique symbols,
so inspect the result rather than assuming the requested length was reached.
complete_genes() returned something odd.
Check via_celltype first. It is a two-step call, and if the cell type it inferred
is wrong, the genes were completed for the wrong cell.
Individual labels differ from a previous run, but the score does not. Some recorded re-scoring checks preserve a similar aggregate score while changing individual labels. Match the weight packaging, prompt, processor, attention and batch settings. A restricted softmax score near 0.5 is not a calibrated clinical uncertainty estimate.
Layout
vgl_gen/
model.py load(), predict(), predict_batch(),
complete_genes(), ask()
router.py (inputs, want) -> route
prompts.py canonical prompt templates
decoding.py constrained label decoding
calibration.py prior correction and its route-specific rules
manifest.py the certification document loader
types.py Prediction plus seven result types
loaders.py weights, label spaces, seeding
errors.py the nine public exceptions
capabilities.internal.json the manifest itself
tests/ 296 tests; GPU ones are marked
weights/
merged_bf16/ default, 8.1 GB
adapter_post6/ LoRA only, 751 MB
MODEL_CARD.md — what it does, how well, and where it fails.
TECHNICAL.md — architecture and internals, for changing the package.
LICENSE, NOTICE — non-commercial research use only. The weights are a
derivative of google/medgemma-1.5-4b-it and carry the HAI-DEF terms; Portrai's
contribution is licensed under CC BY-NC 4.0.
Citation
@misc{shin2026vglstgen,
title = {{VGL-stGen}: Learning a shared multimodal model for lung histology and spatial transcriptomics},
author = {Haenara Shin and Dongjoo Lee and Sungwoo Bae and Kwon Joong Na and Daeseung Lee and Hongyoon Choi},
year = {2026},
eprint = {2610.28266},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.28266}
}
Portrai, Inc., Seoul, Republic of Korea. Correspondence: Hongyoon Choi
(chy@portrai.io). Checkpoint version:
vgl-gen-post6-2026-07-24. Documentation aligned with arXiv v1 on 22 October 2026.
Model tree for Portraier/VGL-stGen-0.5-Lung
Base model
google/medgemma-1.5-4b-it