Instructions to use ctheodoris/Geneformer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ctheodoris/Geneformer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="ctheodoris/Geneformer")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("ctheodoris/Geneformer") model = AutoModelForMaskedLM.from_pretrained("ctheodoris/Geneformer", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
Commit History
-
fix: incorrect condition control flow
this addresses the issue when using GeneformerPrecollator if not using numpy array as tensor backend, a ValueError is raise like `type of tensor([ 2943, 24469, 12039, ..., 4506, 13856, 10511]) unknown: <class 'torch.Tensor'>. Should be one of a python, numpy, pytorch or tensorflow object.`. I believe this is a bug, as I am clearly using a pytorch tensor. And in huggingface `transformers` library, [similar code wrote here](https://github.com/huggingface/transformers/blob/21dc5859421cf0d7d82d374b10f533611745a8c5/src/transformers/feature_extraction_sequence_utils.py#L166)4f17a92
-
Update model card to emphasize need to install git-lfs and suggest https method
dbfac41
-
Add option to output embs as tensor
Christina Theodoris624349c
-
anndata_tokenizer (#170)
- Added anndata tokenizer and switched to Dataset.from_generator (b6ca56647a16de97f6f95785eb76cb1a965f9960) - Generalized (5cb733f4fed85e1568ba9671e5b1f9babd7b8491) - Addressed issues for tokenizer, anndata tokenizer now uses a fraction of memory (b24676d0956565ff7b4f8b57d1e936ad17f8740e) - Fixed issues (94e8d231c41a89232a55c064e5c8ea95edd51d2e) - Fixed issue, loom and h5ad now produce same checksums (9c62f4c6e3ab0afd1943079306655bd20eba2246) - Rename loom tokenizer function and modify example notebook for adata. (4fdb85061180ccf022ab761c2557be6ba67d848e) Co-authored-by: Rico Meinl <ricomnl@users.noreply.huggingface.co>4302f48
-
Fixed bugs related to overexpressing genes (#229)
- Fixed bugs related to overexpressing genes (72c65013e2fab0e3d97757f3a0af55b09a4864c7) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>39ab62e
-
Update installation instructions to include git lfs install and ssh method
4ce81ab
-
Config head node runtime via IP (#234)
- Config head node runtime via IP (748f48ac03faa90488220eb2f9b5a76e128a6c06) - Update minor formatting (c48e37c67945c09e6593d45c40c4cbe0df7cc757) Co-authored-by: Madhavan Venkatesh <madhavanvenkatesh@users.noreply.huggingface.co>b8fda63
-
Add memory-efficient method for computing emb summary statistics
Christina Theodoris6caf480
-
Adding `safetensors` variant of this model (#209)
- Adding `safetensors` variant of this model (bd3f711401ac83e1a40b9e1bd72fe294e191073e) Co-authored-by: Safetensors convertbot <SFconvertbot@users.noreply.huggingface.co>61f15d2
-
Fixed bug with the double removing of indices when cell_states_to_model is false (#188)
- Fixed bug with the double removing of indices when cell_states_to_model is false (add6c3b33a0d94c79837cc92e1a635e9c18934e1) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>0adfe67 -
Upload in_silico_perturber.py (#187)
- Upload in_silico_perturber.py (5615a16ff840b805c274ee0781c6a9ac9a90b511) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>8180caa
-
Added feature to perturb a set of indices to help with debugging and with very large runtimes (#175)
- Added feature to perturb a set of indices to help with debugging and with very large runtimes (5488d176961bcee6c66ea1494151429fb570ba9c) - Update geneformer/in_silico_perturber.py (9151b3e978c1c138039da1090be2147b4631d903) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>f115e8f
-
Re-update stats to handle case of empty alt_states
Christina Theodoris78517d8 -
Add handling for case of alt_states being empty list
Christina Theodoris9e8dbe5
-
Remove print statement from PR
c4b1f94 -
Fixed bug in gen_attention_mask with len > max_len (#158)
- Fixed bug in gen_attention_mask with len > max_len (7c77bae654e0d93a27e1988e107b3258902b3d05) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>3a94209 -
Fixed error with perturbing individual genes and updated ways to specify cell_states_to_model (#146)
- Fixed error with perturbing individual genes and updated ways to specify cell_states_to_model (771c8bd01b1754c4387d742680e65c34697a2336) - Fix isp perturb_group dims, reformat cell states dict to keyed, add attn mask (c2679c41d352513a56feb5490c5302c6f25ae7ba) Co-authored-by: David Wen <davidjwen@users.noreply.huggingface.co>9169bfd
-
Add error message for "gene" embedding extraction under development.
Christina Theodoris65b4915 -
Update README.md
Update model card to include extracting and plotting cell embeddings as available functione820326 -
Rename heatmap legend to be correct label
Christina Theodorisbadcca6 -
Add function to extract and plot cell embeddings
Christina Theodorisd154fee -
Add fine-tuned CellClassifier for cardiomyopathy
Christina Theodoris1786b44 -
Add error for no files found and suppress loompy import warning
Christina Theodorisabdf980
-
Add sorting for aggregating data for goal state shifts
Christina Theodoris50e921d -
Fix min_genes to be >= tokens to perturb as a group
Christina Theodoris268e566
-
Update tokenizer to allow tokenization without custom cell attributes
Christina Theodoris57b9778 -
Update isp to allow modeling single perturbation in multiple cells as batches
Christina Theodorisacd253c
-
Change list of individual IDs to set to ensure unique before subsetting into train/valid/test sets
Christina Theodoris45b9d69
-
Update internal format of anchor token to list for consistency with genes to perturb
Christina Theodorisb36d210
-
Add filtering for start state cells prior to in silico perturbation when modeling cell states
Christina Theodorisbb217cf
-
Correct order of state dict in in silico perturber stats and tensor dims of alt state emb in in silico perturber
Christina Theodoris3d06203 -
Add explanation of output columns and sort by largest shift
Christina Theodoris3072225
-
Add note to recommend tuning hyperparameters for downstream applications
Christina Theodoris98ce6d7
-
Add instructions to convert gene annotations w/ Ensembl Biomart
Christina Theodoris77eb432
-
Update manuscript link to SharedIt
4f6115b -
Update manuscript link in model card to ShareIt
dd016da
-
Modify tokenizer to allow renaming attr names btwn loom and .dataset
Christina Theodorise78c44d -
Modify quant_layers to convert layer nums to integers before calculating max
Christina Theodoris2181aa4
-
Remove unused evalset variable from cell classification example
Christina Theodoris879a878
-
Modify documentation for modeling only 2 cell states
Christina Theodoris019165f -
Add instructions for modeling only 2 states and modify stats script for that option
Christina Theodoris912860d
-
Update links including unsorted example lengths file
Christina Theodorisf0b6641
-
Add function to create remainder emb for in silico overexpression batch
Christina Theodorisfeeecd0
-
Add mixture model option for gene-gene interaction stats
Christina Theodorisdc1481d -
Add uniform max len for padding for predictions
Christina Theodoris67f674c
-
Add stats with mixture model to determine whether test perturbation is in impact component
Christina Theodorisd20ad0a -
Update model card for 12 layer Geneformer model
0637325 -
Update model card for 12 layer Geneformer
514c5fb -
Upload 12 layer Geneformer model
9d41e70
-
Fix bug in clearing cache
Christina Theodoris8c2fae7