Feature Extraction
Transformers
Safetensors
hetero_moe_projspace
mixture-of-experts
heterogeneous
dimension-projection-space
ffn-fusion
custom_code
Instructions to use redmingwei/HeteroMoEProjSpace-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use redmingwei/HeteroMoEProjSpace-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="redmingwei/HeteroMoEProjSpace-v2", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("redmingwei/HeteroMoEProjSpace-v2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download config.json from redmingwei/HeteroMoEProjSpace-v2: direct link, hf CLI and curl.
- Browser
- Download file 592 Bytes
-
https://huggingface.co/redmingwei/HeteroMoEProjSpace-v2/resolve/main/config.json
- Command line
-
hf download hf://redmingwei/HeteroMoEProjSpace-v2/config.json
-
curl -L -o config.json https://huggingface.co/redmingwei/HeteroMoEProjSpace-v2/resolve/main/config.json
592 Bytes
| { | |
| "architectures": [ | |
| "HeteroMoEProjSpaceModel" | |
| ], | |
| "auto_map": { | |
| "AutoConfig": "modeling_hetero_moe_projspace.HeteroMoEProjSpaceConfig", | |
| "AutoModel": "modeling_hetero_moe_projspace.HeteroMoEProjSpaceModel" | |
| }, | |
| "bert_id": "google/bert_uncased_L-4_H-256_A-4", | |
| "d_shared": 256, | |
| "d_space": 1024, | |
| "dtype": "float32", | |
| "modal_dims": [ | |
| 896, | |
| 256, | |
| 768 | |
| ], | |
| "model_type": "hetero_moe_projspace", | |
| "moe_top_k": 2, | |
| "n_experts": 4, | |
| "qwen_id": "Qwen/Qwen2.5-0.5B-Instruct", | |
| "transformers_version": "5.21.0", | |
| "vit_id": "google/vit-base-patch16-224-in21k" | |
| } | |