Instructions to use meihant/AnonymousModel-mht with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use meihant/AnonymousModel-mht with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="meihant/AnonymousModel-mht")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("meihant/AnonymousModel-mht", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use meihant/AnonymousModel-mht with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "meihant/AnonymousModel-mht" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meihant/AnonymousModel-mht", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/meihant/AnonymousModel-mht
- SGLang
How to use meihant/AnonymousModel-mht with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "meihant/AnonymousModel-mht" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meihant/AnonymousModel-mht", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "meihant/AnonymousModel-mht" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meihant/AnonymousModel-mht", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use meihant/AnonymousModel-mht with Docker Model Runner:
docker model run hf.co/meihant/AnonymousModel-mht
Add datasets/FCP-20B artifacts 560-567
Browse files- datasets/FCP-20B/source2b/counts/059.bin +3 -0
- datasets/FCP-20B/source2b/counts/060.bin +3 -0
- datasets/FCP-20B/source2b/counts/061.bin +3 -0
- datasets/FCP-20B/source2b/counts/062.bin +3 -0
- datasets/FCP-20B/source2b/counts/063.bin +3 -0
- datasets/FCP-20B/source2b/counts/064.bin +3 -0
- datasets/FCP-20B/source2b/counts/065.bin +3 -0
- datasets/FCP-20B/source2b/counts/066.bin +3 -0
datasets/FCP-20B/source2b/counts/059.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3c6c0c273f6bb9685b6f519076d273617780f10c7761602fb972ac12b65a3a8f
|
| 3 |
+
size 4184104
|
datasets/FCP-20B/source2b/counts/060.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a6938c8aabc11cbd044787ad5611fa8dbde3c8e5f209ecad5e18f9865da386b
|
| 3 |
+
size 4170968
|
datasets/FCP-20B/source2b/counts/061.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:26bd56505b6bc5719635adadaee18c158315091b749ef8ebadf85cb60c8f597a
|
| 3 |
+
size 4181248
|
datasets/FCP-20B/source2b/counts/062.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cb6e835c6e67b9a6b56e58e56679a2c577229dbc97a35f1bd62d1c93351a5d93
|
| 3 |
+
size 4158560
|
datasets/FCP-20B/source2b/counts/063.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:696b807757e0034c866289b99adc5114304ef9b87e796f4584f0a052c703e26f
|
| 3 |
+
size 4180256
|
datasets/FCP-20B/source2b/counts/064.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f9faf53034c65a14ccff8df6dd819430f5434bd0de4d08891917b88bd71c96df
|
| 3 |
+
size 4181088
|
datasets/FCP-20B/source2b/counts/065.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bbded202c2f94df6cbbd5ca7d876bf02b7eaa1dff5e639dadd1620def8a3b0b6
|
| 3 |
+
size 4165120
|
datasets/FCP-20B/source2b/counts/066.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ab5204ec3ed5e3171cec769258959ffdc15b618641876602985174d9fba30ae
|
| 3 |
+
size 4154384
|