Instructions to use thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21
- SGLang
How to use thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21 with Docker Model Runner:
docker model run hf.co/thrunlab/Mistral_Sparse_refined_web_50p_2024-03-21
Commit History
-
End of training
Verified signatureb63a9b2 -
Training in progress, step 501
Verified signature0e410fb -
Training in progress, step 500
Verified signature24dbdf9 -
Training in progress, step 499
Verified signatured0768af -
Training in progress, step 497
Verified signature127af2f -
Training in progress, step 496
Verified signature4474e89 -
Training in progress, step 494
Verified signaturefa40c68 -
Training in progress, step 492
Verified signature3d4bc42 -
Training in progress, step 490
Verified signatured0082c9 -
Training in progress, step 488
Verified signaturecc164c5 -
Training in progress, step 486
Verified signaturec28de52 -
Training in progress, step 484
Verified signature89bb1bd -
Training in progress, step 482
Verified signature039cb44 -
Training in progress, step 480
Verified signature42d61b3 -
Training in progress, step 478
Verified signaturec46ee49 -
Training in progress, step 476
Verified signature13768be -
Training in progress, step 474
Verified signatured315248 -
Training in progress, step 472
Verified signature01b2072 -
Training in progress, step 470
Verified signature572c10a -
Training in progress, step 468
Verified signature4dba8e6 -
Training in progress, step 466
Verified signaturede32e92 -
Training in progress, step 464
Verified signatureb4d6ffc -
Training in progress, step 462
Verified signature2282199 -
Training in progress, step 460
Verified signature5210d7c -
Training in progress, step 458
Verified signature46b76e2 -
Training in progress, step 456
Verified signaturec6c5f64 -
Training in progress, step 454
Verified signatureb3f6240 -
Training in progress, step 452
Verified signature59bd6cc -
Training in progress, step 450
Verified signaturededd8f9 -
Training in progress, step 448
Verified signature5a965f2 -
Training in progress, step 446
Verified signature3901538 -
Training in progress, step 444
Verified signaturebdda1ae -
Training in progress, step 442
Verified signature192877a -
Training in progress, step 441
Verified signature48a5250 -
Training in progress, step 439
Verified signatureb8763b3 -
Training in progress, step 437
Verified signature8056267 -
Training in progress, step 435
Verified signaturead17f0e -
Training in progress, step 433
Verified signaturea3ee138 -
Training in progress, step 432
Verified signature982e8e8 -
Training in progress, step 430
Verified signaturee28d908 -
Training in progress, step 428
Verified signatureacd26ea -
Training in progress, step 426
Verified signaturec78c40e -
Training in progress, step 424
Verified signaturec4e7ae1 -
Training in progress, step 422
Verified signatured22e719 -
Training in progress, step 420
Verified signature830c848 -
Training in progress, step 418
Verified signature5b52a59 -
Training in progress, step 416
Verified signature9c7b39a -
Training in progress, step 414
Verified signaturec49efd0 -
Training in progress, step 413
Verified signature31b7883 -
Training in progress, step 411
Verified signaturec55580b