Text Generation
Transformers
TensorBoard
Safetensors
llama
Generated from Trainer
text-generation-inference
Instructions to use Harryxun/llama-pretrained with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Harryxun/llama-pretrained with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Harryxun/llama-pretrained")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Harryxun/llama-pretrained") model = AutoModelForCausalLM.from_pretrained("Harryxun/llama-pretrained", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Harryxun/llama-pretrained with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Harryxun/llama-pretrained" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Harryxun/llama-pretrained", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Harryxun/llama-pretrained
- SGLang
How to use Harryxun/llama-pretrained with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Harryxun/llama-pretrained" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Harryxun/llama-pretrained", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Harryxun/llama-pretrained" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Harryxun/llama-pretrained", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Harryxun/llama-pretrained with Docker Model Runner:
docker model run hf.co/Harryxun/llama-pretrained
Commit History
-
Upload LlamaForCausalLM
Verified signatured812c77 -
Training in progress, step 5000
Verified signature00982c6
-
End of training
Verified signature9af4376 -
Training in progress, step 24
Verified signature651b6d8 -
Training in progress, step 12
Verified signaturef104dc6
-
End of training
Verified signature7d0da80 -
Training in progress, step 83824
Verified signature04ba2bb -
Training in progress, step 80000
Verified signature06a8dc7 -
Training in progress, step 75000
Verified signature45bc03f -
Training in progress, step 70000
Verified signature372abe1 -
Training in progress, step 65000
Verified signature2e9ad54 -
Training in progress, step 60000
Verified signature2598cae -
Training in progress, step 55000
Verified signaturebaa2b82 -
Training in progress, step 50000
Verified signature6152895 -
Training in progress, step 45000
Verified signature3d270a2 -
Training in progress, step 40000
Verified signature5bae14f -
Training in progress, step 35000
Verified signatureaeec75d -
Training in progress, step 30000
Verified signature64f27c7
-
Training in progress, step 25000
Verified signatureb6ae804 -
Training in progress, step 20000
Verified signature86f2237 -
Training in progress, step 15000
Verified signature4d486e9 -
Training in progress, step 10000
Verified signature699cd27 -
Training in progress, step 5000
Verified signature40feb99 -
Training in progress, step 20000
Verified signature38f0a9a -
Training in progress, step 15000
Verified signature8d6bae4 -
Training in progress, step 10000
Verified signature72a952c
-
Training in progress, step 5000
Verified signature43b5259
-
End of training
Verified signature76f2642 -
Training in progress, step 37413
Verified signaturefedd8bb
-
Training in progress, step 35000
Verified signature17ce9e6 -
Training in progress, step 30000
Verified signaturef6531ef -
Training in progress, step 25000
Verified signatureeeba746 -
Training in progress, step 20000
Verified signaturec69bba2 -
Training in progress, step 15000
Verified signaturee30f610 -
Training in progress, step 10000
Verified signaturef805301 -
Training in progress, step 5000
Verified signaturee665a46 -
initial commit
Verified signatureb175513