Instructions to use nlparabic/test-aragpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nlparabic/test-aragpt with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="nlparabic/test-aragpt")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("nlparabic/test-aragpt") model = AutoModelForCausalLM.from_pretrained("nlparabic/test-aragpt", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nlparabic/test-aragpt with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nlparabic/test-aragpt" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nlparabic/test-aragpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/nlparabic/test-aragpt
- SGLang
How to use nlparabic/test-aragpt with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nlparabic/test-aragpt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nlparabic/test-aragpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nlparabic/test-aragpt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nlparabic/test-aragpt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use nlparabic/test-aragpt with Docker Model Runner:
docker model run hf.co/nlparabic/test-aragpt
Download trainer_state.json from nlparabic/test-aragpt: direct link, hf CLI and curl.
- Browser
- Download file 2.55 kB
-
https://huggingface.co/nlparabic/test-aragpt/resolve/main/trainer_state.json
- Command line
-
hf download hf://nlparabic/test-aragpt/trainer_state.json
-
curl -L -o trainer_state.json https://huggingface.co/nlparabic/test-aragpt/resolve/main/trainer_state.json
2.55 kB
| { | |
| "best_metric": null, | |
| "best_model_checkpoint": null, | |
| "epoch": 15.0, | |
| "eval_steps": 500, | |
| "global_step": 4785, | |
| "is_hyper_param_search": false, | |
| "is_local_process_zero": true, | |
| "is_world_process_zero": true, | |
| "log_history": [ | |
| { | |
| "epoch": 1.567398119122257, | |
| "grad_norm": 1.1618669033050537, | |
| "learning_rate": 4.4775339602925815e-05, | |
| "loss": 3.4578, | |
| "step": 500 | |
| }, | |
| { | |
| "epoch": 3.134796238244514, | |
| "grad_norm": 1.134594440460205, | |
| "learning_rate": 3.955067920585162e-05, | |
| "loss": 2.7316, | |
| "step": 1000 | |
| }, | |
| { | |
| "epoch": 4.702194357366771, | |
| "grad_norm": 1.16652250289917, | |
| "learning_rate": 3.4326018808777435e-05, | |
| "loss": 2.4393, | |
| "step": 1500 | |
| }, | |
| { | |
| "epoch": 6.269592476489028, | |
| "grad_norm": 1.1664645671844482, | |
| "learning_rate": 2.9101358411703238e-05, | |
| "loss": 2.2559, | |
| "step": 2000 | |
| }, | |
| { | |
| "epoch": 7.836990595611285, | |
| "grad_norm": 1.1747838258743286, | |
| "learning_rate": 2.387669801462905e-05, | |
| "loss": 2.127, | |
| "step": 2500 | |
| }, | |
| { | |
| "epoch": 9.404388714733543, | |
| "grad_norm": 1.2233911752700806, | |
| "learning_rate": 1.865203761755486e-05, | |
| "loss": 2.0242, | |
| "step": 3000 | |
| }, | |
| { | |
| "epoch": 10.971786833855798, | |
| "grad_norm": 1.149554967880249, | |
| "learning_rate": 1.3427377220480669e-05, | |
| "loss": 1.9585, | |
| "step": 3500 | |
| }, | |
| { | |
| "epoch": 12.539184952978056, | |
| "grad_norm": 1.1873332262039185, | |
| "learning_rate": 8.202716823406478e-06, | |
| "loss": 1.9004, | |
| "step": 4000 | |
| }, | |
| { | |
| "epoch": 14.106583072100314, | |
| "grad_norm": 1.1350129842758179, | |
| "learning_rate": 2.9780564263322885e-06, | |
| "loss": 1.8692, | |
| "step": 4500 | |
| }, | |
| { | |
| "epoch": 15.0, | |
| "step": 4785, | |
| "total_flos": 1.500338847744e+16, | |
| "train_loss": 2.280068150184496, | |
| "train_runtime": 1782.6608, | |
| "train_samples_per_second": 21.474, | |
| "train_steps_per_second": 2.684 | |
| } | |
| ], | |
| "logging_steps": 500, | |
| "max_steps": 4785, | |
| "num_input_tokens_seen": 0, | |
| "num_train_epochs": 15, | |
| "save_steps": 500, | |
| "stateful_callbacks": { | |
| "TrainerControl": { | |
| "args": { | |
| "should_epoch_stop": false, | |
| "should_evaluate": false, | |
| "should_log": false, | |
| "should_save": true, | |
| "should_training_stop": true | |
| }, | |
| "attributes": {} | |
| } | |
| }, | |
| "total_flos": 1.500338847744e+16, | |
| "train_batch_size": 8, | |
| "trial_name": null, | |
| "trial_params": null | |
| } | |