Instructions to use not-lain/PyGPT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use not-lain/PyGPT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="not-lain/PyGPT")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("not-lain/PyGPT") model = AutoModelForCausalLM.from_pretrained("not-lain/PyGPT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use not-lain/PyGPT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "not-lain/PyGPT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "not-lain/PyGPT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/not-lain/PyGPT
- SGLang
How to use not-lain/PyGPT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "not-lain/PyGPT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "not-lain/PyGPT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "not-lain/PyGPT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "not-lain/PyGPT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use not-lain/PyGPT with Docker Model Runner:
docker model run hf.co/not-lain/PyGPT
| datasets: | |
| - iamtarun/python_code_instructions_18k_alpaca | |
| widget: | |
| - text: > | |
| Below is an instruction that describes a task. Write a response that | |
| appropriately completes the request. | |
| ### Instruction: | |
| Create a function to calculate the sum of a sequence of integers. | |
| ### Input: | |
| [1, 2, 3, 4, 5] | |
| ### Output: | |
| pipeline_tag: text-generation | |
| tags: | |
| - code | |
| ## Model Details | |
| this is the finetuned version of GPT2 on a coding dataset | |
| ### Model Description | |
| - **Model type:** text-generation | |
| - **Finetuned from model [GPT2](https://huggingface.co/gpt2)** | |
| ### Model Sources | |
| - **Repository:** https://huggingface.co/gpt2 | |
| ## Uses | |
| ```python | |
| # Use a pipeline as a high-level helper | |
| from transformers import pipeline | |
| pipe = pipeline("text-generation", model="not-lain/PyGPT") | |
| prompt = """ | |
| Below is an instruction that describes a task. Write a response that | |
| appropriately completes the request. | |
| ### Instruction: | |
| Create a function to calculate the sum of a sequence of integers. | |
| ### Input: | |
| [1, 2, 3, 4, 5] | |
| ### Output: | |
| """ | |
| pipe(prompt) | |
| ``` | |
| ## Bias, Risks, and Limitations | |
| model may produce biased ,erroneous and output. | |
| ### Recommendations | |
| it is not advised to use this model as it is just a product of testing a finetuning script | |
| ## Training Details | |
| ### Training Data | |
| <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> | |
| [More Information Needed] | |
| ## Evaluation | |
| please refer to the tensorboard tab for full details |