Text Generation
Transformers
Safetensors
English
llama
tiny
toy-model
from-scratch
text-generation-inference
Instructions to use aaaaaaaieuyd/Small-LLM1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use aaaaaaaieuyd/Small-LLM1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="aaaaaaaieuyd/Small-LLM1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("aaaaaaaieuyd/Small-LLM1") model = AutoModelForCausalLM.from_pretrained("aaaaaaaieuyd/Small-LLM1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use aaaaaaaieuyd/Small-LLM1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "aaaaaaaieuyd/Small-LLM1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "aaaaaaaieuyd/Small-LLM1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/aaaaaaaieuyd/Small-LLM1
- SGLang
How to use aaaaaaaieuyd/Small-LLM1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "aaaaaaaieuyd/Small-LLM1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "aaaaaaaieuyd/Small-LLM1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "aaaaaaaieuyd/Small-LLM1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "aaaaaaaieuyd/Small-LLM1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use aaaaaaaieuyd/Small-LLM1 with Docker Model Runner:
docker model run hf.co/aaaaaaaieuyd/Small-LLM1
|
Download README.md from aaaaaaaieuyd/Small-LLM1: direct link, hf CLI and curl.
- Browser
- Download file 2.51 kB
-
https://huggingface.co/aaaaaaaieuyd/Small-LLM1/resolve/main/README.md
- Command line
-
hf download hf://aaaaaaaieuyd/Small-LLM1/README.md
-
curl -L -o README.md https://huggingface.co/aaaaaaaieuyd/Small-LLM1/resolve/main/README.md
2.51 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| datasets: | |
| - roneneldan/TinyStories | |
| tags: | |
| - tiny | |
| - toy-model | |
| - from-scratch | |
| - llama | |
| # Small LLM1 | |
| **์ฌ๋ฏธ๋ก ๋ง๋ ์ฝ 5M(4.98M) ํ๋ผ๋ฏธํฐ์ง๋ฆฌ ์ด์ํ ์ธ์ด ๋ชจ๋ธ์ ๋๋ค.** | |
| > โ ๏ธ **์ฌ๋ฏธ๋ก ๋ง๋ ๋ชจ๋ธ์ด๋ผ ์ฑ๋ฅ์ ๋ฎ์ต๋๋ค.** ์ค์ ์ฉ๋๋ก ์ฐ๊ธฐ์๋ ๋ถ์กฑํ๊ณ , ์งง์ ์์ด ๋ํ ๋ฌธ์ฅ์ ์ด์ด ์ฐ๋ ์ ๋๋ง ํ ์ ์์ต๋๋ค. | |
| > | |
| > โ ๏ธ **ํ์ธํ๋(instruction tuning / chat tuning)์ ํ์ง ์์์ต๋๋ค.** ์ฌ์ ํ์ต๋ง ํ base ๋ชจ๋ธ์ด๋ผ ์ง๋ฌธ์ ๋ตํ๊ฑฐ๋ ๋ํํ์ง ๋ชปํ๊ณ , ์ฃผ์ด์ง ๋ฌธ์ฅ์ ์ด์ด์ ์ฐ๊ธฐ๋ง ํฉ๋๋ค. | |
| ## ๋ชจ๋ธ ์ ๋ณด | |
| | ํญ๋ชฉ | ๊ฐ | | |
| |---|---| | |
| | ๊ตฌ์กฐ | Llama ์คํ์ผ decoder-only Transformer (์ฒ์๋ถํฐ ํ์ต) | | |
| | ํ๋ผ๋ฏธํฐ | 4.98M | | |
| | ๋ ์ด์ด / hidden / heads | 6 / 256 / 8 | | |
| | FFN ํฌ๊ธฐ | 512 | | |
| | ์ดํ ํฌ๊ธฐ | 4,096 (์ง์ ํ์ตํ byte-level BPE) | | |
| | ์ปจํ ์คํธ ๊ธธ์ด | 256 ํ ํฐ | | |
| | ํ์ต ๋ฐ์ดํฐ | [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) ์๋ถ๋ถ ์ฝ 40๋ง ๊ฐ ์ด์ผ๊ธฐ (์ฝ 92M ํ ํฐ) | | |
| | ํ์ต | 6,000 step ร 64 ร 256 ํ ํฐ, RTX 5070 1์ฅ์์ ์ฝ 4๋ถ | | |
| | ๊ฒ์ฆ loss | 1.77 (perplexity ์ฝ 5.9, TinyStories ๊ฒ์ฆ ๊ตฌ๊ฐ) | | |
| ## ์ฌ์ฉ๋ฒ | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model = AutoModelForCausalLM.from_pretrained("aaaaaaaieuyd/Small-LLM1") | |
| tok = AutoTokenizer.from_pretrained("aaaaaaaieuyd/Small-LLM1") | |
| x = tok("Once upon a time", return_tensors="pt") | |
| out = model.generate(**x, max_new_tokens=80, do_sample=True, temperature=0.8, top_k=40) | |
| print(tok.decode(out[0], skip_special_tokens=True)) | |
| ``` | |
| ์ถ๋ ฅ ์์: | |
| > Once upon a time, there was a little girl named Lily. She had a little tooth that was very important to her. The tooth was her special tooth, and she could always count on it to show it to her friends. | |
| ## ํ๊ณ | |
| - ์์ด ๋ํ์ฒด ๋ฌธ์ฅ๋ง ํ์ตํด์ ํ๊ตญ์ด๋ ๋ค๋ฅธ ์ฃผ์ ๋ ๊ฑฐ์ ๋ชป ๋ค๋ฃน๋๋ค. | |
| - ์ฌ์ค ์ ๋ณด, ์ถ๋ก , ์ฝ๋ ๊ฐ์ ๊ฑด ๊ธฐ๋ํ๋ฉด ์ ๋ฉ๋๋ค. | |
| - ์๋ค๊ฐ ์ ๋ง๋ ์ด์ผ๊ธฐ๋ฅผ ์์ฃผ ๋ง๋ญ๋๋ค. | |
| - ํ์ธํ๋์ ํ์ง ์์์ ์ง์๋ฅผ ๋ฐ๋ฅด์ง ๋ชปํฉ๋๋ค. | |
| --- | |
| **English:** Small LLM1 is a ~5M-parameter toy language model trained from scratch on TinyStories, just for fun. Performance is low, and it is a base model with **no fine-tuning** (no instruction or chat tuning). It can only continue short, simple English stories. | |