Text Generation
Transformers
Safetensors
English
llama
causal-lm
pretraining
from-scratch
single-gpu
nvidia-l20
text-generation-inference
Instructions to use Epode/Prelude-1B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Epode/Prelude-1B-Base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Epode/Prelude-1B-Base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Epode/Prelude-1B-Base") model = AutoModelForCausalLM.from_pretrained("Epode/Prelude-1B-Base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Epode/Prelude-1B-Base with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Epode/Prelude-1B-Base" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Epode/Prelude-1B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Epode/Prelude-1B-Base
- SGLang
How to use Epode/Prelude-1B-Base with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Epode/Prelude-1B-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Epode/Prelude-1B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Epode/Prelude-1B-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Epode/Prelude-1B-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Epode/Prelude-1B-Base with Docker Model Runner:
docker model run hf.co/Epode/Prelude-1B-Base
Download model_config.yaml from Epode/Prelude-1B-Base: direct link, hf CLI and curl.
- Browser
- Download file 1.16 kB
-
https://huggingface.co/Epode/Prelude-1B-Base/resolve/main/model_config.yaml
- Command line
-
hf download hf://Epode/Prelude-1B-Base/model_config.yaml
-
curl -L -o model_config.yaml https://huggingface.co/Epode/Prelude-1B-Base/resolve/main/model_config.yaml
1.16 kB
| attention_logit_softcapping: null | |
| attention_scores_scalar: null | |
| attn_bias: false | |
| bias: false | |
| block_size: 2048 | |
| final_logit_softcapping: null | |
| first_k_dense_replace: null | |
| gelu_approximate: none | |
| head_size: 64 | |
| hf_config: | |
| name: TinyLlama-1.1B-intermediate-step-1431k-3T | |
| org: TinyLlama | |
| intermediate_size: 5632 | |
| latent_attention: null | |
| lm_head_bias: false | |
| mlp_class_name: LLaMAMLP | |
| moe_intermediate_size: null | |
| n_embd: 2048 | |
| n_expert: 0 | |
| n_expert_groups: null | |
| n_expert_per_token: 0 | |
| n_head: 32 | |
| n_layer: 22 | |
| n_query_groups: 4 | |
| n_shared_expert: null | |
| n_topk_groups: null | |
| n_topk_scores_per_group: null | |
| name: tiny-llama-1.1b | |
| norm_1: true | |
| norm_2: true | |
| norm_class_name: RMSNorm | |
| norm_eps: 1.0e-05 | |
| norm_qk: false | |
| norm_qk_type: default | |
| norm_topk_prob: false | |
| padded_vocab_size: 32000 | |
| padding_multiple: 64 | |
| parallel_residual: false | |
| post_attention_norm: false | |
| post_mlp_norm: false | |
| rope_adjustments: null | |
| rope_base: 10000 | |
| rope_condense_ratio: 1 | |
| rope_indices: null | |
| rope_interleave: false | |
| rope_local_base_freq: null | |
| rotary_percentage: 1.0 | |
| routed_scaling_factor: 1.0 | |
| scale_embeddings: false | |
| shared_attention_norm: false | |
| sliding_window_indices: null | |
| sliding_window_size: null | |
| vocab_size: 32000 | |