Instructions to use allenai/Bolmo-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use allenai/Bolmo-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="allenai/Bolmo-1B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("allenai/Bolmo-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use allenai/Bolmo-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "allenai/Bolmo-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "allenai/Bolmo-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/allenai/Bolmo-1B
- SGLang
How to use allenai/Bolmo-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "allenai/Bolmo-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "allenai/Bolmo-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "allenai/Bolmo-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "allenai/Bolmo-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use allenai/Bolmo-1B with Docker Model Runner:
docker model run hf.co/allenai/Bolmo-1B
Commit History
-
Re-export with updated config and tokenizer (#1)
- Re-export with updated config and tokenizer (010c7560be7eb808e6778264bd1a2839dc83a97a) - Update README.md (474e6ac178ec15f73d1dbe0ba1e20a194993926f) - Update modeling_bolmo.py (49b278a2867a8b1b6e72c59858d06ef48eff14eb) Co-authored-by: Benjamin Minixhofer <benjamin@users.noreply.huggingface.co>452690a
-
CPU support, stop criteria fix for bs>1
Verified signature7a80d2e
-
Update README.md
Verified signaturedd64475 -
Update README.md
Verified signature9a1e15f
-
Update config.json
Verified signature5f3bab9 -
Update README.md
Verified signature566caac
-
Upload folder using huggingface_hub
Verified signaturebf8dcf8 -
Update README.md
Verified signatured691ab2 -
Update README.md
Verified signature70e9a0b -
Upload folder using huggingface_hub
Verified signaturec1be487 -
Update README.md
Verified signature4855ce7 -
Upload folder using huggingface_hub
Verified signature247323b
-
Update README.md
Verified signatureb5b00b8 -
Update README.md
Verified signature119a22e -
Upload BolmoForCausalLM
Verified signaturef893117 -
Upload BolmoForCausalLM
Verified signature8e42dcc -
initial commit
Verified signatureb8af341