Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
-
Add TensorBoard Traces (#25)
ybelkada- Add TensorBoard traces (24767be90ca0dd069d2559d5577c6e88fdb624c4)5556040 -
Correct number of languages (#24)
- Correct number of languages (69e78440cdbd55e8588883927a460cd8aa222c24) - Update README.md (0de6009ab9827a696a1d2fb1fcdedeb21088c6c0) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>34aa644 -
Update README.md (#23)
- Update README.md (537c688696c47203e5f9018b5b67109f9f1c2cb5) Co-authored-by: Victor M <victor@users.noreply.huggingface.co>eb49b9c -
modify padding side
c42411c -
add pad token id
Younes Belkada0142556
-
Add image for how to use (#20)
- Add image for how to use (e38c183a699782feb22b4252654791ee43c84489) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>46e5752
-
Remove .dev from transformers version (#21)
ybelkada- Remove .dev from transformers version (89303001337263eb28ee69dd51ec28fac73ac3ef) - add architecture (5efe5164c6fab318760c2a98485c73732fdfab78)85e429f -
Add correct dtype on config
Younes Belkadafbda88e
-
new data
acbc8df -
Updating italics to the HTML equivalent.
177dc5b -
Fixing italics that aren't rendering for "BigScience Large Open-science Open-access Multilingual Language Model"
833b211 -
Update README.md (#17)
- Update README.md (1d0ed019f32f5e27c155787f05b0fde415f613f4) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>83839ee -
Update README.md (#18)
ybelkada- Update README.md (2a44f086c0cbfd0609537820938b2172eb34cc32)44651d2 -
Update README.md (#15)
ybelkada- Update README.md (67f39504df45f38db346e853aea20873f434ca6a)0aa20cd -
Update README.md
Younes Belkadaf25baa0 -
Update README.md
Younes Belkada290df47 -
Update README.md (#16)
ybelkada- Update README.md (b3f36225ffaccd6bdeba3ab1d1bd0b9ef0368d90)45cf2c9 -
Adding blurb about mistaking sentience. (#12)
- Adding blurb about mistaking sentience. (64cee78cfab83e9f9a6a4ba68e90ab56eba707bf) - Adding sentience blurb. (09a4f1c6d307f8a080636b355b20da067d06e243) Co-authored-by: Margaret Mitchell <meg@users.noreply.huggingface.co>dc534f6 -
Update README.md (#14)
- Update README.md (52a7843b4d71889e146268f0d3927859f3e7dd5c) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>9d151d5
-
Add link to license article and FAQ (#13)
ybelkada +1: cakiki
- Add link to license article and FAQ (d05dd208e40b1f78168f8b6bc016c002af65fbca) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>ba402e7 -
Update README.md
Younes Belkadaa339d03
-
Updates the model card in light of more developed format. (#11)
- Updates the model card in light of more developed format. (e7e28b1ca071f697ad0cc98d30b32a0d3887768a)6cb7cc9
-
Update README.md
Younes BelkadaUpdate global step826ae56
-
Merge branch 'main' of https://huggingface.co/bigscience/bloom into main
c2f6b0a -
new data
18894c7 -
new data
2690f53
-
Update README.md
215bee9
-
Edit intermediary-checkpoint warning
Teven Le Scao55d00a5
-
Update README.md
Younes Belkadae8fa22e -
Update README.md
Younes Belkadacf39b50 -
Update README.md
Younes Belkada39b727a -
Update README.md
Younes Belkada945bfb3 -
Update README.md
Younes Belkada4320d65
-
Update README.md
Younes Belkada3351c7c -
Update README.md
Younes Belkada6dbe969 -
new data
5e650f0
-
new data
bea8142 -
new data
e0e6b58
-
Update config.json
Younes Belkada1d7d7cc
-
Add correct pipeline tag (#8)
ybelkada- Add correct pipeline tag (9d9c82605d1e3c66d79f1635336eb4f60e77b394)0bc3d2e -
Upload tokenizer.json
Younes Belkada9744a05 -
Upload special_tokens_map.json
Younes Belkada55dd66c -
Upload tokenizer_config.json
Younes Belkada8f193aa
-
Update config.json
Younes Belkadad6e3260
-
Different wording for "derivative" models (#5)
- Different wording for "derivative" models (ccfb08dafdd8a94a202e8c0d613ad4d760fa5f3c) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>bf2a471 -
Open-source -> Open-science in citation
8ffc30e
-
Update date.
908fd7c -
Training cost estimate (#4)
- Training cost estimate (57285788330586941122ac23f9eee9a675de942f) Co-authored-by: Teven Le Scao <teven@users.noreply.huggingface.co>359f0e6
-
Moves Chris up in the author list
Due to his heroic efforts getting this out over the past few days.9b8168a -
Changing name from "Open-source" to "Open-science"
RAIL license is open-source-like; perhaps open-science is a better term. Discussion here: https://huggingface.slack.com/archives/C020B9V3L1K/p16534101950014292701b5e