Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
-
Update README.md
Younes Belkadaadd link to DS checkpoints863f192
-
Fix the tokenizer class (#53)
- Fix the tokenizer class (7a258acf624dd2e54dbb1e61363f6b4b6639aa9f) Co-authored-by: Sylvain Gugger <sgugger@users.noreply.huggingface.co>e1d88ec -
Update README.md
373a962
-
Fix language name
7fc8195
-
Update README.md (#44)
ybelkada +1: Muennighoff - Update README.md (970902790bdc7b9090b897c3002bd01e9155e841) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>e965be3 -
re-add `code` as a language (#38)
ybelkada +1: cakiki
- re-add `code` as a language (593c0dc9dae7c5735815af7c5701a69fd9af90e4) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>e3b568c
-
more modifs on the model card (#37)
ybelkada- more modifs on the model card (10151622e2c7d487700e4ff054c8163e23699469) - Update README.md (033dc990073d975b79d9ea34b2ba8cf8dd248f0b)0200b62 -
update training information (#36)
ybelkada- update training information (6fa8e8a26b9023ec8718ebd2c2e123071ec55878) - Some rephrasing (a73c6202a969387e6f1ea390b0d886240c42b8f4)08b0018 -
Remove dtype (#35)
ybelkada +1: Muennighoff - Remove dtype (92451bc6530686b2170dcb430bc538f7715a83f0) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>436425e -
update programming languages (#33)
ybelkada- update programming languages (f1c45b15cc6ecd729b84cefef545e7dfe764c20b)d74ffd8 -
Revert language error (#32)
ybelkada- Revert language error (ad0a2e52d5f9280870bc0d53de0cac56f37c5587)263f552 -
Remove dup space (#31)
ybelkada +1: Muennighoff - Remove dup space (aed9e737a4d1fa94a7f238fa3a6a149683169c5e) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>7e1e135 -
Add programming languages (#29)
ybelkada +1: cakiki
- Add programming languages (5e375508260e5d8498317c07c71fbed9b777baa6) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>58660e8 -
Add arrows for code evaluation (#28)
ybelkada +1: Muennighoff - Add arrows for code evaluation (3298a1743aedd867ec5054e9c169cee49fd4d956) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>60e49ad -
Add evaluation (#27)
ybelkada +1: Muennighoff - Add evaluation (f4d13b5ba0ecc4e0ef2ac892aadcc7daa33d3529) - eng -> fra for crows_pairs_french (b07b3ae34315243657bf764d1ef5cf87d627dd50) - Move to subsection (f1a18285e419e05ef30872b0f1aa6c0a62ed5782) - Fix typos (d5d729d96775c846213efcc7c802613a4dd2c728) - Update training statistics (5ebccd1f300f458cfed43489512b120a51a2ef2a) - Update README.md (094e07d1004d2663cefd5f19fd8fe6282e0872c7) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>bb3556d -
updated opening blurb
Teven Le Scao168ece4 -
fixed pie chart link
Teven Le Scaobfd53df -
Removed non-standard simplified/traditional language tags
Teven Le Scao79bc63b -
Updated opening blurb
Teven Le Scaof689bf8 -
Fixed example
Teven Le Scaoa7ef188 -
Added reasoning example
Teven Le Scao5d3b298 -
Added widget examples
Teven Le Scaoa6cebd9 -
Added widget examples
Teven Le Scao2d60202
-
Updated pie chart
Teven Le Scao6adf557 -
Add link to interactive corpus treemap (#26)
- Add link to interactive corpus treemap (26e50ada4d39c875fde0c486eb860e4aa4a65c4c) Co-authored-by: Yacine Jernite <yjernite@users.noreply.huggingface.co>0140768 -
Add TensorBoard Traces (#25)
ybelkada- Add TensorBoard traces (24767be90ca0dd069d2559d5577c6e88fdb624c4)5556040 -
Correct number of languages (#24)
- Correct number of languages (69e78440cdbd55e8588883927a460cd8aa222c24) - Update README.md (0de6009ab9827a696a1d2fb1fcdedeb21088c6c0) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>34aa644 -
Update README.md (#23)
- Update README.md (537c688696c47203e5f9018b5b67109f9f1c2cb5) Co-authored-by: Victor M <victor@users.noreply.huggingface.co>eb49b9c -
modify padding side
c42411c -
add pad token id
Younes Belkada0142556
-
Add image for how to use (#20)
- Add image for how to use (e38c183a699782feb22b4252654791ee43c84489) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>46e5752
-
Remove .dev from transformers version (#21)
ybelkada- Remove .dev from transformers version (89303001337263eb28ee69dd51ec28fac73ac3ef) - add architecture (5efe5164c6fab318760c2a98485c73732fdfab78)85e429f -
Add correct dtype on config
Younes Belkadafbda88e
-
new data
acbc8df -
Updating italics to the HTML equivalent.
177dc5b -
Fixing italics that aren't rendering for "BigScience Large Open-science Open-access Multilingual Language Model"
833b211 -
Update README.md (#17)
- Update README.md (1d0ed019f32f5e27c155787f05b0fde415f613f4) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>83839ee -
Update README.md (#18)
ybelkada- Update README.md (2a44f086c0cbfd0609537820938b2172eb34cc32)44651d2 -
Update README.md (#15)
ybelkada- Update README.md (67f39504df45f38db346e853aea20873f434ca6a)0aa20cd -
Update README.md
Younes Belkadaf25baa0 -
Update README.md
Younes Belkada290df47 -
Update README.md (#16)
ybelkada- Update README.md (b3f36225ffaccd6bdeba3ab1d1bd0b9ef0368d90)45cf2c9 -
Adding blurb about mistaking sentience. (#12)
- Adding blurb about mistaking sentience. (64cee78cfab83e9f9a6a4ba68e90ab56eba707bf) - Adding sentience blurb. (09a4f1c6d307f8a080636b355b20da067d06e243) Co-authored-by: Margaret Mitchell <meg@users.noreply.huggingface.co>dc534f6 -
Update README.md (#14)
- Update README.md (52a7843b4d71889e146268f0d3927859f3e7dd5c) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>9d151d5
-
Add link to license article and FAQ (#13)
ybelkada +1: cakiki
- Add link to license article and FAQ (d05dd208e40b1f78168f8b6bc016c002af65fbca) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>ba402e7 -
Update README.md
Younes Belkadaa339d03
-
Updates the model card in light of more developed format. (#11)
- Updates the model card in light of more developed format. (e7e28b1ca071f697ad0cc98d30b32a0d3887768a)6cb7cc9
-
Update README.md
Younes BelkadaUpdate global step826ae56
-
Merge branch 'main' of https://huggingface.co/bigscience/bloom into main
c2f6b0a -
new data
18894c7