Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
-
Fix architecture
752bd85 -
Fix architecture (#188)
- Fix architecture (942a3b868388daa9b8e643872f3e83997e699250) Co-authored-by: Lewis Tunstall <lewtun@users.noreply.huggingface.co>dede1ba
-
Fix typo in Examples (#181)
- Update README.md (abc035f4ec16d2082e2d8f9f2f55f1f4f45d3037) Co-authored-by: Spencer von der Ohe <vonderoh@users.noreply.huggingface.co>b366374
-
Update README.md
f0e3b92 -
Update README.md (#143)
- Update README.md (fa17b4ec0ec2030c94d57c010b91c225bac5c823) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co>0a1bf22
-
Add safetensors index (#142)
- Add safetensors index (eff12ac8a7e7a28d814c7852d40ec4e189a1f2f2) Co-authored-by: Sylvain Gugger <sgugger@users.noreply.huggingface.co>4bf48c6
-
Add carbon emissions metadata (#140)
- Add carbon emissions metadata (1ba44a462e09b5184ecd4621ae125ff9347e775a) Co-authored-by: Julien Chaumond <julien-c@users.noreply.huggingface.co>20ac6a9
-
Adding `safetensors` variant of this model (#121)
- Adding `safetensors` variant of this model (9796bcaf99ef5dc1a49e9bb0065a8885cf3c7de2) - simpler .gitattributes (3c9db30525ed4b355594ecf4ba6b0cbb83c5830b) Co-authored-by: Julien Chaumond <julien-c@users.noreply.huggingface.co>4d8e28c
-
Fix dataset name
4ab0472
-
Update default template for translation to spanish
Thomas Wang11a7c1b -
Improve default translation prompts (#102)
TimeRobber- Improve default translation prompts (339b3707db5c23eb7e74c2a3900e2ba10acf5f2b)8540347
-
Correct HumanEval scores (#79)
- Update README.md (5c5d5cf4a2e4c59287a69f8de35f2e9225df527a)aee8c40
-
Widget multiline examples correct behaviour (#67)
ybelkada +1: mishig - Widget multiline examples correct behaviour (3868ba1273a9e8dbbf96a34cbc28db3ffafe5db2) Co-authored-by: Mishig Davaadorj <mishig@users.noreply.huggingface.co>d9bf58e
-
cross link to meg-ds checkpoint
9e27a90
-
Add Lingala (#60)
- Add Lingala (0bed9c64c743c4272a01f0e37a950aa2d91fa705) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>2a3d62e
-
make the release date human-friendly
483e9e2
-
Update README.md (#57)
ybelkada- Update README.md (863f192ff72ecfd487bb57a5ded5752f18ddedbb)4051715
-
Fix the tokenizer class (#53)
- Fix the tokenizer class (7a258acf624dd2e54dbb1e61363f6b4b6639aa9f) Co-authored-by: Sylvain Gugger <sgugger@users.noreply.huggingface.co>e1d88ec -
Update README.md
373a962
-
Fix language name
7fc8195
-
Update README.md (#44)
ybelkada +1: Muennighoff - Update README.md (970902790bdc7b9090b897c3002bd01e9155e841) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>e965be3 -
re-add `code` as a language (#38)
ybelkada +1: cakiki
- re-add `code` as a language (593c0dc9dae7c5735815af7c5701a69fd9af90e4) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>e3b568c
-
more modifs on the model card (#37)
ybelkada- more modifs on the model card (10151622e2c7d487700e4ff054c8163e23699469) - Update README.md (033dc990073d975b79d9ea34b2ba8cf8dd248f0b)0200b62 -
update training information (#36)
ybelkada- update training information (6fa8e8a26b9023ec8718ebd2c2e123071ec55878) - Some rephrasing (a73c6202a969387e6f1ea390b0d886240c42b8f4)08b0018 -
Remove dtype (#35)
ybelkada +1: Muennighoff - Remove dtype (92451bc6530686b2170dcb430bc538f7715a83f0) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>436425e -
update programming languages (#33)
ybelkada- update programming languages (f1c45b15cc6ecd729b84cefef545e7dfe764c20b)d74ffd8 -
Revert language error (#32)
ybelkada- Revert language error (ad0a2e52d5f9280870bc0d53de0cac56f37c5587)263f552 -
Remove dup space (#31)
ybelkada +1: Muennighoff - Remove dup space (aed9e737a4d1fa94a7f238fa3a6a149683169c5e) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>7e1e135 -
Add programming languages (#29)
ybelkada +1: cakiki
- Add programming languages (5e375508260e5d8498317c07c71fbed9b777baa6) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>58660e8 -
Add arrows for code evaluation (#28)
ybelkada +1: Muennighoff - Add arrows for code evaluation (3298a1743aedd867ec5054e9c169cee49fd4d956) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>60e49ad -
Add evaluation (#27)
ybelkada +1: Muennighoff - Add evaluation (f4d13b5ba0ecc4e0ef2ac892aadcc7daa33d3529) - eng -> fra for crows_pairs_french (b07b3ae34315243657bf764d1ef5cf87d627dd50) - Move to subsection (f1a18285e419e05ef30872b0f1aa6c0a62ed5782) - Fix typos (d5d729d96775c846213efcc7c802613a4dd2c728) - Update training statistics (5ebccd1f300f458cfed43489512b120a51a2ef2a) - Update README.md (094e07d1004d2663cefd5f19fd8fe6282e0872c7) Co-authored-by: Niklas Muennighoff <Muennighoff@users.noreply.huggingface.co>bb3556d -
updated opening blurb
Teven Le Scao168ece4 -
fixed pie chart link
Teven Le Scaobfd53df -
Removed non-standard simplified/traditional language tags
Teven Le Scao79bc63b -
Updated opening blurb
Teven Le Scaof689bf8 -
Fixed example
Teven Le Scaoa7ef188 -
Added reasoning example
Teven Le Scao5d3b298 -
Added widget examples
Teven Le Scaoa6cebd9 -
Added widget examples
Teven Le Scao2d60202
-
Updated pie chart
Teven Le Scao6adf557 -
Add link to interactive corpus treemap (#26)
- Add link to interactive corpus treemap (26e50ada4d39c875fde0c486eb860e4aa4a65c4c) Co-authored-by: Yacine Jernite <yjernite@users.noreply.huggingface.co>0140768 -
Add TensorBoard Traces (#25)
ybelkada- Add TensorBoard traces (24767be90ca0dd069d2559d5577c6e88fdb624c4)5556040 -
Correct number of languages (#24)
- Correct number of languages (69e78440cdbd55e8588883927a460cd8aa222c24) - Update README.md (0de6009ab9827a696a1d2fb1fcdedeb21088c6c0) Co-authored-by: Christopher Akiki <cakiki@users.noreply.huggingface.co>34aa644 -
Update README.md (#23)
- Update README.md (537c688696c47203e5f9018b5b67109f9f1c2cb5) Co-authored-by: Victor M <victor@users.noreply.huggingface.co>eb49b9c -
modify padding side
c42411c -
add pad token id
Younes Belkada0142556