Instructions to use microsoft/Florence-2-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Florence-2-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="microsoft/Florence-2-base", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("microsoft/Florence-2-base", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("microsoft/Florence-2-base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use microsoft/Florence-2-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Florence-2-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/microsoft/Florence-2-base

SGLang

How to use microsoft/Florence-2-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Florence-2-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Florence-2-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use microsoft/Florence-2-base with Docker Model Runner:
```
docker model run hf.co/microsoft/Florence-2-base
```

Florence-2-base

Commit History

Nov 3, 2024

10:53
Add GenerationMixin as parent class⁠

fcakyon

Verified signature
2a2d45e

Jul 20, 2024

00:12
update_model_init_fp16 (#12)⁠

leoxiaobin +1: haipingwu

Verified signature
ee1f1f1

Jul 19, 2024

00:44
update_ocr_postprocess (#11)⁠

leoxiaobin +1: haipingwu

Verified signature
09eb282

Jul 1, 2024

09:36
Update pipeline tag

nielsr HF Staff

Verified signature
53f3fb4

Jun 30, 2024

06:26
Fix modeling code (typos/bugs) (#5)⁠

leoxiaobin +1: Xenova

Verified signature
4fab3c0

Jun 26, 2024

07:27
Fix incorrect bos_token, eos_token, and pad_token ids in config.json (#6)⁠

leoxiaobin +1: Xenova

Verified signature
78e1dd4

Jun 18, 2024

22:40
Update README.md

leoxiaobin

Verified signature
a2f0be7
22:39
update_readme_example (#3)⁠

leoxiaobin +1: haipingwu

Verified signature
0f6358c
18:22
add_grounding_example (#2)⁠

leoxiaobin +1: haipingwu

Verified signature
37070d5

Jun 15, 2024

03:34
init_model (#1)⁠

leoxiaobin +1: haipingwu

Verified signature
a15cd1f
00:57
initial commit

leoxiaobin

Verified signature
c3302ad

Commit History

Nov 3, 2024

Add GenerationMixin as parent class⁠

Jul 20, 2024

update_model_init_fp16 (#12)⁠

Jul 19, 2024

update_ocr_postprocess (#11)⁠

Jul 1, 2024

Update pipeline tag

Jun 30, 2024

Fix modeling code (typos/bugs) (#5)⁠

Jun 26, 2024

Fix incorrect bos_token, eos_token, and pad_token ids in config.json (#6)⁠

Jun 18, 2024

Update README.md

update_readme_example (#3)⁠

add_grounding_example (#2)⁠

Jun 15, 2024

init_model (#1)⁠

initial commit