Instructions to use nvidia/Eagle2-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/Eagle2-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="nvidia/Eagle2-1B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nvidia/Eagle2-1B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/Eagle2-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/Eagle2-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Eagle2-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/Eagle2-1B

SGLang

How to use nvidia/Eagle2-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/Eagle2-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Eagle2-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/Eagle2-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Eagle2-1B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use nvidia/Eagle2-1B with Docker Model Runner:
```
docker model run hf.co/nvidia/Eagle2-1B
```

Zhiding commited on Jan 20, 2025

Commit

7b7790d

1 Parent(s): 7480170

update

Browse files

Files changed (7) hide show

README.md +0 -1
configuration_eagle_chat.py +1 -1
configuration_multi_backbone_channel_concatentation_model.py +1 -1
modeling_eagle_chat.py +1 -1
modeling_siglip.py +12 -0
multi_backbone_channel_concatenation_encoder.py +6 -4
multi_backbone_channel_concatentation_model.py +7 -0

README.md CHANGED Viewed

@@ -42,7 +42,6 @@ We provide the following models:
 | Eagle2-1B | [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) |  Siglip    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-1B)|
 | Eagle2-2B | [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |  Siglip    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-2B)|
 | Eagle2-9B | [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |  Siglip+ConvNext    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-9B)|
-| Eagle2-32B | [Qwen2.5-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct) |  Siglip+ConvNext    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-32B)|
 ## Benchmark Results
 |          Benchmark           | LLaVa-One-Vision-0.5B | InternVL2-1B | InternVL2.5-1B |Qwen2-VL-2B| Eagle2-1B|

 | Eagle2-1B | [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) |  Siglip    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-1B)|
 | Eagle2-2B | [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |  Siglip    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-2B)|
 | Eagle2-9B | [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |  Siglip+ConvNext    | 16K| [🤗 link](https://huggingface.co/NVIDIA/Eagle2-9B)|
 ## Benchmark Results
 |          Benchmark           | LLaVa-One-Vision-0.5B | InternVL2-1B | InternVL2.5-1B |Qwen2-VL-2B| Eagle2-1B|

configuration_eagle_chat.py CHANGED Viewed

@@ -1,7 +1,7 @@
 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
-# Licensed under The MIT License [see LICENSE for details]
 # --------------------------------------------------------
 import copy

 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
+# Licensed under The Apache License [see LICENSE for details]
 # --------------------------------------------------------
 import copy

configuration_multi_backbone_channel_concatentation_model.py CHANGED Viewed

@@ -1,7 +1,7 @@
 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
-# Licensed under The MIT License [see LICENSE for details]
 # --------------------------------------------------------
 import os

 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
+# Licensed under The Apache License [see LICENSE for details]
 # --------------------------------------------------------
 import os

modeling_eagle_chat.py CHANGED Viewed

@@ -1,7 +1,7 @@
 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
-# Licensed under The MIT License [see LICENSE for details]
 # --------------------------------------------------------
 import warnings

 # --------------------------------------------------------
 # Eagle2
 # Copyright (c) 2025 NVIDIA
+# Licensed under The Apache License [see LICENSE for details]
 # --------------------------------------------------------
 import warnings

modeling_siglip.py CHANGED Viewed

@@ -1,3 +1,11 @@
 # coding=utf-8
 # Copyright 2024 Google AI and The HuggingFace Team. All rights reserved.
 #
@@ -374,6 +382,10 @@ class SiglipAttention(nn.Module):
         """Input shape: Batch x Time x Channel"""
         if self.use_flash_attn:
             return self._flash_attn(hidden_states)
         batch_size, q_len, _ = hidden_states.size()
         query_states = self.q_proj(hidden_states)

+# --------------------------------------------------------
+# Eagle2
+# Copyright (c) 2025 NVIDIA
+# Licensed under The MIT License [see LICENSE for details]
+# Support flash-attention in SigLIP
+# --------------------------------------------------------
 # coding=utf-8
 # Copyright 2024 Google AI and The HuggingFace Team. All rights reserved.
 #
         """Input shape: Batch x Time x Channel"""
         if self.use_flash_attn:
             return self._flash_attn(hidden_states)
+        else:
+            return self._vanilla_attn(hidden_states, attention_mask, output_attentions)
+    def _vanilla_attn(self, hidden_states, attention_mask=None, output_attentions=False):
         batch_size, q_len, _ = hidden_states.size()
         query_states = self.q_proj(hidden_states)

multi_backbone_channel_concatenation_encoder.py CHANGED Viewed

@@ -1,13 +1,15 @@
 import torch, os
 import torch.nn as nn
 from torch.utils.checkpoint import checkpoint
 from .siglip_vision_tower import SiglipVisionTower
-# from .hr_clip_encoder import HRCLIPVisionTower
-# from .eva_vit import EVAVITVisionTower
-# from .SAM.modeling_sam import SAMVisionTower
-# from .pix2struct_large import Pix2StructLargeVisionTower
 import torch.nn.functional as F
 from torch.nn.init import trunc_normal_
 from copy import deepcopy

+# --------------------------------------------------------
+# Eagle2
+# Copyright (c) 2025 NVIDIA
+# Licensed under The Apache License [see LICENSE for details]
+# --------------------------------------------------------
 import torch, os
 import torch.nn as nn
 from torch.utils.checkpoint import checkpoint
 from .siglip_vision_tower import SiglipVisionTower
 import torch.nn.functional as F
 from torch.nn.init import trunc_normal_
 from copy import deepcopy

multi_backbone_channel_concatentation_model.py CHANGED Viewed

@@ -1,3 +1,10 @@
 import torch.nn as nn
 from transformers.modeling_outputs import BaseModelOutputWithPooling

+# --------------------------------------------------------
+# Eagle2
+# Copyright (c) 2025 NVIDIA
+# Licensed under The Apache License [see LICENSE for details]
+# --------------------------------------------------------
 import torch.nn as nn
 from transformers.modeling_outputs import BaseModelOutputWithPooling