Image-Text-to-Text
Transformers
Safetensors
Yue Chinese
Chinese
English
qwen3_5
agens
blockway
qwen3.8
cantonese
廣東話
hong-kong
multimodal
vision
agent
agentic
code
long-context
chat
conversational
compressed-tensors
Instructions to use Blockway/Agens-Pilot-Int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Blockway/Agens-Pilot-Int4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Blockway/Agens-Pilot-Int4") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Blockway/Agens-Pilot-Int4") model = AutoModelForMultimodalLM.from_pretrained("Blockway/Agens-Pilot-Int4", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Blockway/Agens-Pilot-Int4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Blockway/Agens-Pilot-Int4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blockway/Agens-Pilot-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Blockway/Agens-Pilot-Int4
- SGLang
How to use Blockway/Agens-Pilot-Int4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Blockway/Agens-Pilot-Int4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blockway/Agens-Pilot-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Blockway/Agens-Pilot-Int4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blockway/Agens-Pilot-Int4", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Blockway/Agens-Pilot-Int4 with Docker Model Runner:
docker model run hf.co/Blockway/Agens-Pilot-Int4
Rename to Agens Pilot; correct param count (~27B)
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ tags:
|
|
| 20 |
---
|
| 21 |
|
| 22 |
<p align="center">
|
| 23 |
-
<img src="./agens-banner.png" alt="Agens Pilot
|
| 24 |
</p>
|
| 25 |
|
| 26 |
<p align="center">
|
|
@@ -31,23 +31,23 @@ tags:
|
|
| 31 |
<p align="center">
|
| 32 |
<img src="https://img.shields.io/badge/License-Apache%202.0-2ea043">
|
| 33 |
<img src="https://img.shields.io/badge/Context-1M%20tokens-22d3ee">
|
| 34 |
-
<img src="https://img.shields.io/badge/Params-~
|
| 35 |
<img src="https://img.shields.io/badge/Modality-Text%20%2B%20Vision-0ea5e9">
|
| 36 |
<img src="https://img.shields.io/badge/Builds-BF16%20%7C%20FP8%20%7C%20INT4-475569">
|
| 37 |
</p>
|
| 38 |
|
| 39 |
<p align="center">
|
| 40 |
🤗 <b>Weights on Hugging Face:</b>
|
| 41 |
-
<a href="https://huggingface.co/Blockway/Agens-Pilot
|
| 42 |
-
<a href="https://huggingface.co/Blockway/Agens-Pilot-
|
| 43 |
-
<a href="https://huggingface.co/Blockway/Agens-Pilot-
|
| 44 |
</p>
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
-
## Meet Agens Pilot
|
| 49 |
|
| 50 |
-
**Agens Pilot
|
| 51 |
that writes code like a much larger model, reads a **whole repository in one context window**,
|
| 52 |
sees images, and speaks English, 廣東話, 简体中文 and 繁體中文 fluently.
|
| 53 |
|
|
@@ -57,7 +57,7 @@ a multi-GPU server to a single high-end card.
|
|
| 57 |
|
| 58 |
- **Developer:** Blockway (BlockWay Link Limited · 博睿鏈科有限公司), Hong Kong · founded 2018
|
| 59 |
- **Website:** https://blockway.io
|
| 60 |
-
- **Model type:** Decoder-only multimodal LLM (text + vision), ~
|
| 61 |
- **Context length:** up to **1,000,000 tokens** (262,144 native; extended to 1M with YaRN)
|
| 62 |
- **Languages:** English, Cantonese, Simplified & Traditional Chinese (and more)
|
| 63 |
- **License:** Apache-2.0 — commercial use welcome
|
|
@@ -66,7 +66,7 @@ a multi-GPU server to a single high-end card.
|
|
| 66 |
|
| 67 |
## 🤖 Purpose-built for Blockway's agent harnesses
|
| 68 |
|
| 69 |
-
Agens Pilot
|
| 70 |
agent stacks**, so the model and the harness are tuned together:
|
| 71 |
|
| 72 |
| Harness | What it is | Status |
|
|
@@ -83,7 +83,7 @@ so you're never locked in.
|
|
| 83 |
|
| 84 |
## Highlights
|
| 85 |
|
| 86 |
-
- ⚡ **Strong coding** — 97.0 HumanEval, ~83 LiveCodeBench v6; top-tier for a ~
|
| 87 |
- 📚 **Ultra-long context** — up to **1M tokens** for whole-repo and long-document work, kept
|
| 88 |
affordable by a hybrid linear-attention design.
|
| 89 |
- 🛠️ **Agentic & tool use** — reliable function calling and multi-step tool workflows.
|
|
@@ -95,12 +95,12 @@ so you're never locked in.
|
|
| 95 |
|
| 96 |
---
|
| 97 |
|
| 98 |
-
## How Agens Pilot
|
| 99 |
|
| 100 |
-
A capability-level positioning against a typical open model in the same ~
|
| 101 |
(characteristics vary by model — verify against the specific model you're evaluating):
|
| 102 |
|
| 103 |
-
| | **Agens Pilot
|
| 104 |
|---|---|---|
|
| 105 |
| Context window | **up to 1,000,000 tokens** | 32K – 128K |
|
| 106 |
| Modality | **Text + Vision (native)** | usually text-only |
|
|
@@ -122,9 +122,9 @@ differ only in weight precision.
|
|
| 122 |
|
| 123 |
| Variant | Repo | Size | Precision recipe | Quality | Suggested hardware |
|
| 124 |
|---|---|---|---|---|---|
|
| 125 |
-
| **BF16** | `Blockway/Agens-Pilot
|
| 126 |
-
| **FP8** ⭐ | `Blockway/Agens-Pilot-
|
| 127 |
-
| **INT4** | `Blockway/Agens-Pilot-
|
| 128 |
|
| 129 |
**Why the quantized builds aren't smaller.** For stability, the hybrid **linear-attention
|
| 130 |
layers** and the token **embeddings / LM head** are always kept in bf16 — quantizing the
|
|
@@ -144,7 +144,7 @@ flag needed.
|
|
| 144 |
```bash
|
| 145 |
python3 -m sglang.launch_server \
|
| 146 |
--model-path <path-to-variant> \
|
| 147 |
-
--served-model-name "Agens Pilot
|
| 148 |
--tp-size 4 \
|
| 149 |
--context-length 1048576 \
|
| 150 |
--tool-call-parser qwen3_coder \
|
|
@@ -157,7 +157,7 @@ python3 -m sglang.launch_server \
|
|
| 157 |
```bash
|
| 158 |
curl http://localhost:8000/v1/chat/completions \
|
| 159 |
-H "Content-Type: application/json" \
|
| 160 |
-
-d '{"model":"Agens Pilot
|
| 161 |
```
|
| 162 |
|
| 163 |
`tp-size` may be 1–4. FP8 fits 2×24 GB; INT4 fits a single ≥32 GB card (or 24 GB with `--cpu-offload-gb`).
|
|
@@ -169,8 +169,8 @@ curl http://localhost:8000/v1/chat/completions \
|
|
| 169 |
from transformers import AutoProcessor, AutoModelForImageTextToText
|
| 170 |
|
| 171 |
model = AutoModelForImageTextToText.from_pretrained(
|
| 172 |
-
"Blockway/Agens-Pilot
|
| 173 |
-
processor = AutoProcessor.from_pretrained("Blockway/Agens-Pilot
|
| 174 |
|
| 175 |
messages = [{"role": "user", "content": "Write a Python function to check if a string is a palindrome."}]
|
| 176 |
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
@@ -186,7 +186,7 @@ print(processor.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=T
|
|
| 186 |
|
| 187 |
Measured on Blockway's internal evaluation harness (single-sample, thinking enabled, temp 0.6).
|
| 188 |
|
| 189 |
-
| Benchmark | Agens Pilot
|
| 190 |
|---|---|
|
| 191 |
| HumanEval (chat) | **97.0** |
|
| 192 |
| LiveCodeBench v6 | **~83** |
|
|
@@ -202,7 +202,7 @@ everyday vision; visual mathematics (MathVision) is the weakest axis and on the
|
|
| 202 |
|
| 203 |
### vs. recent open models (official reported scores)
|
| 204 |
|
| 205 |
-
| Benchmark | **Agens Pilot
|
| 206 |
|---|---|---|---|---|
|
| 207 |
| GPQA Diamond | **80.3** | 57.2 | 45.4 | 42.4 |
|
| 208 |
| HumanEval | **97.0** | — | ~88 | — |
|
|
@@ -247,8 +247,8 @@ See the accompanying `LICENSE` and `NOTICE` files. Training data used in develop
|
|
| 247 |
its own licenses; downstream users are responsible for their own compliance.
|
| 248 |
|
| 249 |
```bibtex
|
| 250 |
-
@misc{
|
| 251 |
-
title = {Agens Pilot
|
| 252 |
author = {Blockway (BlockWay Link Limited)},
|
| 253 |
year = {2026},
|
| 254 |
url = {https://blockway.io}
|
|
|
|
| 20 |
---
|
| 21 |
|
| 22 |
<p align="center">
|
| 23 |
+
<img src="./agens-banner.png" alt="Agens Pilot — by Blockway" width="820">
|
| 24 |
</p>
|
| 25 |
|
| 26 |
<p align="center">
|
|
|
|
| 31 |
<p align="center">
|
| 32 |
<img src="https://img.shields.io/badge/License-Apache%202.0-2ea043">
|
| 33 |
<img src="https://img.shields.io/badge/Context-1M%20tokens-22d3ee">
|
| 34 |
+
<img src="https://img.shields.io/badge/Params-~27B-8b5cf6">
|
| 35 |
<img src="https://img.shields.io/badge/Modality-Text%20%2B%20Vision-0ea5e9">
|
| 36 |
<img src="https://img.shields.io/badge/Builds-BF16%20%7C%20FP8%20%7C%20INT4-475569">
|
| 37 |
</p>
|
| 38 |
|
| 39 |
<p align="center">
|
| 40 |
🤗 <b>Weights on Hugging Face:</b>
|
| 41 |
+
<a href="https://huggingface.co/Blockway/Agens-Pilot">BF16</a> ·
|
| 42 |
+
<a href="https://huggingface.co/Blockway/Agens-Pilot-FP8">FP8</a> ·
|
| 43 |
+
<a href="https://huggingface.co/Blockway/Agens-Pilot-Int4">INT4</a>
|
| 44 |
</p>
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
+
## Meet Agens Pilot
|
| 49 |
|
| 50 |
+
**Agens Pilot** is Blockway's flagship open model: a 27-billion-parameter multimodal LLM
|
| 51 |
that writes code like a much larger model, reads a **whole repository in one context window**,
|
| 52 |
sees images, and speaks English, 廣東話, 简体中文 and 繁體中文 fluently.
|
| 53 |
|
|
|
|
| 57 |
|
| 58 |
- **Developer:** Blockway (BlockWay Link Limited · 博睿鏈科有限公司), Hong Kong · founded 2018
|
| 59 |
- **Website:** https://blockway.io
|
| 60 |
+
- **Model type:** Decoder-only multimodal LLM (text + vision), ~27B params, hybrid attention (linear + full) for efficient long context
|
| 61 |
- **Context length:** up to **1,000,000 tokens** (262,144 native; extended to 1M with YaRN)
|
| 62 |
- **Languages:** English, Cantonese, Simplified & Traditional Chinese (and more)
|
| 63 |
- **License:** Apache-2.0 — commercial use welcome
|
|
|
|
| 66 |
|
| 67 |
## 🤖 Purpose-built for Blockway's agent harnesses
|
| 68 |
|
| 69 |
+
Agens Pilot isn't just a checkpoint — it's the engine **fine-fitted for Blockway's own
|
| 70 |
agent stacks**, so the model and the harness are tuned together:
|
| 71 |
|
| 72 |
| Harness | What it is | Status |
|
|
|
|
| 83 |
|
| 84 |
## Highlights
|
| 85 |
|
| 86 |
+
- ⚡ **Strong coding** — 97.0 HumanEval, ~83 LiveCodeBench v6; top-tier for a ~27B model.
|
| 87 |
- 📚 **Ultra-long context** — up to **1M tokens** for whole-repo and long-document work, kept
|
| 88 |
affordable by a hybrid linear-attention design.
|
| 89 |
- 🛠️ **Agentic & tool use** — reliable function calling and multi-step tool workflows.
|
|
|
|
| 95 |
|
| 96 |
---
|
| 97 |
|
| 98 |
+
## How Agens Pilot compares
|
| 99 |
|
| 100 |
+
A capability-level positioning against a typical open model in the same ~27B class
|
| 101 |
(characteristics vary by model — verify against the specific model you're evaluating):
|
| 102 |
|
| 103 |
+
| | **Agens Pilot** | Typical open ~27B model |
|
| 104 |
|---|---|---|
|
| 105 |
| Context window | **up to 1,000,000 tokens** | 32K – 128K |
|
| 106 |
| Modality | **Text + Vision (native)** | usually text-only |
|
|
|
|
| 122 |
|
| 123 |
| Variant | Repo | Size | Precision recipe | Quality | Suggested hardware |
|
| 124 |
|---|---|---|---|---|---|
|
| 125 |
+
| **BF16** | `Blockway/Agens-Pilot` | ~51 GB | full precision | reference | 2×48 GB or 4×24 GB |
|
| 126 |
+
| **FP8** ⭐ | `Blockway/Agens-Pilot-FP8` | ~34 GB | FP8 on feed-forward + full-attention; linear-attention, embeddings, LM head & vision kept in bf16 | **matches BF16** | 1×48 GB or 2×24 GB |
|
| 127 |
+
| **INT4** | `Blockway/Agens-Pilot-Int4` | ~26 GB | INT4 (GPTQ, group 128) on the same layers | very close; slightly softer on borderline factual topics | ≥32 GB VRAM, or 24 GB + CPU offload |
|
| 128 |
|
| 129 |
**Why the quantized builds aren't smaller.** For stability, the hybrid **linear-attention
|
| 130 |
layers** and the token **embeddings / LM head** are always kept in bf16 — quantizing the
|
|
|
|
| 144 |
```bash
|
| 145 |
python3 -m sglang.launch_server \
|
| 146 |
--model-path <path-to-variant> \
|
| 147 |
+
--served-model-name "Agens Pilot" \
|
| 148 |
--tp-size 4 \
|
| 149 |
--context-length 1048576 \
|
| 150 |
--tool-call-parser qwen3_coder \
|
|
|
|
| 157 |
```bash
|
| 158 |
curl http://localhost:8000/v1/chat/completions \
|
| 159 |
-H "Content-Type: application/json" \
|
| 160 |
+
-d '{"model":"Agens Pilot","messages":[{"role":"user","content":"Write a Python is_prime(n)."}]}'
|
| 161 |
```
|
| 162 |
|
| 163 |
`tp-size` may be 1–4. FP8 fits 2×24 GB; INT4 fits a single ≥32 GB card (or 24 GB with `--cpu-offload-gb`).
|
|
|
|
| 169 |
from transformers import AutoProcessor, AutoModelForImageTextToText
|
| 170 |
|
| 171 |
model = AutoModelForImageTextToText.from_pretrained(
|
| 172 |
+
"Blockway/Agens-Pilot", torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)
|
| 173 |
+
processor = AutoProcessor.from_pretrained("Blockway/Agens-Pilot", trust_remote_code=True)
|
| 174 |
|
| 175 |
messages = [{"role": "user", "content": "Write a Python function to check if a string is a palindrome."}]
|
| 176 |
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
|
|
| 186 |
|
| 187 |
Measured on Blockway's internal evaluation harness (single-sample, thinking enabled, temp 0.6).
|
| 188 |
|
| 189 |
+
| Benchmark | Agens Pilot |
|
| 190 |
|---|---|
|
| 191 |
| HumanEval (chat) | **97.0** |
|
| 192 |
| LiveCodeBench v6 | **~83** |
|
|
|
|
| 202 |
|
| 203 |
### vs. recent open models (official reported scores)
|
| 204 |
|
| 205 |
+
| Benchmark | **Agens Pilot** | Llama 4 Scout · Meta | Mistral Small 3.1 · 24B | Gemma 3 · 27B |
|
| 206 |
|---|---|---|---|---|
|
| 207 |
| GPQA Diamond | **80.3** | 57.2 | 45.4 | 42.4 |
|
| 208 |
| HumanEval | **97.0** | — | ~88 | — |
|
|
|
|
| 247 |
its own licenses; downstream users are responsible for their own compliance.
|
| 248 |
|
| 249 |
```bibtex
|
| 250 |
+
@misc{agenspilot2026,
|
| 251 |
+
title = {Agens Pilot: a coding-focused multimodal LLM},
|
| 252 |
author = {Blockway (BlockWay Link Limited)},
|
| 253 |
year = {2026},
|
| 254 |
url = {https://blockway.io}
|