Instructions to use AItonomy/PhAI-IDE-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AItonomy/PhAI-IDE-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AItonomy/PhAI-IDE-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AItonomy/PhAI-IDE-9B")
model = AutoModelForMultimodalLM.from_pretrained("AItonomy/PhAI-IDE-9B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AItonomy/PhAI-IDE-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AItonomy/PhAI-IDE-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AItonomy/PhAI-IDE-9B

SGLang

How to use AItonomy/PhAI-IDE-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AItonomy/PhAI-IDE-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AItonomy/PhAI-IDE-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AItonomy/PhAI-IDE-9B with Docker Model Runner:
```
docker model run hf.co/AItonomy/PhAI-IDE-9B
```

PhAI-IDE-9B / README.md

leyili6666

Update README with verified ScienceAccelBench results

9981657 verified about 1 month ago

preview code

Raw History Blame

4.96 kB

metadata

model_name: PhAI-IDE-9B
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
datasets:
  - AItonomy/ScienceIDE
library_name: transformers
pipeline_tag: image-text-to-text
license: apache-2.0
tags:
  - phai-ide
  - science
  - code
  - tool-use
  - sft
  - lora
  - safetensors

PhAI-IDE-9B

PhAI-IDE-9B is a supervised fine-tune of Qwen/Qwen3.5-9B for scientific coding and interaction with tools, trained using ms-swift. The release contains full BF16 weights with the final LoRA adapter merged.

The training data is sourced from AItonomy/ScienceIDE.

Quick start

Use Transformers 5.16.1, PyTorch and Accelerate. The example loads the model, configuration and tokenizer directly from this repository.

from transformers import AutoTokenizer, AutoModelForImageTextToText

model_id = "AItonomy/PhAI-IDE-9B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="bfloat16", device_map="auto",
)
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain how to verify a numerical simulation."}],
    add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

BF16 weights occupy approximately 18.82 GB, plus runtime memory and KV cache.

ScienceAccelBench performance

Across all 27 ScienceAccelBench environments, the task-pooled pass rate increases from 17.09% to 20.89% (+3.80 percentage points) on 158 valid paired tasks.

Task-held-out, localized scientific-code repair on familiar codebases, with original numerical verification. Pass rates (%); gains in percentage points.

Environment	Tasks	Qwen3.5-9B	PhAI-IDE-9B	Gain (pp)
laps	16	31.25	50.00	+18.75
mitgcm-biogeo	8	0.00	12.50	+12.50
pluto-rmhd-resrmhd	7	0.00	28.57	+28.57

Training procedure

ScienceIDE demonstrations were collected with GPT-5.6-sol and filtered using a numerical-equivalence verifier. They capture code inspection, tool use, and responses to execution feedback.

Training uses ms-swift supervised fine-tuning with LoRA across trainable linear layers for three epochs. The release merges the final checkpoint's adapter into the base model. Retained assistant targets provide the next-token training signal, while conversation history and tool observations provide context. The Codex-only data retains the native exec / wait interaction format. Heuristic target masking selects assistant actions for supervision while preserving the surrounding interaction history.

Fine-tuning uses text trajectories with the vision tower and aligner frozen. The merged export supports standard autoregressive Qwen3.5 generation.

Setting	Value
Training dataset	Codex trajectories
Training examples / tasks	4,567 segments / 564 tasks
Validation examples / tasks	544 segments / 81 tasks
Train/validation task overlap	0
Training epochs	3
LoRA rank / alpha / dropout	32 / 64 / 0.05
Released weights	LoRA merged into BF16 Safetensors
Maximum training sequence length	36,864 tokens
Learning rate / schedule	2e-5 / cosine
Warmup ratio	0.03
Batch size per device / accumulation	1 / 16
LoRA targets	Linear modules in the unfrozen language model
Optimizer steps	216
Packing	Disabled

Long trajectories are organized into segments. Source partition assignments are preserved, with no task identifiers shared between training and validation.

Framework versions

The release was validated with the following environment.

Component	Version
Python	3.11
ms-swift	4.5.3
Transformers	5.16.1
PyTorch	2.6.0+cu124
PEFT	0.20.0
Datasets	4.8.4
Tokenizers	0.23.2
Accelerate	1.14.0

Citations

The training framework is ms-swift, and adapter training uses LoRA.

@misc{zhao2024swift,
  title = {SWIFT: A Scalable lightWeight Infrastructure for Fine-Tuning},
  author = {Yuze Zhao and others},
  year = {2024},
  eprint = {2408.05517},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.05517}
}

@misc{hu2021lora,
  title = {LoRA: Low-Rank Adaptation of Large Language Models},
  author = {Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen},
  year = {2021},
  eprint = {2106.09685},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.09685}
}