Instructions to use AItonomy/PhAI-IDE-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AItonomy/PhAI-IDE-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AItonomy/PhAI-IDE-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AItonomy/PhAI-IDE-9B")
model = AutoModelForMultimodalLM.from_pretrained("AItonomy/PhAI-IDE-9B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AItonomy/PhAI-IDE-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AItonomy/PhAI-IDE-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AItonomy/PhAI-IDE-9B

SGLang

How to use AItonomy/PhAI-IDE-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AItonomy/PhAI-IDE-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AItonomy/PhAI-IDE-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AItonomy/PhAI-IDE-9B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AItonomy/PhAI-IDE-9B with Docker Model Runner:
```
docker model run hf.co/AItonomy/PhAI-IDE-9B
```

PhAI-IDE-9B / README.md

leyili6666

Simplify published comparisons to complete benchmark configurations

71fa225 verified about 2 months ago

preview code

Raw History Blame

8.56 kB

metadata

model_name: PhAI-IDE-9B
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
datasets:
  - AItonomy/ScienceIDE
library_name: transformers
pipeline_tag: image-text-to-text
license: apache-2.0
tags:
  - phai-ide
  - science
  - code
  - tool-use
  - sft
  - lora
  - safetensors

PhAI-IDE

PhAI-IDE is a family of models for scientific coding and interaction with tools, available in 4B, 9B, and 72B sizes. Each model is supervised fine-tuned with ms-swift and released as full BF16 weights with the final LoRA adapter merged, together with its configuration and tokenizer.

The training dataset is Codex trajectories, sourced from AItonomy/ScienceIDE.

Models

Model	Base model	BF16 weights	License
PhAI-IDE-4B	Qwen3.5-4B	9.08 GB	Apache-2.0
PhAI-IDE-9B	Qwen3.5-9B	18.82 GB	Apache-2.0
PhAI-IDE-72B	Qwen2.5-72B-Instruct	145.41 GB	Qwen

Weight sizes are approximate; inference also requires memory for runtime allocations and the KV cache.

ScienceAccelBench performance

PhAI-IDE-9B improves the task-pooled pass rate from 17.09% to 20.89% (+3.80 percentage points) across all 27 ScienceAccelBench environments, using 158 valid paired tasks.

Task-held-out, localized scientific-code repair on familiar codebases, with original numerical verification. Each row compares the initial model with its corresponding fine-tuned model on identical tasks. Pass rates are percentages; gains are percentage points.

Model	Environment	Tasks	Initial model	PhAI-IDE	Gain (pp)
4B	PLUTO-Particles-Dust	3	0.00	33.33	+33.33
9B	LAPS	16	31.25	50.00	+18.75
9B	MITgcm-biogeo	8	0.00	12.50	+12.50
9B	PLUTO-RMHD	7	0.00	28.57	+28.57

Comparison with published models

Scores (%), grouped by benchmark and model size. Each reference entry gives its published score and the PhAI-IDE score difference in percentage points. Reference models are approximately the same size: 3–4B, 7–9B, and 67–72B, respectively.

PhAI-IDE	Benchmark	Score	Reference models: score (difference)
4B	BBH multistep-arithmetic-two	97.60	Llama-3.2-3B-Instruct (3.21B): 53.2 (+44.40); Phi-3.5-mini-8k-instruct (3.82B): 95.6 (+2.00)
9B	BBH word-sorting	60.40	Llama-3.1-8B-Instruct (8.03B): 51.2 (+9.20); Qwen2.5-7B-Instruct (7.62B): 15.6 (+44.80)
9B	MATH-500	92.20	InternLM3-8B-Instruct (8B): 83 (+9.20); Qwen2.5-7B-Instruct (7B): 72.4 (+19.80); Llama-3.1-8B-Instruct (8B): 48.4 (+43.80)
72B	AQuA-RAT	77.56	Llama-2-70B-Chat (70B): 31.32 (+46.24)
72B	ARC-Easy	84.64	Llama-2-70B (70B): 76.5 (+8.14); DeepSeek-LLM-67B-Chat (67B): 81.6 (+3.04)
72B	ARC-Challenge	64.42	Llama-2-70B (70B): 59.5 (+4.92); DeepSeek-LLM-67B-Chat (67B): 64.1 (+0.32)

Reference scores come from the linked publications, model cards, and independent evaluation reports; evaluation settings and sample counts vary by source. Differences describe reported scores across evaluations, rather than matched-protocol head-to-head gains. BBH entries refer to the named tasks.

Quick start

Use Transformers 5.16.1, PyTorch and Accelerate. Set model_id to any model in the table above; the example selects the matching model class.

from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForImageTextToText

model_id = "AItonomy/PhAI-IDE-4B"
loader = AutoModelForCausalLM if model_id.endswith("72B") else AutoModelForImageTextToText
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = loader.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain how to verify a numerical simulation."}],
    add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training procedure

ScienceIDE demonstrations were collected with GPT-5.6-sol and filtered using a numerical-equivalence verifier. They capture code inspection, tool use, and responses to execution feedback.

All three models use ms-swift supervised fine-tuning with LoRA across trainable linear layers for three epochs. The release merges each final checkpoint's adapter into its base model. Retained assistant targets provide the next-token training signal, while conversation history and tool observations provide context. The trajectories retain the native exec / wait interaction format. Heuristic target masking selects assistant actions for supervision while preserving the surrounding interaction history.

Shared setting	Value
Training dataset	Codex trajectories
Training examples / tasks	4,567 segments / 564 tasks
Validation examples / tasks	544 segments / 81 tasks
Train/validation task overlap	0
Training epochs	3
LoRA rank / alpha / dropout	32 / 64 / 0.05
Released weights	LoRA merged into BF16 Safetensors

Long trajectories are organized into segments. Source partition assignments are preserved, with no task identifiers shared between training and validation.

For 4B and 9B, fine-tuning uses text trajectories with the vision tower and aligner frozen. Their additional settings are:

Setting	4B and 9B
Maximum training sequence length	36,864 tokens
Learning rate / schedule	2e-5 / cosine
Warmup ratio	0.03
Batch size per device / accumulation	1 / 16
LoRA targets	Linear modules in the unfrozen language model
Optimizer steps	216
Packing	Disabled

Framework versions

The release was validated with the following environment.

Component	Version
Python	3.11
ms-swift	4.5.3
Transformers	5.16.1
PyTorch	2.6.0+cu124
PEFT	0.20.0
Datasets	4.8.4
Tokenizers	0.23.2
Accelerate	1.14.0

Citations

The training framework is ms-swift, and adapter training uses LoRA.

@misc{zhao2024swift,
  title = {SWIFT: A Scalable lightWeight Infrastructure for Fine-Tuning},
  author = {Yuze Zhao and others},
  year = {2024},
  eprint = {2408.05517},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.05517}
}

@misc{hu2021lora,
  title = {LoRA: Low-Rank Adaptation of Large Language Models},
  author = {Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen},
  year = {2021},
  eprint = {2106.09685},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.09685}
}