Instructions to use Divij/Llama-3.2-3B-Instruct-sft-with-thoughts with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Divij/Llama-3.2-3B-Instruct-sft-with-thoughts with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Divij/Llama-3.2-3B-Instruct-sft-with-thoughts")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Divij/Llama-3.2-3B-Instruct-sft-with-thoughts")
model = AutoModelForCausalLM.from_pretrained("Divij/Llama-3.2-3B-Instruct-sft-with-thoughts", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Divij/Llama-3.2-3B-Instruct-sft-with-thoughts with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Divij/Llama-3.2-3B-Instruct-sft-with-thoughts

SGLang

How to use Divij/Llama-3.2-3B-Instruct-sft-with-thoughts with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Divij/Llama-3.2-3B-Instruct-sft-with-thoughts with Docker Model Runner:
```
docker model run hf.co/Divij/Llama-3.2-3B-Instruct-sft-with-thoughts
```

Divij/Llama-3.2-3B-Instruct-sft-with-thoughts

Supervised fine-tune of meta-llama/Llama-3.2-3B-Instruct on a scientific-methodology instruction dataset, where each assistant response interleaves <Thought_i> reasoning with <Step_i> actions.

The project goal is to compare whether including explicit <Thought_i> reasoning traces alongside each <Step_i> action during SFT produces stronger scientific-methodology generators than training on step-only plans.

Variant

This checkpoint is the with-thoughts variant: The assistant target alternates <Thought_i> / <Step_i> pairs, so the model learns to produce explicit reasoning before each action. Trained with max_seq_length=6144 to fit the longer sequences.

Training data

Source: sft_with_thoughts.jsonl from the verl_scientific_discovery repeated-sampling pipeline.
4,990 messages-format examples (system + user + assistant).
Each assistant response is a step-by-step research methodology for a given Research Goal + Constraints prompt.

Training setup

Framework: open-instruct finetune.py (accelerate + FSDP2).
Hardware: 2× NVIDIA H100 NVL (96 GB).
Precision: bf16 mixed precision.
Attention: FlashAttention-2.
Memory: gradient checkpointing enabled.

Hyperparameters


`max_seq_length`	6144
`num_train_epochs`	3
`per_device_train_batch_size`	1
`gradient_accumulation_steps`	8
Effective batch size	16 (1 × 2 GPU × 8 accum)
`learning_rate`	2e-5
`lr_scheduler_type`	linear
`warmup_ratio`	0.03
`weight_decay`	0.0
`seed`	42
Optimizer	fused AdamW
Total optimization steps	936
Final training loss	0.839

The chat template is inherited from the base model (meta-llama/Llama-3.2-3B-Instruct). Labels are masked on the system and user turns so only the assistant response contributes to the loss (open-instruct's sft_tulu_tokenize_and_truncate_v1 transform).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "Divij/Llama-3.2-3B-Instruct-sft-with-thoughts"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are an expert research scientist. Produce reasoning/action pairs: <Thought_i> followed by <Step_i>."},
    {"role": "user", "content": (
        "You are given a scientific research problem.\n\n"
        "Research Goal:\n<your research goal here>\n\n"
        "Constraints:\n1) <constraint 1>\n2) <constraint 2>"
    )},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

output = model.generate(
    inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)
print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))

Notes

Context length. Use max_seq_length ≥ 6144 at inference time to match the training regime; generations longer than this were not seen during training.
Intended use. Research artifact for generating structured scientific research plans. Not aligned for general-purpose chat or safety-critical use.
Compared to sibling. A matching without-thoughts checkpoint at Divij/Llama-3.2-3B-Instruct-sft-without-thoughts is trained on the same data but with the opposite treatment of reasoning traces.

Downloads last month: 36

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for Divij/Llama-3.2-3B-Instruct-sft-with-thoughts

Base model

meta-llama/Llama-3.2-3B-Instruct

Finetuned

(2044)

this model