Instructions to use NagusameCS/saiyan-trunks with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NagusameCS/saiyan-trunks with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NagusameCS/saiyan-trunks")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NagusameCS/saiyan-trunks")
model = AutoModelForCausalLM.from_pretrained("NagusameCS/saiyan-trunks", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NagusameCS/saiyan-trunks with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NagusameCS/saiyan-trunks"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NagusameCS/saiyan-trunks",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NagusameCS/saiyan-trunks

SGLang

How to use NagusameCS/saiyan-trunks with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NagusameCS/saiyan-trunks" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NagusameCS/saiyan-trunks",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NagusameCS/saiyan-trunks" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NagusameCS/saiyan-trunks",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NagusameCS/saiyan-trunks with Docker Model Runner:
```
docker model run hf.co/NagusameCS/saiyan-trunks
```

saiyan-trunks

The language-model weights in this repo are an unmodified copy of Qwen/Qwen2.5-1.5B-Instruct, stored in float16. All 338 tensors equal the original bfloat16 tensors cast to float16. All nine saiyan-* repos contain the same model.safetensors (sha256 60d573380111ed3f…). As a language model, this repo behaves exactly like Qwen2.5-1.5B-Instruct. On 60 documents of 512 tokens it gives NLL 1.7283 against 1.7286 for the original, with 98.2% top-1 agreement. That matches the agreement between two bf16 runs of the identical weights, so the remaining differences are numerical noise.

state.pt holds a HyperTensor "living manifold" experiment state: a 20x20 metric and a few 20-dim projections of last-layer hidden states for domain prompts. transformers and other inference stacks never read it, and it does not change the model's outputs. In HyperTensor it is only used by the "geometric jury" similarity-scoring scripts.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("NagusameCS/saiyan-trunks")
model = AutoModelForCausalLM.from_pretrained("NagusameCS/saiyan-trunks")
msgs = [{"role": "user", "content": "What is the capital of France?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

License: Apache-2.0, inherited from Qwen2.5-1.5B-Instruct (see LICENSE).

Downloads last month: 11

Safetensors

Model size

2B params

Tensor type

F16

Model tree for NagusameCS/saiyan-trunks

Base model

Qwen/Qwen2.5-1.5B

Finetuned

Qwen/Qwen2.5-1.5B-Instruct

Finetuned

(2,110)

this model