Instructions to use smoky1496/SpawnPoint-Coder-7B-AWQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use smoky1496/SpawnPoint-Coder-7B-AWQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="smoky1496/SpawnPoint-Coder-7B-AWQ")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("smoky1496/SpawnPoint-Coder-7B-AWQ")
model = AutoModelForCausalLM.from_pretrained("smoky1496/SpawnPoint-Coder-7B-AWQ", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use smoky1496/SpawnPoint-Coder-7B-AWQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "smoky1496/SpawnPoint-Coder-7B-AWQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smoky1496/SpawnPoint-Coder-7B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/smoky1496/SpawnPoint-Coder-7B-AWQ

SGLang

How to use smoky1496/SpawnPoint-Coder-7B-AWQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "smoky1496/SpawnPoint-Coder-7B-AWQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smoky1496/SpawnPoint-Coder-7B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "smoky1496/SpawnPoint-Coder-7B-AWQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "smoky1496/SpawnPoint-Coder-7B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use smoky1496/SpawnPoint-Coder-7B-AWQ with Docker Model Runner:
```
docker model run hf.co/smoky1496/SpawnPoint-Coder-7B-AWQ
```

smoky1496 commited on 12 days ago

Commit

0de517a

verified ·

1 Parent(s): 75aed62

SpawnPoint-Coder-7B v5.1: indirectly named heroes fixed, older prompt sets recovered

Browse files

Files changed (2) hide show

README.md +20 -18
model-00001-of-00002.safetensors +1 -1

README.md CHANGED Viewed

@@ -20,7 +20,9 @@ What it does inside the pipeline:
 4-bit AWQ, runs on a single 12 GB GPU.
-**What's new in v5**
 - **Side-scrolling shooters** ("a falcon ship fights drones and saucers, the boss is a mothership"): enemies that
   fly straight, in waves, dive at you or stop and shoot; power-ups; a boss.
 - **Vietnamese is read directly**: no separate translation step, so names come out right ("chú vịt" is a Duck).
@@ -47,30 +49,30 @@ Hand-written prompts in English and Vietnamese through the whole pipeline, same
 Sealed sets were written before training and not looked at until the release decision. The first two rows include
 prompts about the genre v4 did not know, so v4 cannot be expected to answer them.
-| benchmark (prompts fully right) | v4 | v5 (this release) |
-|---|---|---|
-| **sealed set: shooters and the v4 fixes** (16 prompts) | 56% | 88% |
-| dev set: shooters and the v4 fixes (24 prompts) | 54% | 83% |
-| sealed set: farms, tower defense, cartoon (12 prompts) | 67% | 92% |
-| 16 prompts: farms, tower defense, melee, cartoon | 75% | 75% |
-| sealed set: the first nine genres (12 prompts) | 67% | 58% |
-| 30 human-written prompts | 87% | 80% |
-| 12 prompts: bubble, rescue, survivor | 92% | 92% |
-**Where v5 is a little behind v4**: on the 30 prompts and the sealed set written before shooters and farms existed.
-Some of those prompts are now genuinely ambiguous ("a space shooter" can be a side-scrolling shooter, "a farm game"
-a farming game), and a few name the player only indirectly ("a tank shooting monsters": the tank can come out as the
-default hero). Results vary by one to three prompts per set from run to run. A patch for this is planned.
-Held-out synthetic examples with one exact answer (v5 only: they use the v5 format): briefs 73%, tweaks 98%, entity edits 100%.
 The 4-bit AWQ files you download, measured the same way:
 | benchmark (prompts fully right) | AWQ build |
 |---|---|
-| **sealed set: shooters and the v4 fixes** (16 prompts) | 75% |
-| 30 human-written prompts | 77% |
-| dev set: shooters and the v4 fixes (24 prompts) | 83% |
 ## Limitations

 4-bit AWQ, runs on a single 12 GB GPU.
+**What's new in v5.1 and v5**
+- **v5.1** fixes prompts that name the player only indirectly ("a tank shooting monsters": the tank is now the hero,
+  not the default "Hero").
 - **Side-scrolling shooters** ("a falcon ship fights drones and saucers, the boss is a mothership"): enemies that
   fly straight, in waves, dive at you or stop and shoot; power-ups; a boss.
 - **Vietnamese is read directly**: no separate translation step, so names come out right ("chú vịt" is a Duck).
 Sealed sets were written before training and not looked at until the release decision. The first two rows include
 prompts about the genre v4 did not know, so v4 cannot be expected to answer them.
+| benchmark (prompts fully right) | v4 | v5 | v5.1 (this release) |
+|---|---|---|---|
+| **sealed set: shooters and the v4 fixes** (16 prompts) | 56% | 88% | 75% |
+| dev set: shooters and the v4 fixes (24 prompts) | 54% | 83% | 83% |
+| sealed set: farms, tower defense, cartoon (12 prompts) | 67% | 92% | 92% |
+| 16 prompts: farms, tower defense, melee, cartoon | 75% | 75% | 94% |
+| sealed set: the first nine genres (12 prompts) | 67% | 58% | 75% |
+| 30 human-written prompts | 87% | 80% | 87% |
+| 12 prompts: bubble, rescue, survivor | 92% | 92% | 83% |
+**Where v5.1 is still behind**: the sealed set of 16 shooter prompts at full precision (88% in v5, 75% in v5.1; the 4-bit builds below score
+higher, which is within run-to-run noise) and the newest 12-prompt set. Some prompts are genuinely ambiguous ("a space shooter" can be a side-scrolling shooter, "a farm game"
+a farming game). Results vary by one to three prompts per set from run to run, so read differences of one or two
+prompts as noise.
+Held-out synthetic examples with one exact answer (v5 format only): briefs 74%, tweaks 98%, entity edits 100%.
 The 4-bit AWQ files you download, measured the same way:
 | benchmark (prompts fully right) | AWQ build |
 |---|---|
+| **sealed set: shooters and the v4 fixes** (16 prompts) | 88% |
+| 30 human-written prompts | 80% |
+| dev set: shooters and the v4 fixes (24 prompts) | 88% |
 ## Limitations

model-00001-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4776735268176e396e1d4bda1b305aa0702ebc9070e141f3e3cea0149e059b8b
 size 4480835056

 version https://git-lfs.github.com/spec/v1
+oid sha256:cd012729342b3d366097a5c73bbace2d9ee1fbca064615bde2025a328943e10f
 size 4480835056