Instructions to use HemmingwayAI/Hemmingway-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HemmingwayAI/Hemmingway-1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HemmingwayAI/Hemmingway-1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HemmingwayAI/Hemmingway-1")
model = AutoModelForCausalLM.from_pretrained("HemmingwayAI/Hemmingway-1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HemmingwayAI/Hemmingway-1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HemmingwayAI/Hemmingway-1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HemmingwayAI/Hemmingway-1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HemmingwayAI/Hemmingway-1

SGLang

How to use HemmingwayAI/Hemmingway-1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HemmingwayAI/Hemmingway-1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HemmingwayAI/Hemmingway-1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HemmingwayAI/Hemmingway-1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HemmingwayAI/Hemmingway-1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use HemmingwayAI/Hemmingway-1 with Docker Model Runner:
```
docker model run hf.co/HemmingwayAI/Hemmingway-1
```

Gemma Version

by Alptekinege - opened Sep 20

Discussion

Alptekinege

Sep 20

Is there a chance the Gemma version will be released?26 billion or 31 billion version it doesn't matter

nonetrix

Sep 21

•

edited Sep 21

I agree, Qwen model is horrible base for such a fine-tune. Qwen is too hyper-focused on coding, Gemma has considerably better creative writing performance out of the box very much at the sacrifice of everything else before anyone calls me dumb, it's a joke when it comes to coding in comparison to 3.8

nmh2k3

Sep 22

Gemma 4 is really good at creative works, not just English, Chinese like Qwen(which's actually trained for coding purpose).
Gemma can handle novel writing,role play in many languages.

mcgonzo

Sep 23

yes, but be aware that gemma4 doesnt have that good toolcalling, logic and more refusals than qwen, and that gemma already prove to have strongly degraded coherence after its 64k attention window is filled (even if you allow it a ctx-window of 256k).

Ryan2025A

Sep 26

Gemma 4 models—at least the 12B and 26B variants, which I have tested many times - often perform poorly when a task requires reliably aggregating details distributed across long passages, especially details that cannot be retained within its single 1k-token local-attention window.
Give it the longest novels (in my case, 128K+) you're familiar with and ask it to survey the text for details, entities, and facts, e.g. how many people character A has fought, who are they, list by gender, or by faction etc. You'll see what that means. One frustrating pattern is that it would try to summarize the material by chapter at the beginning of the reasoning process and work from there. And even worse sometimes it gets stuck in the endless summarize-and-analyze loops.

matfax

Oct 2

If gemma 4 is tuned on their dataset, I'd at least suggest using either a MoE or the 12B variant, to fill a different role where speed matters more.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment