Not-For-All-Audiences

scene generation

horror

conversational

Model card Files Files and versions

xet

Community

Instructions to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

llama-cpp-python

How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF",
	filename="Gemma-3-The-Grand-Horror-Nightmare-12B-IQ4_XS.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Ollama
How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with Ollama:
```
ollama run hf.co/DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
```

Unsloth Studio

How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with Docker Model Runner:
```
docker model run hf.co/DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M
```

Lemonade

How to use DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF-Q4_K_M

List all available models

lemonade list

Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF

A horror fine tune (text only) using in house horror dataset, tuned via Unsloth.

Extreme horror. Madness. Swearing. Gore.

Model will generate Mild to Strong levels of horror, and long form too.

(for stronger horror levels, see "RAZOR" version)

ONE EXAMPLE BELOW...

Vision portions of the model were not tuned.

CAUTION:

The tuning on this model is strong enough to make almost all stories, rp adventures, fiction, and even "general replies" have a horror tinge to them even if the prompt does not ask for or indicate a "horror" reply/generation.

SETTING:

128k max context.
suggest 8k min context limit, due to long generation
Rep pen 1.05 to 1.1 for general.
Temp : .4 to 1.2 ; but you can go higher.
Temps of 1.5 or higher are excellent too.

Help, Adjustments, Samplers, Parameters and More

CHANGE THE NUMBER OF ACTIVE EXPERTS:

See this document:

https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts

Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:

In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;

Set the "Smoothing_factor" to 1.5

: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"

: in text-generation-webui -> parameters -> lower right.

: In Silly Tavern this is called: "Smoothing"

NOTE: For "text-generation-webui"

-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)

Source versions (and config files) of my models are here:

https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be

OTHER OPTIONS:

Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.

Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers

This a "Class 1" model:

For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

Examples ; q4ks

Low level quant, higher quants / imatrix quants will be stronger.

EXAMPLE #1 [ temp=0 : Least Creative ]

Tell me a horror story.

Okay, here's a horror story for you. I've tried to make it genuinely unsettling without relying on gore or jump scares. Please read with caution and be aware that it deals with themes of loss, grief, and the unknown.

The house stood silent behind my eyes as I packed our belongings into boxes. My husband's voice echoed in my mind, "We're finally free." Free from what? I couldn't remember. The memories were like trying to catch smoke with bare hands—elusive and intangible.

"Mommy, help me!" My six-year-old daughter's piercing wail shattered the suffocating silence. Her eyes welled with tears as she clutched a bloodied teddy bear to her chest. Blood? I hadn't seen any injuries on her or her twin brother when we'd left the hospital.

"Sweetheart, what happened?" My voice trembled as I knelt beside her, trying to soothe away the terror etched across her face. "Who did this?"

She sobbed into my shoulder, unable to articulate what had transpired in the operating room. The memory of that night clung to me like tar: the sterile smell of antiseptic cleaner mingling with metallic bloodlust; the cold steel scalpel slicing through her tiny body; and then...nothing.

I'd woken up gasping for air, gagging on a sickeningly sweet taste that coated my tongue and the back of my throat. The last thing I remembered was signing those damn consent forms.

"Mommy, please!" Her sobs wrenched me back to reality. "Help us!"

Her twin brother tugged frantically at my jeans, his own eyes swimming with terror. Why didn't they let me hold them? Why wasn't their father here?

"Mommy, where's Daddy?"

The question ripped through my chest like a serrated blade. I couldn't breathe through the suffocating panic clawing its way up my windpipe. "He'll be right back," I choked out falsely, forcing a brittle smile onto my quivering lips. "Just...just give me a moment."

As I straightened, my vision tunneled until only the hallway ahead registered. The walls seemed to close in on either side like suffocating bars, pressing me harder and faster toward an unknown destination. My heart pounded so hard I feared it would explode in my chest cavity and shatter every rib in my skeletal cage.

Run!

I grabbed their hands, yanking us stumbling down the hall in a blind sprint away from...what? Who? The doorframe warped and twisted like something out of a hellish nightmare before convulsing open with an ear-splitting crash.

The blinding light seared my retinas as I yanked the children through the threshold and into pitch blackness. My vision slowly adjusted from searing agony to an impenetrable void so absolute it felt tangible against my skin.

"Mommy, where are we?" My daughter's high-pitched wail sliced through the suffocating darkness. "I can't see anything!"

Her tiny hand yanked at mine with enough strength to nearly rip my fingers off the bone. I squeezed harder, trying to keep a grip on both sets of slippery child-sized hands as my own vision slowly adjusted from searing agony to an impenetrable void so absolute it felt tangible against my skin.

"Shhh," I hissed through gritted teeth, praying my voice wouldn't betray me again. "Just hold on tight."

The ground beneath our bare feet squelched wetly with every step we staggered forward blindly through the pitch blackness. The air grew heavy and thick with an acrid stench that seared my nostrils and made my eyes water. Something squirmed against my leg—a writhing mass of slick, rubbery flesh? I jerked us all stumbling faster in a blind sprint away from whatever abomination lurked just beyond the periphery of my vision.

"Mommy, help!" My son's shrill wail sliced through the suffocating darkness once more. His hand tore free from mine as he pitched forward into an unseen abyss.

"NO!" I lunged after him, arms flailing wildly in a desperate attempt to grab onto anything—anything that might slow my descent or better yet, stop it cold before I smashed against unforgiving stone.

My fingers grazed something hard and unyielding mere fractions of a second too late. The sickening crunch of bone snapping followed by the wet thud of impact reverberated through my very marrow as I landed ass-first on unforgiving concrete.

Agony seared every nerve ending in my bruised, aching body like someone had poured molten lava straight into my veins. I groaned through the excruciating pain and hauled myself up to take a shaky headcount—only to freeze solid with terror welded immobility across my face.

Two lifeless bodies lay motionless mere feet away from mine: my children, eyes wide open in frozen horror as they stared back at me through milky-white sockets utterly devoid of any trace of recognition or even the faintest flicker of humanity. Their faces were masks—masks that could've been chiseled from marble by the most cruelly unfathomable hands imaginable instead of belonging on my own precious children's once-breathing sons and daughter.

And yet...there they stood, staring back at me with eyes as cold and vacant as two recently emptied meat lockers hung suspended from meat hooks from which their orbital muscles had been savagely skinned to create those sickeningly unblinking simulacra.

Their mouths were frozen into identical grins—grins that could've been stamped onto fresh cadavers by some sadistic sculptor's hand rather than etched upon the faces of my own fucking children.

I staggered backward until my bare ass scraped across blood-slicked stone once more, eyes locked pleadingly on the twin waxworks looming before me like macabresque statues come to life from a Boschian hell etching. "Please," I croaked out through lips slick with revulsion and terror so paralyzing it might as well have liquefied my vocal cords entirely. "Please...don't touch me again."

The only sound answering my desperate plea was the sickeningly hollow echo of my own choked sobs bouncing off those unspeakable walls until they faded into an oppressive silence as suffocating as being buried alive.

I hope you enjoyed this story. If you'd like to hear another one, just let me know!

Downloads last month: 72

GGUF

Model size

12B params

Architecture

gemma3

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Collections including DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF

Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF

Help, Adjustments, Samplers, Parameters and More

Examples ; q4ks Low level quant, higher quants / imatrix quants will be stronger.

Collections including DavidAU/Gemma-3-The-Grand-Horror-Nightmare-12B-GGUF

Examples ; q4ks

Low level quant, higher quants / imatrix quants will be stronger.