---
license: apache-2.0
tags:
- text-generation
- llama
- quartz
- causal-lm
- conversational
- safetensors
- gguf
- llama.cpp
pipeline_tag: text-generation
language:
- en
---

# Quartz-M1-100M-Instruct

Quartz-M1-100M-Instruct is a lightweight, ultra-fast 100M parameter language model based on the LLaMA architecture. It is designed for low-latency edge deployment, local chat, and high-throughput inference (reaching **400–485 tokens/second** on consumer GPUs like the RTX 3060).

---

## Model Files & Formats

| File | Format / Quant | Size | Recommended Use |
| :--- | :---: | :---: | :--- |
| `model.safetensors` | Safetensors (BF16/FP16) | ~202 MB | PyTorch, Hugging Face Transformers |
| `Quartz-M1-100M-Instruct-f16.gguf` | GGUF FP16 | ~202 MB | Full precision, llama.cpp, LM Studio, Ollama |
| `Quartz-M1-100M-Instruct-Q8_0.gguf` | GGUF Q8_0 | ~107 MB | Near-lossless 8-bit quantization |
| `Quartz-M1-100M-Instruct-Q4_K_M.gguf` | GGUF Q4_K_M | ~65 MB | Ultra-compact 4-bit quantization |

---

## Performance Benchmarks

* **RTX 3060 (12GB) with CUDA**:
  * **Text Generation:** ~400 – 485 Tokens/Second (TPS)
  * **Prompt Processing:** ~23,000 – 33,000 Tokens/Second
  * **VRAM Usage:** < 250 MB

---

## Quickstart

### 1. Using `llama.cpp` CLI
```bash
llama-cli -m Quartz-M1-100M-Instruct-f16.gguf -ngl 99 -p "Explain quantum computing simply." -n 128
```

### 2. Using `transformers` (Python)
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "honvert/Quartz-M1-100M-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

messages = [
    {"role": "user", "content": "What are three key habits of effective developers?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

---

## Architecture & Parameters
* **Parameters:** 101,343,744 (101M)
* **Hidden Size:** 512
* **Layers:** 24
* **Attention Heads:** 8 (KV Heads: 4 - Grouped Query Attention)
* **Context Window:** 1,024 tokens