File size: 2,845 Bytes
7d7ca54 bc49ab1 e5ea373 7d7ca54 bc49ab1 3ad914a 2c5b59b bc49ab1 9aec1c3 bc49ab1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
license: apache-2.0
datasets:
- HuggingFaceTB/smol-smoltalk
base_model:
- DedeProGames/DynamicMind-Mini
---
# DynamicMind-Mini-Instruct
DynamicMind-Mini-Instruct is the instruction-tuned version of [DynamicMind-Mini](https://huggingface.co/DedeProGames/DynamicMind-Mini). It was fully fine-tuned on [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) with loss applied only to assistant tokens and the assistant-ending EOS token.
The model has about 8.9M, a **1,024-token context window**, and a custom **8,192-token digit-aware byte-level BPE tokenizer**. It supports system prompts, multi-turn conversations, and KV-cached generation.
## Model Details
| Field | Value |
|---|---:|
| Parameters | 8,884,992 |
| Architecture | Custom Llama-style decoder |
| Layers | 9 |
| Hidden size | 256 |
| Intermediate size | 768 |
| Attention heads | 8 |
| KV heads | 2 |
| Vocabulary size | 8,192 |
| Context length | 1,024 |
| Embeddings | Tied input/output embeddings |
| Weight format | safetensors |
## Benchmarks
## Usage
This model uses custom architecture code and must be loaded with `trust_remote_code=True`.
```bash
pip install -U transformers safetensors torch
```
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "DedeProGames/DynamicMind-Mini-Instruct"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" and torch.cuda.is_bf16_supported() else torch.float32
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
torch_dtype=dtype,
).to(device).eval()
messages = [
{"role": "system", "content": "You are a concise and helpful assistant."},
{"role": "user", "content": "Hello!"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
)
inputs = {name: tensor.to(device) for name, tensor in inputs.items()}
with torch.inference_mode():
output = model.generate(
**inputs,
max_new_tokens=192,
do_sample=False,
repetition_penalty=1.1,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id,
use_cache=True,
)
new_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))
```
For multi-turn chat, append the generated assistant response and the next user message to `messages`, then render the chat template again. |