---
library_name: transformers
pipeline_tag: text-generation
tags:
- weevil
- gdn2
- custom_code
---

# Weevil-3M

Pretrained causal language model. 2,993,576 parameters, eight parallel GDN2/SwiGLU blocks,
and a shared 4,096-token BPE tokenizer. Uses the shared Weevil Transformers implementation.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "ELBennett/Weevil-3M"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)
inputs = tokenizer("The purpose of this article is", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64, do_sample=True,
                        temperature=0.8, top_k=40)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

Requires PyTorch and Transformers (`pip install torch 'transformers>=4.57,<6'`).
Weights are safetensors; generation uses fixed-size recurrent/convolution caches.
Source checkpoint: `step-00038147`. This is a continuation model, with no chat template.