Instructions to use ForgeWorks/ForgePlex-M2-9M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ForgeWorks/ForgePlex-M2-9M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ForgeWorks/ForgePlex-M2-9M", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ForgeWorks/ForgePlex-M2-9M", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ForgeWorks/ForgePlex-M2-9M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ForgeWorks/ForgePlex-M2-9M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ForgeWorks/ForgePlex-M2-9M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ForgeWorks/ForgePlex-M2-9M

SGLang

How to use ForgeWorks/ForgePlex-M2-9M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ForgeWorks/ForgePlex-M2-9M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ForgeWorks/ForgePlex-M2-9M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ForgeWorks/ForgePlex-M2-9M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ForgeWorks/ForgePlex-M2-9M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ForgeWorks/ForgePlex-M2-9M with Docker Model Runner:
```
docker model run hf.co/ForgeWorks/ForgePlex-M2-9M
```

ForgePlex-M2-9M / README.md

Jsandero

Update README.md

3cc1d37 verified 9 days ago

preview code

Raw History Blame Contribute Delete

2.74 kB

	---
	license: apache-2.0
	language:
	- en
	library_name: transformers
	tags:
	- language-model
	- forgeplex
	- forgeworks
	- rope
	- swiglu
	- gqa
	- attn-output-gate
	- refresh-gate
	- custom_code
	pipeline_tag: text-generation
	---

	We would like to thank [Axiomic Labs](https://huggingface.co/AxiomicLabs) for allowing us to use their TrainWork framework to train this model.

	# ForgePlex-M2-9M

	ForgePlex-M2-9M is a ~9.95M-parameter decoder-only language model from ForgeWorks. Trained on 30B tokens. Our second attempt at creating a <10m parameter model. We're proud of this product, while M1 was a promising start, M2 shows what we can do.

	![ForgePlex-M2-9M architecture](M2-Complete.jpg)

	## Q&A

	What was the motivation behind M2?

	"That is an excellent question. To be completely honest. George Mallory was asked why he wanted to climb Everest. He said, “Because it’s there.” I just see M2 as a mountain to climb"

	\| Metric \| Value \|
	\|---\|---\|
	\| Unique parameters \| 9,949,698 \|
	\| Intelligence Index \| 9.14 \|
	\| HellaSwag \| 28.02% \|
	\| ARC (combined) \| 30.00% \|
	\| PIQA \| 57.18% \|
	\| ArithMark-3 \| 34.60% \|
	\| License \| Apache-2.0 \|

	## Training

	Trained on an all new dataset

	\| Data \| Percentage \|
	\|---\|---\|
	\| Finephrase \| 50% \|
	\| DCLM - Baseline \| 30% \|
	\| Proprietary Axiomic Labs dataset releasing soon \| 10% \|
	\| Finemath \| 10% \|

	## Architecture

	GQA + NeoX-style RoPE + RMSNorm + SwiGLU, with Qwen3.5-style attention output gates and Axiomic Labs TX4 style refresh gates on inject layers `[5, 10]` (kernel 9). XSA is off. Weights keep training key layout (no Llama remapping).

	\| Component \| Details \|
	\|---\|---\|
	\| Position encoding \| RoPE (theta=5,000, NeoX even/odd) \|
	\| Normalization \| RMSNorm (eps=1e-6) \|
	\| Feed-forward \| SwiGLU (intermediate 707) \|
	\| Attention \| GQA — 8Q / 2KV, head_dim=32 + attn output gate \|
	\| Refresh \| Layers 5, 10, kernel 9 \|
	\| Bias \| None \|
	\| Embedding \| Weight tying \|
	\| Depth × width \| 11 layers × 256 hidden \|
	\| Context \| 1024 tokens \|
	\| Vocab \| 4,096 custom BPE \|

	## Usage

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_path = r"C:\slm\ForgePlex-M2-9M"
	tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained(
	model_path,
	trust_remote_code=True,
	torch_dtype=torch.float32,
	device_map="auto",
	)

	prompt = "Once upon a time"
	inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
	with torch.inference_mode():
	out = model.generate(**inputs, max_new_tokens=80, do_sample=False)
	print(tokenizer.decode(out[0], skip_special_tokens=True))
	```

	Or run `python usage.py` from this folder.