Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

DeepSeek-R1-Distill-Llama-8B-MNN

Pre-converted DeepSeek R1 Distill Llama 8B in MNN format for on-device inference with TokForge.

Original model by DeepSeek, converted to MNN Q4 for mobile deployment.

Model Details

Architecture	Llama 3.1 (standard attention, 32 layers, GQA, distilled from DeepSeek-R1)
Parameters	8B (4-bit quantized)
Format	MNN (Alibaba Mobile Neural Network)
Quantization	W4A16 (4-bit weights, block size 128)
Vocab	128,256 tokens
Source	deepseek-ai/DeepSeek-R1-Distill-Llama-8B

Description

DeepSeek's R1 reasoning capability distilled into a Llama 3.1 8B body. Brings chain-of-thought reasoning to mobile devices. Shows its thinking process step-by-step, making it excellent for math, logic puzzles, coding, and complex analysis.

Files

File	Description
llm.mnn	Model computation graph
llm.mnn.weight	Quantized weight data (Q4, block=128)
llm_config.json	Model config with Jinja chat template
tokenizer.txt	Tokenizer vocabulary
config.json	MNN runtime config

Usage with TokForge

This model is optimized for TokForge, a free Android app for private, on-device LLM inference.

1. Download TokForge from Google Play
2. Open the app → Models → Download this model
3. Start chatting: runs 100% locally, no internet required

Recommended Settings

Setting	Value
Backend	OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback)
Precision	Low
Threads	4
Thinking	Off (or On for thinking-capable models)

Credits

• DeepSeek-R1-Distill-Llama-8B by DeepSeek (Hugging Face). Used under the MIT license.
• Llama 3.1 8B by Meta (Hugging Face). Used under the Llama 3.1 Community License. The distilled weights derive from it.
• Built with Llama. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The license text and this notice are in the LICENSE and NOTICE files in this repo.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).

Limitations

• Intended for TokForge / MNN on-device inference on Android
• This is a runtime bundle, not a standard Transformers training checkpoint
• Quantization (Q4) may slightly reduce quality compared to the full-precision original

Community

• Website: tokforge.ai
• Discord: Join our Discord

Export Details

Converted using MNN's llmexport pipeline:

python llmexport.py --path deepseek-ai/DeepSeek-R1-Distill-Llama-8B --export mnn --quant_bit 4 --quant_block 128