Llama 3.2 3B Instruct (MNN)

Meta's Llama 3.2 3B Instruct as a 4-bit MNN bundle for chatting on your phone with TokForge.

Published by TokForge for on-device inference.

What it is Chat model
Format MNN bundle, 4-bit weights (block 128)
Runs on Android phones (CPU, OpenCL or Vulkan GPU); also listed in TokForge for iOS
Phone class 6 GB RAM or more (TokForge catalog)
Download 2.60 GB (10 files)
Base model meta-llama/Llama-3.2-3B-Instruct
License Llama 3.2 Community License

TokForge

Runs on-device in the TokForge app.

Use it in TokForge

  1. Install TokForge from Google Play, or join the iOS TestFlight.
  2. On Android, open Models > Recommended > Chat and download Llama 3.2 3B Instruct. TokForge lists it on phones with 6 GB RAM or more.
  3. Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.

Collections: TokForge Chat Models (MNN), TokForge Llama, Mistral and Dolphin (MNN).

Files

File Size What it is
config.json 210 B MNN runtime settings
embeddings_bf16.bin 788 MB Token embeddings (bf16)
export_args.json 1.11 KB Export settings used for the conversion
llm.mnn 489 KB Model graph
llm.mnn.json 983 KB Model graph as JSON (reference copy)
llm.mnn.weight 1.81 GB Model weights
llm_config.json 4.26 KB Model settings and chat template
tokenizer.txt 1.41 MB Tokenizer
LICENSE 7.71 KB Llama 3.2 Community License (Meta's text)
NOTICE 116 B Llama 3.2 notice (Meta)

About the model

Llama 3.2 3B Instruct is Meta's compact instruction-tuned model, built for edge and mobile use. The original model supports a 128K-token context and eight languages: English, German, French, Italian, Portuguese, Hindi, Spanish and Thai.

Architecture Llama 3.2 (standard attention, 28 layers, GQA 24Q/8KV)
Parameters 3B (4-bit quantized)
Quantization W4A16 (4-bit weights, block size 128)
Vocab 128,256 tokens

Recommended settings

Setting Value
Backend OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback)
Precision Low
Threads 4
Thinking Off

How it was made

Converted using MNN's llmexport pipeline:

python llmexport.py --path meta-llama/Llama-3.2-3B-Instruct --export mnn --quant_bit 4 --quant_block 128

Limitations

  • Intended for TokForge / MNN on-device inference
  • This is a runtime bundle, not a standard Transformers training checkpoint
  • Quantization (Q4) may slightly reduce quality compared to the full-precision original

License

Llama 3.2 is licensed under the Llama 3.2 Community License. Meta's license text is in LICENSE and the required notice is in NOTICE. Built with Llama.

Credits

  • Llama 3.2 3B by Meta (Hugging Face). Used under the Llama 3.2 Community License.
  • Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The license text and this notice are in the LICENSE and NOTICE files in this repo.
  • Converted with llmexport from MNN by Alibaba (Apache-2.0).
Downloads last month
155
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for darkmaniac7/Llama-3.2-3B-Instruct-MNN

Quantized
(561)
this model

Collections including darkmaniac7/Llama-3.2-3B-Instruct-MNN