Instructions to use mlx-community/Qwen2-Math-72B-Instruct-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Qwen2-Math-72B-Instruct-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/Qwen2-Math-72B-Instruct-8bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/Qwen2-Math-72B-Instruct-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "mlx-community/Qwen2-Math-72B-Instruct-8bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "mlx-community/Qwen2-Math-72B-Instruct-8bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mlx-community/Qwen2-Math-72B-Instruct-8bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Atomic Chat
Commit History
-
b8d613e988a63e42a08e64e83e9c058b219ab9816d6d9aa642e36a2985537c27
Verified signaturee12dd1e -
4ea2af8def5b3450e1d1968aaa6dc74ebd544affc5649a8ea6d837029568481d
Verified signaturef43bce9 -
8fbc545c0fcf28eac2e40a631ada1ac59ef5bdbf74199a1f9f8e90a2da6c40be
Verified signature54b48d2 -
320f7eb9626de200a1f0380f767685f714b93d4f4fbf86d69ccd304d1843cf90
Verified signaturee524825 -
1a400eaa4b3cf8946d54b58345586c43d84d432ce999feac84f0fa639436902c
Verified signature1886640 -
8c9cb713310efd6d5cf5af31f8e02085fbb38df310e0726bb701aa81899a6b5d
Verified signature5d78aca -
74b15acf354c2ed5ce75dce9de1d085955c115639d00134e1de908bf4238fa60
Verified signaturef2c2200 -
182970dd5abe4fe350cb750846cf43694dad335a8306136c4b69820569b7a18b
Verified signaturee813a37 -
b69cf8822739a60f961499bf52d3ec4d9bb6ed8cba9cc27d324942ff71eef3ad
Verified signature38e9b1b -
1ec52663ba922c6eb8602b914219a67d02a9099d7ff52f01528f0d4f80e5e0f5
Verified signature22d4160 -
5a57103480713e8a889e4ea485ffdb1cf7ac5c2845add3077296bbde5e4c0709
Verified signaturecc86207 -
b490dcd25219f9899dac5b26d4f124b13e7abafa0e83f5126a8e742ba08333f8
Verified signaturefb5a259 -
fc85e5fd4808c432af7bd75ae8e72fc7c97b024a5d22a2e932ed7bf7be0d8b7f
Verified signaturec56638f -
f134b89daea17165299704d1f44b6fec5d7c3a93fcc163c72a402acdae915afa
Verified signature123cb60 -
edfec6ed588944c71450f7b524a0768f75e70be6216da936a7500a43d0509af7
Verified signatureb471895 -
67d3b889ec1cb557b2972d1c0075fd1602a7b75dba2ba90f5108b2d714639886
Verified signature61c0e14 -
initial commit
Verified signaturec049a3b