Instructions to use mlx-community/Qwen2-Math-72B-Instruct-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Qwen2-Math-72B-Instruct-bf16 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/Qwen2-Math-72B-Instruct-bf16") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/Qwen2-Math-72B-Instruct-bf16 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "mlx-community/Qwen2-Math-72B-Instruct-bf16"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "mlx-community/Qwen2-Math-72B-Instruct-bf16" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mlx-community/Qwen2-Math-72B-Instruct-bf16", "messages": [ {"role": "user", "content": "Hello"} ] }' - Atomic Chat
Commit History
-
Upload folder using huggingface_hub (#2)
Verified signature- f9b9cf17687e4b0215a5f3406280deb961c64a05586ba057925b0bf5e12e12d1 (2ad59dc2558c7f75e697c6b871d32d39c4edbc86) - 12e092bd8f49fc38e8d189dd8d21bd9eca2ca74d52fc0bfe11adfa5abf171b5b (a2d641f7f9b125eaa2efc7df1ea6076639d2b443) - 74de202e71f71406af7907d780687b1825b67d448dc18b50f8bb4a628358b0dc (5167f40381f0a3d3ad608bf0a056e74f2939cece) - 4e888c52b978d4fd488c5f07afb90a4c1e99b35cbd80aabecc93557fe402edd2 (8cb0ad963c5d13cd3007c88073c42fd261643848) - 84ab543f4845fdaa7b82382711760062dfcab5b0ff6dddd63e5191c586bbc24d (9613717f755f70b8bb5c030e9681b16cce620fe9) - 99127673e188e50e7be083c557e558deef21927f3cb71631b4f2de499441f76f (f24eebdcf518b0f1d54b6b5a0463fbfa4aa5951b) - 9e0e54e49a7e7006eeeb65120cfcf5d0e3c6a713ac2ee2b260fc24e154f277cf (bde24bfe4bebe4b2717ed9aef6767baab9b216e5) - c2fbe2a3471944d1dfdaf5454c6534a9fab452a5fe86685ed5f39bb870d1d7f4 (2f3d287a0782af287681c9f24adc8710165096be) - dd037cff1cba466f0b7dd32c22451a9b837c30da474f5078686c8d10604b5e59 (036a399fc952ff0d5f4ed21730bf1faf50c1a822) - 4290a783…3f2d68f -
initial commit
Verified signature1ff0544