Instructions to use mlx-community/Qwen2-72B-Instruct-unquantized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Qwen2-72B-Instruct-unquantized with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/Qwen2-72B-Instruct-unquantized") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/Qwen2-72B-Instruct-unquantized with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "mlx-community/Qwen2-72B-Instruct-unquantized"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "mlx-community/Qwen2-72B-Instruct-unquantized" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mlx-community/Qwen2-72B-Instruct-unquantized", "messages": [ {"role": "user", "content": "Hello"} ] }' - Atomic Chat
Commit History
-
Upload folder using huggingface_hub (#1)
Verified signature- 0397ffffb2e679aa4523477454161b24006913f6064327c6280e1a5b9b71a6ac (6f6131fd418ed07e69a921a3c26c7b090f98f2f7) - f501d72d5946d1912e22f3d73f8a88979871b2c9d81197a192ee4b5c2b0537e5 (7c30422e8334672faf693627d9779176b7c3b01d) - 284c40322cb9f003b2a4ffb694e17c19969264a226c27ea60e58fd53d65db2e8 (536cc4b0ebd27cbd54723099b3ccca14967f515b) - 43be37a76e95db330d8fced5915c38799bc4ad9d849f1fadf3359d0ad7832217 (88a8c270566c698a14e86cacc62d021a59b6d394) - 039a446a427dedbf392b18b1b44ae2e4361567ea95a6fce622317e9909d9adfb (f6f19398540fbd33093bd4b7552dd7a217fa7f3d) - 46614c00984f876998aaba8da59e0a0507925a984997ae97efc9979870e81427 (ecb816e590b9448ee43a54f3945d96f1038f5df9) - 4cd7ef8b35c93976acc046746f23851e33e69fa73d1500e5c925a4efbf728559 (0aabac346ac03884e029eb59b5d7738e813097c4) - fde5811e53922afd8c7823dcbf7c5be87c369a1abda073f29284ac0ca81db45a (d664359726f9b5d11ecccd9ab4214e7aa4c991d0) - eb14427874108790655041273409357e44e01c228e4dbc322148d518740de878 (6d053f87c4745308978a18a1877c696afd21b1ed) - e34f5871…228af64