Instructions to use carlmuck/Star-Command-R-32B-v1-mlx-4Bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use carlmuck/Star-Command-R-32B-v1-mlx-4Bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download carlmuck/Star-Command-R-32B-v1-mlx-4Bit --local-dir Star-Command-R-32B-v1-mlx-4Bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Download chat_template.jinja from carlmuck/Star-Command-R-32B-v1-mlx-4Bit: direct link, hf CLI and curl.
- Browser
- Download file 1.2 kB
-
https://huggingface.co/carlmuck/Star-Command-R-32B-v1-mlx-4Bit/resolve/main/chat_template.jinja
- Command line
-
hf download hf://carlmuck/Star-Command-R-32B-v1-mlx-4Bit/chat_template.jinja
-
curl -L -o chat_template.jinja https://huggingface.co/carlmuck/Star-Command-R-32B-v1-mlx-4Bit/resolve/main/chat_template.jinja
1.2 kB
| {{ bos_token }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% elif false == true %}{% set loop_messages = messages %}{% set system_message = 'You are Command-R, a brilliant, sophisticated, AI-assistant trained to assist human users by providing thorough responses. You are trained by Cohere.' %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% if system_message != false %}{{ '<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>' + system_message + '<|END_OF_TURN_TOKEN|>' }}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|START_OF_TURN_TOKEN|><|USER_TOKEN|>' + content.strip() + '<|END_OF_TURN_TOKEN|>' }}{% elif message['role'] == 'assistant' %}{{ '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>' + content.strip() + '<|END_OF_TURN_TOKEN|>' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>' }}{% endif %} |