Instructions to use rbizzoni/Velvet2-3B-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use rbizzoni/Velvet2-3B-MLX-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("rbizzoni/Velvet2-3B-MLX-8bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use rbizzoni/Velvet2-3B-MLX-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rbizzoni/Velvet2-3B-MLX-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "rbizzoni/Velvet2-3B-MLX-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use rbizzoni/Velvet2-3B-MLX-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "rbizzoni/Velvet2-3B-MLX-8bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "rbizzoni/Velvet2-3B-MLX-8bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "rbizzoni/Velvet2-3B-MLX-8bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use rbizzoni/Velvet2-3B-MLX-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rbizzoni/Velvet2-3B-MLX-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default rbizzoni/Velvet2-3B-MLX-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use rbizzoni/Velvet2-3B-MLX-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rbizzoni/Velvet2-3B-MLX-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "rbizzoni/Velvet2-3B-MLX-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Velvet2-3B-MLX-8bit
๐ฎ๐น Conversione non ufficiale in formato MLX (quantizzazione a 8 bit) di AlmawaveLabs/Velvet2-3B, per l'uso su Mac con Apple Silicon tramite
mlx-lmo LM Studio.๐ฌ๐ง Unofficial 8-bit MLX conversion of AlmawaveLabs/Velvet2-3B, for Apple Silicon Macs via
mlx-lmor LM Studio.
This repository is not affiliated with or endorsed by Almawave. All credit for the model goes to the original authors. For the model description, training details, evaluations and intended use, please refer to the original model card.
Conversion details
| Original model | AlmawaveLabs/Velvet2-3B (BF16 safetensors) |
| Original revision | f36ebe942894130d86199d3abe70d0f7a7a4d446 |
| Format | MLX |
| Quantization | 8-bit (mlx-lm default group size) |
| Tool | mlx-lm |
| Converted on | Apple MacBook (M4) |
Command used:
mlx_lm.convert \
--hf-path AlmawaveLabs/Velvet2-3B \
--mlx-path Velvet2-3B-MLX-8bit \
-q --q-bits 8
No fine-tuning or other changes were made to the weights beyond quantization. Tokenizer and chat template are copied unchanged from the original repository.
Usage
LM Studio
Search for rbizzoni/Velvet2-3B-MLX-8bit in LM Studio's model search and download it (requires the MLX runtime on Apple Silicon).
mlx-lm
pip install -U mlx-lm
mlx_lm.generate \
--model rbizzoni/Velvet2-3B-MLX-8bit \
--prompt "Spiegami in tre frasi cos'รจ la Costituzione italiana."
from mlx_lm import load, generate
model, tokenizer = load("rbizzoni/Velvet2-3B-MLX-8bit")
messages = [
{"role": "system", "content": "You are Velvet2-3B, an LLM created by Almawave Labs."},
{"role": "user", "content": "Qual รจ la capitale d'Italia?"},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True))
Notes
- Tokenizer warning: when loading,
transformersmay print a warning about an "incorrect regex pattern" and suggestfix_mistral_regex=True. This check targets an issue in some Mistral tokenizers. Velvet2 uses its own tokenizer (from Velvet-14B), so the warning is likely a false positive. The tokenizer was not modified in this conversion. - Tool calling: the original model provides a dedicated tool-calling template and parser for vLLM (
tool_chat_template.jinja,tool_parser_plugin.py). These are not used by mlx-lm or LM Studio, so tool calling may not work as documented in the original card. - Quantization can slightly reduce output quality compared to the original BF16 weights.
License and usage restrictions
This conversion is distributed under the same license as the original model (Apache 2.0). The original model card also lists usage restrictions and governance terms, including limitations on commercial and institutional use and on processing sensitive healthcare, financial or government data without appropriate safeguards. These terms apply to this conversion as well. Please read them on the original model page before use.
Credits
- Original model: Almawave / AlmawaveLabs
- Conversion: rbizzoni, using mlx-lm
- Downloads last month
- 43
8-bit
Model tree for rbizzoni/Velvet2-3B-MLX-8bit
Base model
AlmawaveLabs/Velvet2-3B