Text Generation
MLX
Safetensors
llama
code-search
tool-use
apple-silicon
conversational
8-bit precision
Instructions to use mlx-community/ColGrep-Agent-2B-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/ColGrep-Agent-2B-MLX-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/ColGrep-Agent-2B-MLX-8bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use mlx-community/ColGrep-Agent-2B-MLX-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/ColGrep-Agent-2B-MLX-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "mlx-community/ColGrep-Agent-2B-MLX-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use mlx-community/ColGrep-Agent-2B-MLX-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "mlx-community/ColGrep-Agent-2B-MLX-8bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "mlx-community/ColGrep-Agent-2B-MLX-8bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mlx-community/ColGrep-Agent-2B-MLX-8bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use mlx-community/ColGrep-Agent-2B-MLX-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/ColGrep-Agent-2B-MLX-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default mlx-community/ColGrep-Agent-2B-MLX-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use mlx-community/ColGrep-Agent-2B-MLX-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "mlx-community/ColGrep-Agent-2B-MLX-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "mlx-community/ColGrep-Agent-2B-MLX-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from mlx-community/ColGrep-Agent-2B-MLX-8bit: direct link, hf CLI and curl.
- Browser
- Download file 1.73 kB
-
https://huggingface.co/mlx-community/ColGrep-Agent-2B-MLX-8bit/resolve/main/README.md
- Command line
-
hf download hf://mlx-community/ColGrep-Agent-2B-MLX-8bit/README.md
-
curl -L -o README.md https://huggingface.co/mlx-community/ColGrep-Agent-2B-MLX-8bit/resolve/main/README.md
1.73 kB
| license: apache-2.0 | |
| library_name: mlx | |
| pipeline_tag: text-generation | |
| base_model: lightonai/colgrep-agent-2B | |
| tags: | |
| - mlx | |
| - code-search | |
| - tool-use | |
| - apple-silicon | |
| # ColGrep-Agent-2B - MLX 8BIT | |
| Native Apple Silicon MLX conversion of [LightOn's ColGrep-Agent-2B](https://huggingface.co/lightonai/colgrep-agent-2B), a specialized repository code-localization agent built on MiniCPM5-2B. | |
| - Upstream checkpoint: `lightonai/colgrep-agent-2B` at `f798a61129c6decb5166864e0a3c98373e62c764` | |
| - Conversion: MLX-LM 0.32.0, 8-bit affine, group size 64 | |
| - Compatibility: source `chat_template.jinja` preserved; top-level `rope_theta=5000000` added so MLX-LM's Llama RoPE implementation matches the source configuration. | |
| - Original model authors: LightOn AI; original model license: Apache-2.0. | |
| ## Usage | |
| ```bash | |
| pip install 'mlx-lm==0.32.0' 'transformers>=5.6.2,<6' | |
| mlx_lm.generate --model . --prompt 'Hello' --max-tokens 128 | |
| ``` | |
| **This is not a general coding assistant.** It was trained for iterative code search with `colgrep`, `terminal`, and `finish`. Use the upstream chat template, system prompt, tool schemas, and thinking disabled. The official `colgrep --agent` CLI currently runs GGUF through llama.cpp; this MLX checkpoint is **not** a drop-in replacement for that CLI. An MLX-backed harness is separate integration work. | |
| This checkpoint is not claimed to match upstream BF16 on benchmarks until an apples-to-apples evaluation of full search trajectories is reported. The smoke test validates basic load, format, and first tool-call structure only. | |
| - Upstream blog: https://huggingface.co/blog/lightonai/colgrep-agent | |
| - Upstream harness: https://github.com/lightonai/next-plaid | |
| - MLX-LM: https://github.com/ml-explore/mlx-lm | |