Image-Text-to-Text
MLX
Safetensors
Cosmos
cosmos3_edge
vision-language
video
apple-silicon
conversational
Instructions to use netprojects/Cosmos3-Edge-mlx-6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Cosmos3-Edge-mlx-6bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Cosmos3-Edge-mlx-6bit") config = load_config("netprojects/Cosmos3-Edge-mlx-6bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Cosmos
How to use netprojects/Cosmos3-Edge-mlx-6bit with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Cosmos3-Edge-mlx-6bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Cosmos3-Edge-mlx-6bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Cosmos3-Edge-mlx-6bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Cosmos3-Edge-mlx-6bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Cosmos3-Edge-mlx-6bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Cosmos3-Edge-mlx-6bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Cosmos3-Edge-mlx-6bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Cosmos3-Edge-mlx-6bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Cosmos3-Edge-mlx-6bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from netprojects/Cosmos3-Edge-mlx-6bit: direct link, hf CLI and curl.
- Browser
- Download file 5.97 kB
-
https://huggingface.co/netprojects/Cosmos3-Edge-mlx-6bit/resolve/main/README.md
- Command line
-
hf download hf://netprojects/Cosmos3-Edge-mlx-6bit/README.md
-
curl -L -o README.md https://huggingface.co/netprojects/Cosmos3-Edge-mlx-6bit/resolve/main/README.md
5.97 kB
| license: other | |
| license_name: openmdw1.1-license | |
| license_link: https://openmdw.ai/license/1-1/ | |
| base_model: nvidia/Cosmos3-Edge | |
| base_model_relation: quantized | |
| pipeline_tag: image-text-to-text | |
| library_name: mlx | |
| tags: | |
| - mlx | |
| - cosmos | |
| - vision-language | |
| - video | |
| - apple-silicon | |
| # Cosmos3-Edge reasoner for Apple Silicon (MLX, 6-bit) | |
| An MLX port of the reasoner in [nvidia/Cosmos3-Edge](https://huggingface.co/nvidia/Cosmos3-Edge) (revision `344d602`). It takes | |
| text, images and video and answers in text, with optional thinking. This build quantizes the language model to 6-bit | |
| (affine, group 64) and keeps the SigLIP2 vision tower and projector in bf16. Size: 2.6 GB. | |
| Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with NVIDIA. | |
| The video generator from the same checkpoint: [Cosmos3-Edge-Omni-mlx-8bit](https://huggingface.co/netprojects/Cosmos3-Edge-Omni-mlx-8bit) · [Cosmos3-Edge-Omni-mlx-bf16](https://huggingface.co/netprojects/Cosmos3-Edge-Omni-mlx-bf16). | |
| | Build | Size | Same next token as NVIDIA's reference | RealWorldQA (765) | CV-Bench (25% sample) | | |
| |---|---|---|---|---| | |
| | [netprojects/Cosmos3-Edge-mlx-8bit](https://huggingface.co/netprojects/Cosmos3-Edge-mlx-8bit) (recommended) | 3.1 GB | 98.4% | 69.2 | 87.0 | | |
| | [netprojects/Cosmos3-Edge-mlx-6bit](https://huggingface.co/netprojects/Cosmos3-Edge-mlx-6bit) | 2.6 GB | 97.7% | 68.6 | 87.3 | | |
| | bf16 (original weights, same code) | 4.9 GB | 98.7% | 68.9 | 87.2 | | |
| | NVIDIA's published scores | — | — | 70.5 | 85.3 (full set) | | |
| Measured on a MacBook Pro M5 Max (128 GB), 7 November 2026. "Same next token" compares each build with NVIDIA's model as | |
| implemented in transformers 5.22.2 (bf16, CPU) over 19 prompts (text, single and multiple images, video), 48 teacher-forced | |
| tokens each; mean KL over the top 64 tokens is 0.002 (8-bit) and 0.007 (6-bit). Benchmarks run with thinking off, greedy | |
| decoding and exact-answer scoring (letters, words and numbers); CV-Bench uses every fourth item of each source, scored the official way. NVIDIA's own | |
| evaluation settings are not published, so the scores show the port behaves like the model rather than reproduce NVIDIA's setup. | |
| ## Use | |
| ```bash | |
| git clone https://github.com/internet-projects/cosmos3-edge-mlx && cd cosmos3-edge-mlx | |
| uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt | |
| hf download netprojects/Cosmos3-Edge-mlx-6bit --local-dir cosmos3-edge-mlx-6bit | |
| # one prompt | |
| .venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-6bit --image photo.jpg --prompt "Describe the scene." | |
| .venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-6bit --video clip.mp4 --fps 4 --prompt "What happens?" | |
| # OpenAI-compatible server: /v1/chat/completions with text, image_url and video_url parts, streaming | |
| .venv/bin/python -m cosmos3_edge_mlx.server --model cosmos3-edge-mlx-6bit --port 8796 | |
| ``` | |
| Thinking is on by default, as in NVIDIA's chat template; turn it off with `--no-think` or | |
| `"chat_template_kwargs": {"enable_thinking": false}`. The server returns the thinking text as `reasoning_content`. | |
| Input preprocessing uses NVIDIA's own processor from transformers (5.22.2 or later), so image resizing, frame sampling and | |
| timestamps match the original. Videos are decoded with PyAV. | |
| Memory: the MLX buffer cache is capped (`COSMOS_CACHE_LIMIT_GB`, default 2) and cleared after each request, and | |
| `COSMOS_MEMORY_LIMIT_GB` (default 24) sets a memory limit. With the 8-bit build, a 16K-token prompt peaked at 5.5 GB and a 32K-token prompt at 7.3 GB. Long-context answers | |
| match NVIDIA's code: in a needle test up to 8K tokens both give the same answer on every prompt, and at 16K the port | |
| finds the needle on the same prompts as NVIDIA's code or more. | |
| ## How it works | |
| - **Language model:** 28-layer decoder (hidden 2048, 16 query / 8 key-value heads, ReLU² MLP, vocabulary 131,072) with | |
| interleaved 3D M-RoPE (sections 24/20/20, theta 1e8), as in NVIDIA's code. | |
| - **Vision:** SigLIP2-style tower (27 layers, patch 16) at native resolution. The learned 16×16 position grid is resized per | |
| image or frame with the same antialiased bilinear filter as the original, and attention stays inside each image or frame. | |
| A 2×2 patch merger projects to the language model. | |
| - **Video:** frames are encoded one by one with NVIDIA's timestamps between them, and positions follow NVIDIA's | |
| `get_rope_index`. | |
| - **Reasoner only:** the checkpoint also holds Cosmos3-Edge's video generator. Those tensors are not part of this build. | |
| ## Build and check it yourself | |
| ```bash | |
| hf download nvidia/Cosmos3-Edge --revision 344d602b128d1bbdacb43b08d0a3626f46343e29 --local-dir cosmos3-edge-hf \ | |
| --include "*.json" --include "*.jinja" --include "*.md" --include "LICENSE*" --include "NOTICE*" \ | |
| --include "transformer/*" --include "vision_encoder/*" | |
| .venv/bin/python convert.py --src cosmos3-edge-hf --dst cosmos3-edge-mlx-8bit --bits 8 # or --bits 6 | |
| .venv/bin/python tests/test_tiny.py # tiny random model vs transformers' Cosmos3-Edge, no real weights | |
| .venv/bin/python parity.py ref --model cosmos3-edge-hf --out ref.npz # NVIDIA's model in transformers, CPU | |
| .venv/bin/python parity.py mlx --model cosmos3-edge-mlx-8bit --ref ref.npz --out q8.json && .venv/bin/python parity.py report q8.json | |
| .venv/bin/python bench.py --model cosmos3-edge-mlx-8bit --data evaldata --out bench-q8.json | |
| ``` | |
| ## License | |
| Cosmos3-Edge is released by NVIDIA under the [OpenMDW-1.1](https://openmdw.ai/license/1-1/) license, and these conversions | |
| keep it: `LICENSE` is the OpenMDW-1.1 text and `NOTICE` describes the conversion. | |
| NVIDIA's model documents (`BIAS.md`, `EXPLAINABILITY.md`, `PRIVACY.md`, `SAFETY.md`) are included unchanged. | |
| Credits: NVIDIA (Cosmos3-Edge), Hugging Face transformers (reference implementation and processor), Apple MLX. | |
| Port by Paris Michalas, Partner at [NetProjects](https://netprojects.com). Not affiliated with NVIDIA. | |