Image-Text-to-Text
MLX
English
qwen3_5
native-mlx
apple-silicon
fp32
ct
medical-imaging
nv-reason-ct
3d-vlm
vision-language
radiology
report-generation
conversational
custom_code
Instructions to use MedMLX/MLX-Reason-CT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use MedMLX/MLX-Reason-CT with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("MedMLX/MLX-Reason-CT") config = load_config("MedMLX/MLX-Reason-CT") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use MedMLX/MLX-Reason-CT with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "MedMLX/MLX-Reason-CT" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use MedMLX/MLX-Reason-CT with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default MedMLX/MLX-Reason-CT
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use MedMLX/MLX-Reason-CT with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "MedMLX/MLX-Reason-CT" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download native-run.json from MedMLX/MLX-Reason-CT: direct link, hf CLI and curl.
- Browser
- Download file 5.51 kB
-
https://huggingface.co/MedMLX/MLX-Reason-CT/resolve/main/native-run.json
- Command line
-
hf download hf://MedMLX/MLX-Reason-CT/native-run.json
-
curl -L -o native-run.json https://huggingface.co/MedMLX/MLX-Reason-CT/resolve/main/native-run.json
5.51 kB
| { | |
| "schema_version": "medmlx.run.v1", | |
| "model": "nv-reason-ct", | |
| "configuration": "NV-Reason-CT 386b93e0, FP32, chest structured report, greedy, max_new_tokens 512, thinking off", | |
| "date": "2026-10-08", | |
| "package": { | |
| "name": "MLX-Reason-CT", | |
| "path": "<repositories>/MLX-Reason-CT", | |
| "git_revision": "eb2c10baf688b32eee082dbb8973e88b4555db7a", | |
| "branch": "main", | |
| "dirty": false, | |
| "remote": "https://github.com/MedMLX/MLX-Reason-CT.git" | |
| }, | |
| "environment": { | |
| "machine": "MacBook Pro (2021), Apple M1 Max, 10-core CPU, 24-core GPU, 64 GB", | |
| "macos": "27.0", | |
| "python": "3.12.15", | |
| "mlx_version": "0.32.3", | |
| "device": "gpu" | |
| }, | |
| "command": { | |
| "entry_point": "mlx_reason_ct.medmlx:run", | |
| "kwargs": { | |
| "image_path": "<paper-repo>/work/runs/inputs/luna-case.nii.gz", | |
| "weights_path": "<paper-repo>/work/runs/weights/nv-reason-ct", | |
| "output_dir": "<paper-repo>/work/runs/nv-reason-ct/out", | |
| "anatomy_region": "chest", | |
| "overwrite": true | |
| }, | |
| "invocation": "cd <repositories>/MLX-Reason-CT && .venv/bin/python <paper-repo>/work/runs/_harness/run_one.py mlx_reason_ct.medmlx <kwargs.json> <run_result.json>", | |
| "process": "fresh Python process per model" | |
| }, | |
| "checkpoint": { | |
| "path": "<paper-repo>/work/runs/weights/nv-reason-ct", | |
| "manifest_sha256": "42495083b60360358b1599687d3ace2f199ac164efe3efe3f884389c793e1b2b", | |
| "source_path": "<repositories>/radnn/checkpoints/nv-reason-ct/386b93e034983f6c1fc841a43833a1b6a0cd9c13", | |
| "source_sha256": "655c15e59dfa1957f7d5f475b1702d689fb7107160f21256028391bb5f6e6ba5", | |
| "registry_sha256_match": true, | |
| "conversion": "mlx-reason-ct convert --source-dir <pinned snapshot> --output-dir work/runs/weights/nv-reason-ct (lossless BF16->FP32); the pre-existing radnn conversion was rejected by the package (engine radnn.nv_reason_ct_mlx.v1 differs from the pinned nv_reason_ct_mlx.fp32.v1)" | |
| }, | |
| "inputs": [ | |
| { | |
| "role": "image", | |
| "path": "<paper-repo>/work/runs/inputs/luna-case.nii.gz", | |
| "sha256": "5a7c3a4e681dcba5585bb49e2f355cbc08311b2944614003966409f9b6a71dfe", | |
| "dataset": "LUNA16 subset0", | |
| "case_id": "1.3.6.1.4.1.14519.5.2.1.6279.6001.105756658031515062000744821260", | |
| "derivation": "MHD to NIfTI format conversion; HU and geometry preserved", | |
| "annotations": "no accepted LUNA16 nodule annotation" | |
| } | |
| ], | |
| "outputs": { | |
| "report": { | |
| "path": "<paper-repo>/work/runs/nv-reason-ct/out/report.txt", | |
| "sha256": "06d5943d8465cbda9743273566eb8ac802ebf44e0621f08a83f7eb78977bc282" | |
| }, | |
| "response": { | |
| "path": "<paper-repo>/work/runs/nv-reason-ct/out/model_response.json", | |
| "sha256": "8b49949cef1c73823a9d6ef73568523acace72030f387475a0571ccba07b3da9" | |
| }, | |
| "run": { | |
| "path": "<paper-repo>/work/runs/nv-reason-ct/out/run.json", | |
| "sha256": "a9af3ecaf7d50ee2669a10699a773bc96274cef4aa073885eb6fcf2ad6b20fb9" | |
| }, | |
| "report_text": "TECHNIQUE: Non-contrast CT.\n\nFINDINGS:\nMedical devices: None present.\n\nCHEST:\nLungs and airways: Trachea and main bronchi are patent. No active infiltration or mass lesion. A few tiny nonspecific nodules.\nPleura: No pleural effusion, pleural thickening, or pneumothorax.\nMediastinum and hila: No mediastinal lymphadenopathy or mass.\nHeart and pericardium: Normal heart size. No pericardial effusion.\nGreat vessels: Normal caliber thoracic aorta and main pulmonary artery.\nEsophagus: No pathological esophageal wall thickening.\nChest wall and breast: No chest wall or breast mass.\nBones: No lytic or destructive thoracic osseous lesion or acute fracture.\n\nIMPRESSION:\n\n1. A few tiny nonspecific pulmonary nodules.\n<answer> Lung nodule </answer>" | |
| }, | |
| "checks": [ | |
| { | |
| "name": "report_nonempty", | |
| "result": "pass", | |
| "detail": { | |
| "words": 103, | |
| "generated_tokens": 208 | |
| } | |
| }, | |
| { | |
| "name": "terminated_by_eos_not_truncated", | |
| "result": "pass", | |
| "detail": { | |
| "terminated_by_eos": true, | |
| "truncated_by_max_new_tokens": false, | |
| "prompt_tokens": 13844 | |
| } | |
| }, | |
| { | |
| "name": "structured_sections_present", | |
| "result": "pass", | |
| "detail": "TECHNIQUE / FINDINGS / IMPRESSION with organ subsections" | |
| }, | |
| { | |
| "name": "coherent_english_no_degenerate_repetition", | |
| "result": "pass", | |
| "detail": "no repeated lines" | |
| }, | |
| { | |
| "name": "json_outputs_reopen", | |
| "result": "pass", | |
| "detail": null | |
| }, | |
| { | |
| "name": "bundle_provenance_pinned", | |
| "result": "pass", | |
| "detail": null | |
| } | |
| ], | |
| "duration_s": 82.591, | |
| "timing_boundary": "wall clock around the single run() call: weight loading, preprocessing, MLX inference and export; interpreter start-up and package import excluded; one run, no warmup", | |
| "mlx_peak_gb": 22.746, | |
| "memory_definition": "mx.get_peak_memory() after mx.reset_peak_memory() immediately before run(); decimal GB", | |
| "panel": "paper/figures/runs/nv-reason-ct.png", | |
| "status": "success", | |
| "failure_cause": null, | |
| "notes": "Report content is not evaluated for accuracy. The case has no accepted LUNA16 nodule annotation; the report's 'a few tiny nonspecific nodules' is not verified.", | |
| "key_check": "208 tokens, EOS-terminated structured chest report", | |
| "first_attempt": "The first call used the radnn checkpoint store's older conversion and failed with AssetNotReadyError (cache provenance differs from the pinned conversion); reconverted with the package CLI and rerun." | |
| } | |