Image-Text-to-Text
MLX
English
qwen3_5
native-mlx
apple-silicon
fp32
ct
medical-imaging
nv-reason-ct
3d-vlm
vision-language
radiology
report-generation
conversational
custom_code
Instructions to use MedMLX/MLX-Reason-CT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use MedMLX/MLX-Reason-CT with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("MedMLX/MLX-Reason-CT") config = load_config("MedMLX/MLX-Reason-CT") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use MedMLX/MLX-Reason-CT with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "MedMLX/MLX-Reason-CT" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use MedMLX/MLX-Reason-CT with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default MedMLX/MLX-Reason-CT
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use MedMLX/MLX-Reason-CT with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "MedMLX/MLX-Reason-CT"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "MedMLX/MLX-Reason-CT" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 5,514 Bytes
56baf43 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | {
"schema_version": "medmlx.run.v1",
"model": "nv-reason-ct",
"configuration": "NV-Reason-CT 386b93e0, FP32, chest structured report, greedy, max_new_tokens 512, thinking off",
"date": "2026-10-08",
"package": {
"name": "MLX-Reason-CT",
"path": "<repositories>/MLX-Reason-CT",
"git_revision": "eb2c10baf688b32eee082dbb8973e88b4555db7a",
"branch": "main",
"dirty": false,
"remote": "https://github.com/MedMLX/MLX-Reason-CT.git"
},
"environment": {
"machine": "MacBook Pro (2021), Apple M1 Max, 10-core CPU, 24-core GPU, 64 GB",
"macos": "27.0",
"python": "3.12.15",
"mlx_version": "0.32.3",
"device": "gpu"
},
"command": {
"entry_point": "mlx_reason_ct.medmlx:run",
"kwargs": {
"image_path": "<paper-repo>/work/runs/inputs/luna-case.nii.gz",
"weights_path": "<paper-repo>/work/runs/weights/nv-reason-ct",
"output_dir": "<paper-repo>/work/runs/nv-reason-ct/out",
"anatomy_region": "chest",
"overwrite": true
},
"invocation": "cd <repositories>/MLX-Reason-CT && .venv/bin/python <paper-repo>/work/runs/_harness/run_one.py mlx_reason_ct.medmlx <kwargs.json> <run_result.json>",
"process": "fresh Python process per model"
},
"checkpoint": {
"path": "<paper-repo>/work/runs/weights/nv-reason-ct",
"manifest_sha256": "42495083b60360358b1599687d3ace2f199ac164efe3efe3f884389c793e1b2b",
"source_path": "<repositories>/radnn/checkpoints/nv-reason-ct/386b93e034983f6c1fc841a43833a1b6a0cd9c13",
"source_sha256": "655c15e59dfa1957f7d5f475b1702d689fb7107160f21256028391bb5f6e6ba5",
"registry_sha256_match": true,
"conversion": "mlx-reason-ct convert --source-dir <pinned snapshot> --output-dir work/runs/weights/nv-reason-ct (lossless BF16->FP32); the pre-existing radnn conversion was rejected by the package (engine radnn.nv_reason_ct_mlx.v1 differs from the pinned nv_reason_ct_mlx.fp32.v1)"
},
"inputs": [
{
"role": "image",
"path": "<paper-repo>/work/runs/inputs/luna-case.nii.gz",
"sha256": "5a7c3a4e681dcba5585bb49e2f355cbc08311b2944614003966409f9b6a71dfe",
"dataset": "LUNA16 subset0",
"case_id": "1.3.6.1.4.1.14519.5.2.1.6279.6001.105756658031515062000744821260",
"derivation": "MHD to NIfTI format conversion; HU and geometry preserved",
"annotations": "no accepted LUNA16 nodule annotation"
}
],
"outputs": {
"report": {
"path": "<paper-repo>/work/runs/nv-reason-ct/out/report.txt",
"sha256": "06d5943d8465cbda9743273566eb8ac802ebf44e0621f08a83f7eb78977bc282"
},
"response": {
"path": "<paper-repo>/work/runs/nv-reason-ct/out/model_response.json",
"sha256": "8b49949cef1c73823a9d6ef73568523acace72030f387475a0571ccba07b3da9"
},
"run": {
"path": "<paper-repo>/work/runs/nv-reason-ct/out/run.json",
"sha256": "a9af3ecaf7d50ee2669a10699a773bc96274cef4aa073885eb6fcf2ad6b20fb9"
},
"report_text": "TECHNIQUE: Non-contrast CT.\n\nFINDINGS:\nMedical devices: None present.\n\nCHEST:\nLungs and airways: Trachea and main bronchi are patent. No active infiltration or mass lesion. A few tiny nonspecific nodules.\nPleura: No pleural effusion, pleural thickening, or pneumothorax.\nMediastinum and hila: No mediastinal lymphadenopathy or mass.\nHeart and pericardium: Normal heart size. No pericardial effusion.\nGreat vessels: Normal caliber thoracic aorta and main pulmonary artery.\nEsophagus: No pathological esophageal wall thickening.\nChest wall and breast: No chest wall or breast mass.\nBones: No lytic or destructive thoracic osseous lesion or acute fracture.\n\nIMPRESSION:\n\n1. A few tiny nonspecific pulmonary nodules.\n<answer> Lung nodule </answer>"
},
"checks": [
{
"name": "report_nonempty",
"result": "pass",
"detail": {
"words": 103,
"generated_tokens": 208
}
},
{
"name": "terminated_by_eos_not_truncated",
"result": "pass",
"detail": {
"terminated_by_eos": true,
"truncated_by_max_new_tokens": false,
"prompt_tokens": 13844
}
},
{
"name": "structured_sections_present",
"result": "pass",
"detail": "TECHNIQUE / FINDINGS / IMPRESSION with organ subsections"
},
{
"name": "coherent_english_no_degenerate_repetition",
"result": "pass",
"detail": "no repeated lines"
},
{
"name": "json_outputs_reopen",
"result": "pass",
"detail": null
},
{
"name": "bundle_provenance_pinned",
"result": "pass",
"detail": null
}
],
"duration_s": 82.591,
"timing_boundary": "wall clock around the single run() call: weight loading, preprocessing, MLX inference and export; interpreter start-up and package import excluded; one run, no warmup",
"mlx_peak_gb": 22.746,
"memory_definition": "mx.get_peak_memory() after mx.reset_peak_memory() immediately before run(); decimal GB",
"panel": "paper/figures/runs/nv-reason-ct.png",
"status": "success",
"failure_cause": null,
"notes": "Report content is not evaluated for accuracy. The case has no accepted LUNA16 nodule annotation; the report's 'a few tiny nonspecific nodules' is not verified.",
"key_check": "208 tokens, EOS-terminated structured chest report",
"first_attempt": "The first call used the radnn checkpoint store's older conversion and failed with AssetNotReadyError (cache provenance differs from the pinned conversion); reconverted with the package CLI and rerun."
}
|