josand commited on
Commit
d79006c
·
verified ·
1 Parent(s): b25ef23

Add native MLX model card and conversion guide

Browse files
Files changed (1) hide show
  1. README.md +119 -0
README.md ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: mlx
3
+ license: other
4
+ tags:
5
+ - medical-imaging
6
+ - apple-silicon
7
+ - research
8
+ - inference
9
+ - conversion
10
+ - model-card-only
11
+ language:
12
+ - en
13
+ license_name: upstream-component-terms
14
+ license_link: https://github.com/HengruiTianSJTU/LesionDiffusion
15
+ pipeline_tag: image-to-image
16
+ ---
17
+
18
+ # LesionDiffusion for MLX
19
+
20
+ Native MLX LesionDiffusion synthesis on Apple Silicon using LMNet, LINet, and VQ components. The recorded paper configuration inserts a liver tumor into a prepared CT crop; current conditioned inference is documented separately below.
21
+
22
+ **Repository contents:** model card and local conversion instructions. No checkpoint, tokenizer, medical image, or output data is included. Obtain original assets directly from their publishers under the applicable terms.
23
+
24
+ Requires an Apple Silicon Mac with Metal and Python 3.12/3.13. Model inference uses MLX; PyTorch is used for one-time checkpoint conversion and optional upstream comparisons. Intended for research and education; clinical diagnostic performance has not been established.
25
+
26
+ Native MLX package: [lesiondiffusion-mlx](https://github.com/MedMLX/lesiondiffusion-mlx).
27
+ Evaluated revision: `9b7a15eef808a38ab56e82325c22f209f69b9b78`.
28
+
29
+ This card describes the one configuration evaluated in the MedMLX paper. It is research software; the single-case comparison does not estimate clinical accuracy.
30
+
31
+ ## Source and checkpoint
32
+
33
+ Original publisher: HengruiTianSJTU.
34
+ Original implementation: [https://github.com/HengruiTianSJTU/LesionDiffusion](https://github.com/HengruiTianSJTU/LesionDiffusion).
35
+ Source revision or version: `880736e2b8a4d9becdb6594da7f8af6db19d04d1`.
36
+ Original checkpoint: Component locations are in the native run record and the upstream package documentation.
37
+
38
+ Conversion: lesiondiffusion_mlx.write_conversion (README recipe), using the upstream component checkpoints and an explicitly supplied BiomedCLIP tokenizer.
39
+ Conversion changes tensor layout or storage as specified by the native package; it does not retrain the model.
40
+
41
+ ## Install and convert
42
+
43
+ Install the native package with conversion support:
44
+
45
+ ```bash
46
+ uv pip install "lesiondiffusion-mlx[conversion] @ git+https://github.com/MedMLX/lesiondiffusion-mlx.git"
47
+ ```
48
+
49
+ Obtain the LMNet, LINet, and VQ checkpoints through the [official upstream project](https://github.com/HengruiTianSJTU/LesionDiffusion), together with its BiomedCLIP `tokenizer.json`. Component terms remain separate. Convert the explicitly selected local assets:
50
+
51
+ ```python
52
+ from pathlib import Path
53
+ from lesiondiffusion_mlx import write_conversion
54
+
55
+ write_conversion(
56
+ checkpoints={
57
+ "lmnet": Path("LMNet.ckpt"),
58
+ "linet": Path("LINet.ckpt"),
59
+ "vq": Path("VQ.ckpt"),
60
+ },
61
+ tokenizer_path=Path("tokenizer.json"),
62
+ output_dir=Path("lesiondiffusion"),
63
+ )
64
+ ```
65
+
66
+ The [package README](https://github.com/MedMLX/lesiondiffusion-mlx#input-and-output) defines the CT and organ-mask preparation contract. The example below explicitly selects the paper's `source` inference mode. The package currently defaults to `conditioned`; its source-parity result must not be applied to that mode.
67
+
68
+ ## Recorded example
69
+
70
+ AMOS22 (validation) / amos_0008; AMOS22 (validation) / amos_0008 label 6 (liver)
71
+
72
+ ```python
73
+ from lesiondiffusion_mlx.medmlx import run
74
+
75
+ # Replace file and directory names with your locally obtained inputs and weights.
76
+ kwargs = {
77
+ "image_path": "amos_0008_ct_1mm_ras_160x160x128.nii.gz",
78
+ "organ_mask_path": "amos_0008_liver_1mm_ras_160x160x128.nii.gz",
79
+ "output_dir": "out",
80
+ "weights_path": "lesiondiffusion",
81
+ "organ": "liver",
82
+ "lesion_type": "tumor",
83
+ "bbox_size_mm": [
84
+ 30,
85
+ 30,
86
+ 20
87
+ ],
88
+ "placement": "inside_parenchyma",
89
+ "inference_mode": "source",
90
+ "diffusion_steps": 200,
91
+ "overwrite": False,
92
+ "seed": 23
93
+ }
94
+ result = run(**kwargs)
95
+ print(result["outputs"])
96
+ ```
97
+
98
+ Native result: **success**. Original-code comparison: **pass**, `cosine_similarity` = `0.9999697715207947`.
99
+
100
+ Full controlled 200-step paired reference completed October 12; image cosine 0.99996977 and lesion Dice 1.0. Density and outside-lesion reconstruction limitations remain.
101
+
102
+ The complete [native run](https://github.com/MedMLX/Med-MLX/blob/main/results/runs/lesiondiffusion.json) and [original-code comparison](https://github.com/MedMLX/Med-MLX/blob/main/results/reference/lesiondiffusion.json) retain commands, geometry, precision, and output-specific checks. Use the [replay harness](https://github.com/MedMLX/Med-MLX/blob/main/benchmarks/README.md) to substitute local paths. The package README documents installation and conversion.
103
+
104
+ The current package defaults to `inference_mode="conditioned"`, which enables LINet text context and native FP16 convolution. The example above explicitly selects `source` to retain the paper's evaluated mode. The [October 12 follow-up](https://github.com/MedMLX/Med-MLX/blob/main/results/lesiondiffusion-conditioned-20261012.json) records a 452.6 s 200-step run with −20.6 HU lesion-minus-nearby-liver contrast and a 128.4 s 50-step draw with −19.0 HU contrast on the same case. This additional visibility check does not extend the paper's source-parity result to conditioned mode or establish clinical validity.
105
+
106
+ ## Distribution terms
107
+
108
+ Upstream code: **MIT**. Weights: **Unresolved; retrieve directly from the original publisher**.
109
+ Eligible route: **converter**. Prepared locally; no converted weights uploaded.
110
+
111
+ Full controlled 200-step paired reference completed October 12; image cosine 0.99996977 and lesion Dice 1.0. Density and outside-lesion reconstruction limitations remain. No unambiguous complete-component redistribution grant verified; distribute conversion code and refer users to the original weights.
112
+
113
+ Required notices:
114
+
115
+ - Retain the applicable license, copyright and attribution notices; identify the upstream checkpoint, authors, and MLX conversion changes.
116
+
117
+ Review sources:
118
+
119
+ - https://github.com/HengruiTianSJTU/LesionDiffusion