tariqkhalif commited on
Commit
5550ebc
·
verified ·
1 Parent(s): 6e5a511

Crack-tuned SlimSAM decoder (ONNX, transformers.js layout)

Browse files
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: nielsr/slimsam-77-uniform
4
+ library_name: transformers.js
5
+ pipeline_tag: mask-generation
6
+ tags:
7
+ - sam
8
+ - slimsam
9
+ - onnx
10
+ - crack-segmentation
11
+ - insurance
12
+ ---
13
+
14
+ # SlimSAM 77% uniform, crack-tuned (SnapClaim)
15
+
16
+ The SlimSAM model SnapClaim runs in the browser to outline damage, with the mask
17
+ decoder fine-tuned on concrete cracks. The vision encoder is unchanged, so the
18
+ encoder ONNX files are the same as
19
+ [Xenova/slimsam-77-uniform](https://huggingface.co/Xenova/slimsam-77-uniform) and
20
+ this repo drops in for it with transformers.js:
21
+
22
+ ```js
23
+ const model = await SamModel.from_pretrained("<this repo>");
24
+ ```
25
+
26
+ ## Why
27
+
28
+ Cracks are thin. With one tap on a crack, stock SlimSAM's three candidate masks
29
+ are usually the wall around it, or a blob. SnapClaim keeps the smallest
30
+ candidate that still scores 0.5 or more. This model trains the decoder so that
31
+ candidate actually traces the crack.
32
+
33
+ ## Results
34
+
35
+ Held-out split: 92 images (20%, seed 7) that training never saw. Five random
36
+ taps on the crack per image, 460 taps in all. mIoU against the ground-truth
37
+ mask, picking the candidate the same way the app does.
38
+
39
+ | model | mIoU (app's pick) | mIoU (best of 3) |
40
+ |---|---|---|
41
+ | stock SlimSAM (PyTorch) | 0.131 | 0.234 |
42
+ | tuned, ONNX fp32 (WebGPU path) | 0.459 | 0.476 |
43
+ | tuned, ONNX quantized (wasm path) | 0.457 | 0.474 |
44
+
45
+ The ONNX decoder matches PyTorch to within 0.0008 on the raw logits.
46
+
47
+ ## Training
48
+
49
+ - Data: Concrete Crack Segmentation Dataset, Özgenel (2019), Mendeley Data V1,
50
+ doi:10.17632/jwsn7tfbrp.1, CC BY 4.0. 458 image and mask pairs, 80/20 split.
51
+ - Frozen encoder, embeddings cached once (plus a horizontal flip). Only the
52
+ prompt encoder's mask decoder is trained.
53
+ - 12 epochs, AdamW lr 1e-4, cosine schedule, one random positive tap per
54
+ image per step. Loss: SAM's focal x20 plus dice on the best candidate, plus
55
+ MSE on the IoU head. The checkpoint was chosen on a validation slice of the
56
+ training split.
57
+ - Quantized with the same settings as Xenova's export (dynamic, per-channel,
58
+ QInt8 decoder weights).
59
+
60
+ ## Limits
61
+
62
+ The training photos are phone shots (mostly 4032x3024) of concrete surfaces
63
+ with cracks. Results on other materials (plaster, brick, tile), on wide shots, and on
64
+ non-crack damage such as dents or water stains have not been measured.
65
+
66
+ ## License
67
+
68
+ Apache-2.0, like the base model. The training data is CC BY 4.0. Credit:
69
+ Çağlar Fırat Özgenel, Concrete Crack Segmentation Dataset.
config.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "nielsr/slimsam-77-uniform",
3
+ "architectures": [
4
+ "SamModel"
5
+ ],
6
+ "initializer_range": 0.02,
7
+ "mask_decoder_config": {
8
+ "model_type": ""
9
+ },
10
+ "model_type": "sam",
11
+ "prompt_encoder_config": {
12
+ "model_type": ""
13
+ },
14
+ "transformers_version": "4.36.2",
15
+ "vision_config": {
16
+ "hidden_size": 168,
17
+ "mlp_dim": 696,
18
+ "model_type": ""
19
+ }
20
+ }
onnx/prompt_encoder_mask_decoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46633bca518a23ae8c3390d8a5b8da7f485bd5827df3bea7eb9be296a856078b
3
+ size 16552371
onnx/prompt_encoder_mask_decoder_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce38ec71baf2902a4283105c4c8e75850b1de2aca3f70233c71bf13765e34af4
3
+ size 4896433
onnx/vision_encoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f8433273a6750b587779baa0cf5508111001bf7e7acfcf585d370139fd366d0
3
+ size 23276014
onnx/vision_encoder_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cce23c7b2e5d4f330932738fb67ba518e04b0d99ccdd1cccd22a7da4e01f2971
3
+ size 8882165
preprocessor_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_convert_rgb": true,
3
+ "do_normalize": true,
4
+ "do_pad": true,
5
+ "do_rescale": true,
6
+ "do_resize": true,
7
+ "image_mean": [
8
+ 0.485,
9
+ 0.456,
10
+ 0.406
11
+ ],
12
+ "image_processor_type": "SamImageProcessor",
13
+ "image_std": [
14
+ 0.229,
15
+ 0.224,
16
+ 0.225
17
+ ],
18
+ "pad_size": {
19
+ "height": 1024,
20
+ "width": 1024
21
+ },
22
+ "processor_class": "SamProcessor",
23
+ "resample": 2,
24
+ "rescale_factor": 0.00392156862745098,
25
+ "size": {
26
+ "longest_edge": 1024
27
+ }
28
+ }
quantize_config.json ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "per_channel": true,
3
+ "reduce_range": true,
4
+ "per_model_config": {
5
+ "prompt_encoder_mask_decoder": {
6
+ "op_types": [
7
+ "Gather",
8
+ "Sin",
9
+ "Unsqueeze",
10
+ "Not",
11
+ "Cast",
12
+ "Equal",
13
+ "Neg",
14
+ "MatMul",
15
+ "Cos",
16
+ "OneHot",
17
+ "Transpose",
18
+ "Sub",
19
+ "Where",
20
+ "Erf",
21
+ "ConstantOfShape",
22
+ "ReduceMean",
23
+ "Shape",
24
+ "Range",
25
+ "Expand",
26
+ "Div",
27
+ "Relu",
28
+ "ConvTranspose",
29
+ "Tile",
30
+ "Concat",
31
+ "ScatterND",
32
+ "Sqrt",
33
+ "Slice",
34
+ "Mul",
35
+ "Constant",
36
+ "Pow",
37
+ "Reshape",
38
+ "Add",
39
+ "Softmax"
40
+ ],
41
+ "weight_type": "QInt8"
42
+ },
43
+ "vision_encoder": {
44
+ "op_types": [
45
+ "Gather",
46
+ "Sin",
47
+ "Unsqueeze",
48
+ "Cast",
49
+ "MatMul",
50
+ "Cos",
51
+ "Transpose",
52
+ "Sub",
53
+ "Pad",
54
+ "Erf",
55
+ "Conv",
56
+ "ConstantOfShape",
57
+ "Resize",
58
+ "ReduceMean",
59
+ "Shape",
60
+ "Range",
61
+ "Expand",
62
+ "Div",
63
+ "Tile",
64
+ "Mod",
65
+ "Squeeze",
66
+ "Concat",
67
+ "Sqrt",
68
+ "Split",
69
+ "Mul",
70
+ "Slice",
71
+ "Identity",
72
+ "Constant",
73
+ "Pow",
74
+ "Einsum",
75
+ "Reshape",
76
+ "Add",
77
+ "Softmax"
78
+ ],
79
+ "weight_type": "QUInt8"
80
+ }
81
+ }
82
+ }