Instructions to use tariqkhalif/slimsam-77-crack with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use tariqkhalif/slimsam-77-crack with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('mask-generation', 'tariqkhalif/slimsam-77-crack');
Crack-tuned SlimSAM decoder (ONNX, transformers.js layout)
Browse files- README.md +69 -0
- config.json +20 -0
- onnx/prompt_encoder_mask_decoder.onnx +3 -0
- onnx/prompt_encoder_mask_decoder_quantized.onnx +3 -0
- onnx/vision_encoder.onnx +3 -0
- onnx/vision_encoder_quantized.onnx +3 -0
- preprocessor_config.json +28 -0
- quantize_config.json +82 -0
README.md
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: nielsr/slimsam-77-uniform
|
| 4 |
+
library_name: transformers.js
|
| 5 |
+
pipeline_tag: mask-generation
|
| 6 |
+
tags:
|
| 7 |
+
- sam
|
| 8 |
+
- slimsam
|
| 9 |
+
- onnx
|
| 10 |
+
- crack-segmentation
|
| 11 |
+
- insurance
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# SlimSAM 77% uniform, crack-tuned (SnapClaim)
|
| 15 |
+
|
| 16 |
+
The SlimSAM model SnapClaim runs in the browser to outline damage, with the mask
|
| 17 |
+
decoder fine-tuned on concrete cracks. The vision encoder is unchanged, so the
|
| 18 |
+
encoder ONNX files are the same as
|
| 19 |
+
[Xenova/slimsam-77-uniform](https://huggingface.co/Xenova/slimsam-77-uniform) and
|
| 20 |
+
this repo drops in for it with transformers.js:
|
| 21 |
+
|
| 22 |
+
```js
|
| 23 |
+
const model = await SamModel.from_pretrained("<this repo>");
|
| 24 |
+
```
|
| 25 |
+
|
| 26 |
+
## Why
|
| 27 |
+
|
| 28 |
+
Cracks are thin. With one tap on a crack, stock SlimSAM's three candidate masks
|
| 29 |
+
are usually the wall around it, or a blob. SnapClaim keeps the smallest
|
| 30 |
+
candidate that still scores 0.5 or more. This model trains the decoder so that
|
| 31 |
+
candidate actually traces the crack.
|
| 32 |
+
|
| 33 |
+
## Results
|
| 34 |
+
|
| 35 |
+
Held-out split: 92 images (20%, seed 7) that training never saw. Five random
|
| 36 |
+
taps on the crack per image, 460 taps in all. mIoU against the ground-truth
|
| 37 |
+
mask, picking the candidate the same way the app does.
|
| 38 |
+
|
| 39 |
+
| model | mIoU (app's pick) | mIoU (best of 3) |
|
| 40 |
+
|---|---|---|
|
| 41 |
+
| stock SlimSAM (PyTorch) | 0.131 | 0.234 |
|
| 42 |
+
| tuned, ONNX fp32 (WebGPU path) | 0.459 | 0.476 |
|
| 43 |
+
| tuned, ONNX quantized (wasm path) | 0.457 | 0.474 |
|
| 44 |
+
|
| 45 |
+
The ONNX decoder matches PyTorch to within 0.0008 on the raw logits.
|
| 46 |
+
|
| 47 |
+
## Training
|
| 48 |
+
|
| 49 |
+
- Data: Concrete Crack Segmentation Dataset, Özgenel (2019), Mendeley Data V1,
|
| 50 |
+
doi:10.17632/jwsn7tfbrp.1, CC BY 4.0. 458 image and mask pairs, 80/20 split.
|
| 51 |
+
- Frozen encoder, embeddings cached once (plus a horizontal flip). Only the
|
| 52 |
+
prompt encoder's mask decoder is trained.
|
| 53 |
+
- 12 epochs, AdamW lr 1e-4, cosine schedule, one random positive tap per
|
| 54 |
+
image per step. Loss: SAM's focal x20 plus dice on the best candidate, plus
|
| 55 |
+
MSE on the IoU head. The checkpoint was chosen on a validation slice of the
|
| 56 |
+
training split.
|
| 57 |
+
- Quantized with the same settings as Xenova's export (dynamic, per-channel,
|
| 58 |
+
QInt8 decoder weights).
|
| 59 |
+
|
| 60 |
+
## Limits
|
| 61 |
+
|
| 62 |
+
The training photos are phone shots (mostly 4032x3024) of concrete surfaces
|
| 63 |
+
with cracks. Results on other materials (plaster, brick, tile), on wide shots, and on
|
| 64 |
+
non-crack damage such as dents or water stains have not been measured.
|
| 65 |
+
|
| 66 |
+
## License
|
| 67 |
+
|
| 68 |
+
Apache-2.0, like the base model. The training data is CC BY 4.0. Credit:
|
| 69 |
+
Çağlar Fırat Özgenel, Concrete Crack Segmentation Dataset.
|
config.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_name_or_path": "nielsr/slimsam-77-uniform",
|
| 3 |
+
"architectures": [
|
| 4 |
+
"SamModel"
|
| 5 |
+
],
|
| 6 |
+
"initializer_range": 0.02,
|
| 7 |
+
"mask_decoder_config": {
|
| 8 |
+
"model_type": ""
|
| 9 |
+
},
|
| 10 |
+
"model_type": "sam",
|
| 11 |
+
"prompt_encoder_config": {
|
| 12 |
+
"model_type": ""
|
| 13 |
+
},
|
| 14 |
+
"transformers_version": "4.36.2",
|
| 15 |
+
"vision_config": {
|
| 16 |
+
"hidden_size": 168,
|
| 17 |
+
"mlp_dim": 696,
|
| 18 |
+
"model_type": ""
|
| 19 |
+
}
|
| 20 |
+
}
|
onnx/prompt_encoder_mask_decoder.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:46633bca518a23ae8c3390d8a5b8da7f485bd5827df3bea7eb9be296a856078b
|
| 3 |
+
size 16552371
|
onnx/prompt_encoder_mask_decoder_quantized.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ce38ec71baf2902a4283105c4c8e75850b1de2aca3f70233c71bf13765e34af4
|
| 3 |
+
size 4896433
|
onnx/vision_encoder.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9f8433273a6750b587779baa0cf5508111001bf7e7acfcf585d370139fd366d0
|
| 3 |
+
size 23276014
|
onnx/vision_encoder_quantized.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cce23c7b2e5d4f330932738fb67ba518e04b0d99ccdd1cccd22a7da4e01f2971
|
| 3 |
+
size 8882165
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_convert_rgb": true,
|
| 3 |
+
"do_normalize": true,
|
| 4 |
+
"do_pad": true,
|
| 5 |
+
"do_rescale": true,
|
| 6 |
+
"do_resize": true,
|
| 7 |
+
"image_mean": [
|
| 8 |
+
0.485,
|
| 9 |
+
0.456,
|
| 10 |
+
0.406
|
| 11 |
+
],
|
| 12 |
+
"image_processor_type": "SamImageProcessor",
|
| 13 |
+
"image_std": [
|
| 14 |
+
0.229,
|
| 15 |
+
0.224,
|
| 16 |
+
0.225
|
| 17 |
+
],
|
| 18 |
+
"pad_size": {
|
| 19 |
+
"height": 1024,
|
| 20 |
+
"width": 1024
|
| 21 |
+
},
|
| 22 |
+
"processor_class": "SamProcessor",
|
| 23 |
+
"resample": 2,
|
| 24 |
+
"rescale_factor": 0.00392156862745098,
|
| 25 |
+
"size": {
|
| 26 |
+
"longest_edge": 1024
|
| 27 |
+
}
|
| 28 |
+
}
|
quantize_config.json
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"per_channel": true,
|
| 3 |
+
"reduce_range": true,
|
| 4 |
+
"per_model_config": {
|
| 5 |
+
"prompt_encoder_mask_decoder": {
|
| 6 |
+
"op_types": [
|
| 7 |
+
"Gather",
|
| 8 |
+
"Sin",
|
| 9 |
+
"Unsqueeze",
|
| 10 |
+
"Not",
|
| 11 |
+
"Cast",
|
| 12 |
+
"Equal",
|
| 13 |
+
"Neg",
|
| 14 |
+
"MatMul",
|
| 15 |
+
"Cos",
|
| 16 |
+
"OneHot",
|
| 17 |
+
"Transpose",
|
| 18 |
+
"Sub",
|
| 19 |
+
"Where",
|
| 20 |
+
"Erf",
|
| 21 |
+
"ConstantOfShape",
|
| 22 |
+
"ReduceMean",
|
| 23 |
+
"Shape",
|
| 24 |
+
"Range",
|
| 25 |
+
"Expand",
|
| 26 |
+
"Div",
|
| 27 |
+
"Relu",
|
| 28 |
+
"ConvTranspose",
|
| 29 |
+
"Tile",
|
| 30 |
+
"Concat",
|
| 31 |
+
"ScatterND",
|
| 32 |
+
"Sqrt",
|
| 33 |
+
"Slice",
|
| 34 |
+
"Mul",
|
| 35 |
+
"Constant",
|
| 36 |
+
"Pow",
|
| 37 |
+
"Reshape",
|
| 38 |
+
"Add",
|
| 39 |
+
"Softmax"
|
| 40 |
+
],
|
| 41 |
+
"weight_type": "QInt8"
|
| 42 |
+
},
|
| 43 |
+
"vision_encoder": {
|
| 44 |
+
"op_types": [
|
| 45 |
+
"Gather",
|
| 46 |
+
"Sin",
|
| 47 |
+
"Unsqueeze",
|
| 48 |
+
"Cast",
|
| 49 |
+
"MatMul",
|
| 50 |
+
"Cos",
|
| 51 |
+
"Transpose",
|
| 52 |
+
"Sub",
|
| 53 |
+
"Pad",
|
| 54 |
+
"Erf",
|
| 55 |
+
"Conv",
|
| 56 |
+
"ConstantOfShape",
|
| 57 |
+
"Resize",
|
| 58 |
+
"ReduceMean",
|
| 59 |
+
"Shape",
|
| 60 |
+
"Range",
|
| 61 |
+
"Expand",
|
| 62 |
+
"Div",
|
| 63 |
+
"Tile",
|
| 64 |
+
"Mod",
|
| 65 |
+
"Squeeze",
|
| 66 |
+
"Concat",
|
| 67 |
+
"Sqrt",
|
| 68 |
+
"Split",
|
| 69 |
+
"Mul",
|
| 70 |
+
"Slice",
|
| 71 |
+
"Identity",
|
| 72 |
+
"Constant",
|
| 73 |
+
"Pow",
|
| 74 |
+
"Einsum",
|
| 75 |
+
"Reshape",
|
| 76 |
+
"Add",
|
| 77 |
+
"Softmax"
|
| 78 |
+
],
|
| 79 |
+
"weight_type": "QUInt8"
|
| 80 |
+
}
|
| 81 |
+
}
|
| 82 |
+
}
|