mudler commited on
Commit
aa02851
Β·
verified Β·
1 Parent(s): 1f55f9f

Add/update model card for seg-medium

Browse files
Files changed (1) hide show
  1. README.md +14 -13
README.md CHANGED
@@ -17,18 +17,19 @@ base_model: roboflow/rfdetr
17
 
18
  GGUF-format weights of [Roboflow RF-DETR Seg-Medium](https://github.com/roboflow/rf-detr) (segmentation variant) for use with [rfdetr.cpp](https://github.com/mudler/rf-detr.cpp), a C++/ggml implementation that matches the upstream PyTorch model on CPU.
19
 
20
- This repo contains all four standard quantizations of this variant. **F16 is the recommended default** β€” same accuracy as F32, 1.85Γ— smaller, and typically the fastest on modern CPUs thanks to ggml's F32Γ—F16 matmul fast path.
21
 
22
  ## Available files
23
 
24
  | File | Quant | Size (MB) | Recall @ IoU 0.5 | Recall @ IoU 0.95 | Mean mask IoU | Pixel agreement | Latency (median ms, T=8) |
25
  |---|---|---:|---:|---:|---:|---:|---:|
26
- | `rfdetr-seg-medium-f32.gguf` | F32 | 133.7 | 0.9711 | 0.9582 | 0.9807 | 0.9996 | 199.5 |
27
- | `rfdetr-seg-medium-f16.gguf` ← **recommended** | F16 | 71.5 | 1.0000 | 0.9870 | 0.9813 | 0.9996 | 208.5 |
28
- | `rfdetr-seg-medium-q8_0.gguf` | Q8_0 | 42.4 | 0.9870 | 0.9536 | 0.9810 | 0.9996 | 218.5 |
29
- | `rfdetr-seg-medium-q4_K.gguf` | Q4_K | 33.6 | 0.9426 | 0.6468 | 0.9658 | 0.9989 | 244.2 |
30
 
31
- All accuracy numbers are computed against the upstream PyTorch reference (`rfdetr 1.7.0`) on 7 COCO val2017 images at threshold 0.5. Latency is measured with `rfdetr-cli bench` (8 iters + 3 warmup) at T=8 threads on a single AMD Ryzen 9 9950X3D image (`coco_kitchen.jpg`, 640x427).
 
32
 
33
  ## Architecture
34
 
@@ -42,17 +43,17 @@ All accuracy numbers are computed against the upstream PyTorch reference (`rfdet
42
 
43
  ## Quantization notes
44
 
45
- - **F32** β€” full-precision reference, ~120 MB. Bit-exact PyTorch parity.
46
- - **F16** β€” matmul-multiplicand weights only; LayerNorms, conv kernels, embeddings, biases, and layer-scale gammas stay F32. Lossless on this model and consistently the fastest variant on CPU.
47
- - **Q8_0** β€” best size/accuracy tradeoff under F16; ~3Γ— smaller than F32 with effectively identical detections.
48
- - **Q4_K** β€” smallest practical quant. Rows with `ne[0] % 256 != 0` (the decoder's 128-dim MLP halves, 60 tensors) silently fall back to Q8_0 per ggml's quantizer logic β€” net compression is still ~3.8Γ— over F32. Use only when the size budget is tight; expect a measurable Recall@0.95 drop relative to F16/Q8_0 (see file table above).
49
 
50
  ## Usage
51
 
52
  ```bash
53
  # 1. Clone + build rfdetr.cpp
54
  git clone https://github.com/mudler/rf-detr.cpp
55
- cd rt-detr.cpp
56
  cmake -B build -DRFDETR_BUILD_CLI=ON && cmake --build build -j
57
 
58
  # 2. Download a quant (F16 recommended)
@@ -69,11 +70,11 @@ build/bin/rfdetr-cli detect \
69
 
70
  ## Accuracy methodology
71
 
72
- All accuracy metrics are computed against the upstream PyTorch reference (rfdetr 1.7.0) on 7 COCO val2017 images at threshold 0.5. Each detection match uses greedy Hungarian-style assignment by IoU (β‰₯ 0.5 lenient, β‰₯ 0.95 strict) with class equality required.
73
 
74
  Mask metrics are pixel-wise IoU between binary masks at the **original** image resolution (not the network's working resolution), after sigmoid + bicubic upsample of the per-query mask logits. **Pixel agreement** is the fraction of pixels where the C++ and PyTorch binary masks match.
75
 
76
- See [BENCHMARK.md](https://github.com/mudler/rf-detr.cpp/blob/main/BENCHMARK.md) and [`benchmarks/results/accuracy_sweep.json`](https://github.com/mudler/rf-detr.cpp/blob/main/benchmarks/results/accuracy_sweep.json) for the full sweep across all 32 (variant Γ— quant) cells.
77
 
78
  ## License
79
 
 
17
 
18
  GGUF-format weights of [Roboflow RF-DETR Seg-Medium](https://github.com/roboflow/rf-detr) (segmentation variant) for use with [rfdetr.cpp](https://github.com/mudler/rf-detr.cpp), a C++/ggml implementation that matches the upstream PyTorch model on CPU.
19
 
20
+ This repo contains all four standard quantizations of this variant. **F16 is the recommended default** β€” near-identical accuracy to F32 (see the table below), 1.87Γ— smaller than F32, and it takes ggml's F32Γ—F16 matmul fast path.
21
 
22
  ## Available files
23
 
24
  | File | Quant | Size (MB) | Recall @ IoU 0.5 | Recall @ IoU 0.95 | Mean mask IoU | Pixel agreement | Latency (median ms, T=8) |
25
  |---|---|---:|---:|---:|---:|---:|---:|
26
+ | `rfdetr-seg-medium-f32.gguf` | F32 | 133.9 | 0.9841 | 0.9841 | 0.9975 | 0.9999 | β€” |
27
+ | `rfdetr-seg-medium-f16.gguf` ← **recommended** | F16 | 71.6 | 0.9841 | 0.9841 | 0.9975 | 0.9999 | β€” |
28
+ | `rfdetr-seg-medium-q8_0.gguf` | Q8_0 | 42.5 | 0.9841 | 0.9637 | 0.9938 | 0.9999 | β€” |
29
+ | `rfdetr-seg-medium-q4_K.gguf` | Q4_K | 33.6 | 0.9296 | 0.6420 | 0.9699 | 0.9993 | β€” |
30
 
31
+ All accuracy numbers are computed against the upstream PyTorch reference (`rfdetr 1.9.0`) on 7 COCO val2017 images at threshold 0.5.
32
+ No latency benchmark has been recorded for this variant yet, so the latency column is left empty. Run `scripts/quick_bench.sh` locally for timings on your own hardware.
33
 
34
  ## Architecture
35
 
 
43
 
44
  ## Quantization notes
45
 
46
+ - **F32** β€” the full-precision conversion, 134 MB, and the closest match to the PyTorch reference (see the table above for measured agreement).
47
+ - **F16** β€” matmul-multiplicand weights only; LayerNorms, conv kernels, embeddings, biases, and layer-scale gammas stay F32. Accuracy tracks F32 closely on this model, and it takes ggml's F32Γ—F16 matmul fast path.
48
+ - **Q8_0** β€” best size/accuracy tradeoff under F16; ~3.2Γ— smaller than F32 with effectively identical detections.
49
+ - **Q4_K** β€” smallest practical quant. Rows with `ne[0] % 256 != 0` (the decoder's 128-dim MLP halves) silently fall back to Q8_0 per ggml's quantizer logic β€” net compression is still ~4.0Γ— over F32. Use only when the size budget is tight; expect a measurable Recall@0.95 drop relative to F16/Q8_0 (see file table above).
50
 
51
  ## Usage
52
 
53
  ```bash
54
  # 1. Clone + build rfdetr.cpp
55
  git clone https://github.com/mudler/rf-detr.cpp
56
+ cd rf-detr.cpp
57
  cmake -B build -DRFDETR_BUILD_CLI=ON && cmake --build build -j
58
 
59
  # 2. Download a quant (F16 recommended)
 
70
 
71
  ## Accuracy methodology
72
 
73
+ All accuracy metrics are computed against the upstream PyTorch reference (`rfdetr 1.9.0`) on 7 COCO val2017 images at threshold 0.5. Each detection match uses greedy Hungarian-style assignment by IoU (β‰₯ 0.5 lenient, β‰₯ 0.95 strict) with class equality required.
74
 
75
  Mask metrics are pixel-wise IoU between binary masks at the **original** image resolution (not the network's working resolution), after sigmoid + bicubic upsample of the per-query mask logits. **Pixel agreement** is the fraction of pixels where the C++ and PyTorch binary masks match.
76
 
77
+ See [BENCHMARK.md](https://github.com/mudler/rf-detr.cpp/blob/main/BENCHMARK.md) and [`benchmarks/results/accuracy_sweep.json`](https://github.com/mudler/rf-detr.cpp/blob/main/benchmarks/results/accuracy_sweep.json) for the full sweep across the (variant Γ— quant) cells.
78
 
79
  ## License
80