mudler commited on
Commit
8afaa56
Β·
verified Β·
1 Parent(s): da15aa2

Add/update model card for seg-large

Browse files
Files changed (1) hide show
  1. README.md +15 -16
README.md CHANGED
@@ -17,20 +17,19 @@ base_model: roboflow/rfdetr
17
 
18
  GGUF-format weights of [Roboflow RF-DETR Seg-Large](https://github.com/roboflow/rf-detr) (segmentation variant) for use with [rfdetr.cpp](https://github.com/mudler/rf-detr.cpp), a C++/ggml implementation that matches the upstream PyTorch model on CPU.
19
 
20
- This repo contains all four standard quantizations of this variant. **F16 is the recommended default** β€” same accuracy as F32, 1.85Γ— smaller, and typically the fastest on modern CPUs thanks to ggml's F32Γ—F16 matmul fast path.
21
 
22
  ## Available files
23
 
24
- | File | Quant | Size (MB) |
25
- |---|---|---:|
26
- | `rfdetr-seg-large-f32.gguf` | F32 | 140.9 |
27
- | `rfdetr-seg-large-f16.gguf` ← **recommended** | F16 | 75.7 |
28
- | `rfdetr-seg-large-q8_0.gguf` | Q8_0 | 45.1 |
29
- | `rfdetr-seg-large-q4_K.gguf` | Q4_K | 35.9 |
30
 
31
- > Accuracy + latency for this variant haven't been added to the [BENCHMARK.md](https://github.com/mudler/rf-detr.cpp/blob/main/BENCHMARK.md) sweep yet; the C++ implementation is verified to load and run `rfdetr-cli detect` end-to-end on every quant. Run `scripts/sweep_accuracy.py --variant seg-large` locally for parity numbers.
32
-
33
- All accuracy numbers are computed against the upstream PyTorch reference (`rfdetr 1.7.0`) on 7 COCO val2017 images at threshold 0.5. Latency is measured with `rfdetr-cli bench` (8 iters + 3 warmup) at T=8 threads on a single AMD Ryzen 9 9950X3D image (`coco_kitchen.jpg`, 640x427).
34
 
35
  ## Architecture
36
 
@@ -44,17 +43,17 @@ All accuracy numbers are computed against the upstream PyTorch reference (`rfdet
44
 
45
  ## Quantization notes
46
 
47
- - **F32** β€” full-precision reference, ~120 MB. Bit-exact PyTorch parity.
48
- - **F16** β€” matmul-multiplicand weights only; LayerNorms, conv kernels, embeddings, biases, and layer-scale gammas stay F32. Lossless on this model and consistently the fastest variant on CPU.
49
- - **Q8_0** β€” best size/accuracy tradeoff under F16; ~3Γ— smaller than F32 with effectively identical detections.
50
- - **Q4_K** β€” smallest practical quant. Rows with `ne[0] % 256 != 0` (the decoder's 128-dim MLP halves, 60 tensors) silently fall back to Q8_0 per ggml's quantizer logic β€” net compression is still ~3.8Γ— over F32. Use only when the size budget is tight; expect a measurable Recall@0.95 drop relative to F16/Q8_0 (see file table above).
51
 
52
  ## Usage
53
 
54
  ```bash
55
  # 1. Clone + build rfdetr.cpp
56
  git clone https://github.com/mudler/rf-detr.cpp
57
- cd rt-detr.cpp
58
  cmake -B build -DRFDETR_BUILD_CLI=ON && cmake --build build -j
59
 
60
  # 2. Download a quant (F16 recommended)
@@ -71,7 +70,7 @@ build/bin/rfdetr-cli detect \
71
 
72
  ## Accuracy methodology
73
 
74
- All accuracy metrics are computed against the upstream PyTorch reference (rfdetr 1.7.0) on 7 COCO val2017 images at threshold 0.5. Each detection match uses greedy Hungarian-style assignment by IoU (β‰₯ 0.5 lenient, β‰₯ 0.95 strict) with class equality required.
75
 
76
  Mask metrics are pixel-wise IoU between binary masks at the **original** image resolution (not the network's working resolution), after sigmoid + bicubic upsample of the per-query mask logits. **Pixel agreement** is the fraction of pixels where the C++ and PyTorch binary masks match.
77
 
 
17
 
18
  GGUF-format weights of [Roboflow RF-DETR Seg-Large](https://github.com/roboflow/rf-detr) (segmentation variant) for use with [rfdetr.cpp](https://github.com/mudler/rf-detr.cpp), a C++/ggml implementation that matches the upstream PyTorch model on CPU.
19
 
20
+ This repo contains all four standard quantizations of this variant. **F16 is the recommended default** β€” near-identical accuracy to F32 (see the table below), 1.86Γ— smaller than F32, and it takes ggml's F32Γ—F16 matmul fast path.
21
 
22
  ## Available files
23
 
24
+ | File | Quant | Size (MB) | Recall @ IoU 0.5 | Recall @ IoU 0.95 | Mean mask IoU | Pixel agreement | Latency (median ms, T=8) |
25
+ |---|---|---:|---:|---:|---:|---:|---:|
26
+ | `rfdetr-seg-large-f32.gguf` | F32 | 134.6 | 0.9762 | 0.9762 | 0.9982 | 0.9999 | β€” |
27
+ | `rfdetr-seg-large-f16.gguf` ← **recommended** | F16 | 72.3 | 1.0000 | 0.9762 | 0.9977 | 0.9999 | β€” |
28
+ | `rfdetr-seg-large-q8_0.gguf` | Q8_0 | 43.1 | 0.9881 | 0.9881 | 0.9943 | 0.9999 | β€” |
29
+ | `rfdetr-seg-large-q4_K.gguf` | Q4_K | 34.3 | 0.9238 | 0.6119 | 0.9729 | 0.9994 | β€” |
30
 
31
+ All accuracy numbers are computed against the upstream PyTorch reference (`rfdetr 1.9.0`) on 7 COCO val2017 images at threshold 0.5.
32
+ No latency benchmark has been recorded for this variant yet, so the latency column is left empty. Run `scripts/quick_bench.sh` locally for timings on your own hardware.
 
33
 
34
  ## Architecture
35
 
 
43
 
44
  ## Quantization notes
45
 
46
+ - **F32** β€” the full-precision conversion, 135 MB, and the closest match to the PyTorch reference (see the table above for measured agreement).
47
+ - **F16** β€” matmul-multiplicand weights only; LayerNorms, conv kernels, embeddings, biases, and layer-scale gammas stay F32. Accuracy tracks F32 closely on this model, and it takes ggml's F32Γ—F16 matmul fast path.
48
+ - **Q8_0** β€” best size/accuracy tradeoff under F16; ~3.1Γ— smaller than F32 with effectively identical detections.
49
+ - **Q4_K** β€” smallest practical quant. Rows with `ne[0] % 256 != 0` (the decoder's 128-dim MLP halves) silently fall back to Q8_0 per ggml's quantizer logic β€” net compression is still ~3.9Γ— over F32. Use only when the size budget is tight; expect a measurable Recall@0.95 drop relative to F16/Q8_0 (see file table above).
50
 
51
  ## Usage
52
 
53
  ```bash
54
  # 1. Clone + build rfdetr.cpp
55
  git clone https://github.com/mudler/rf-detr.cpp
56
+ cd rf-detr.cpp
57
  cmake -B build -DRFDETR_BUILD_CLI=ON && cmake --build build -j
58
 
59
  # 2. Download a quant (F16 recommended)
 
70
 
71
  ## Accuracy methodology
72
 
73
+ All accuracy metrics are computed against the upstream PyTorch reference (`rfdetr 1.9.0`) on 7 COCO val2017 images at threshold 0.5. Each detection match uses greedy Hungarian-style assignment by IoU (β‰₯ 0.5 lenient, β‰₯ 0.95 strict) with class equality required.
74
 
75
  Mask metrics are pixel-wise IoU between binary masks at the **original** image resolution (not the network's working resolution), after sigmoid + bicubic upsample of the per-query mask logits. **Pixel agreement** is the fraction of pixels where the C++ and PyTorch binary masks match.
76