Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Github: https://github.com/Asher-1/recognize-anything-ggml
|
| 2 |
+
|
| 3 |
+
# RAM family GGUF model card
|
| 4 |
+
|
| 5 |
+
## Source, intended use and download
|
| 6 |
+
|
| 7 |
+
These nine files are exported directly from this repository's three official
|
| 8 |
+
inference checkpoints. The GGUF filename retains the checkpoint stem. All
|
| 9 |
+
downloads are hosted at the [RAM release](https://github.com/Asher-1/cloudViewer_downloads/releases/tag/RAM).
|
| 10 |
+
The release assets' byte counts and SHA-256 digests were checked against the
|
| 11 |
+
local exports on 2026-09-29; [SHA256SUMS](SHA256SUMS) lists the exact hashes.
|
| 12 |
+
|
| 13 |
+
| Source model | Format | Size (MiB) | Relative size | Input/output | Download |
|
| 14 |
+
| --- | --- | ---: | ---: | --- | --- |
|
| 15 |
+
| RAM Swin-L | F32 | 1941.5 | 100% | 384 px RGB / 4585 tags | [ram_swin_large_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-f32.gguf) |
|
| 16 |
+
| RAM Swin-L | F16 | 970.8 | 50.0% | 384 px RGB / 4585 tags | [ram_swin_large_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-f16.gguf) |
|
| 17 |
+
| RAM Swin-L | Q8_0 | 521.4 | 26.9% | 384 px RGB / 4585 tags | [ram_swin_large_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-q8_0.gguf) |
|
| 18 |
+
| RAM++ Swin-L | F32 | 1268.0 | 100% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-f32.gguf) |
|
| 19 |
+
| RAM++ Swin-L | F16 | 634.0 | 50.0% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-f16.gguf) |
|
| 20 |
+
| RAM++ Swin-L | Q8_0 | 447.0 | 35.3% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-q8_0.gguf) |
|
| 21 |
+
| Tag2Text Swin-B | F32 | 1576.9 | 100% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-f32.gguf) |
|
| 22 |
+
| Tag2Text Swin-B | F16 | 788.5 | 50.0% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-f16.gguf) |
|
| 23 |
+
| Tag2Text Swin-B | Q8_0 | 427.0 | 27.1% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-q8_0.gguf) |
|
| 24 |
+
|
| 25 |
+
F32 is the useful reference for investigating numerical differences and has
|
| 26 |
+
the highest weight-storage cost. F16 halves the weight file and preserved the
|
| 27 |
+
official caption wording on the four measured default-tag images, making it
|
| 28 |
+
the first choice when caption wording matters on those samples. Q8_0 minimizes
|
| 29 |
+
download/storage needs while maintaining strong tagging agreement on the
|
| 30 |
+
measured image; it can change
|
| 31 |
+
caption wording. RAM and RAM++ are tagging models. Tag2Text adds a 12-layer
|
| 32 |
+
tag-text encoder, BERT WordPiece tokenizer and 12-layer autoregressive caption
|
| 33 |
+
decoder. The [BERT uncased vocabulary](bert-base-uncased-vocab.txt) is a
|
| 34 |
+
separate runtime file and is **not** embedded in GGUF. The tag names and
|
| 35 |
+
thresholds come from the repository's canonical `ram/data/` files.
|
| 36 |
+
|
| 37 |
+
Q8_0 applies block quantization only to eligible 2D weight matrices. Other
|
| 38 |
+
weights remain F16, including the label embeddings excluded by the exporter.
|
| 39 |
+
File-size ratios are therefore model-dependent. A smaller file does not prove
|
| 40 |
+
proportional runtime VRAM or speed savings: activations, graph buffers, and
|
| 41 |
+
backend fallback operations also consume memory and time. All variants use
|
| 42 |
+
ggml v0.21.0 via a pinned submodule; CUDA and Vulkan builds do not use cuDNN.
|
| 43 |
+
|
| 44 |
+
## Measured inference metrics
|
| 45 |
+
|
| 46 |
+
The following numbers are for `images/demo/demo1.jpg` on one RTX 3060 12 GB,
|
| 47 |
+
driver 550.144.03, 10 warmups and 20 measured runs. The timing scope includes
|
| 48 |
+
image decoding, resize/normalization, transfer, inference, readback and label
|
| 49 |
+
selection; model loading is excluded. PyTorch CUDA baselines were RAM 164.507
|
| 50 |
+
ms, RAM++ 170.223 ms and Tag2Text tags 138.882 ms. Logit MAE is against the
|
| 51 |
+
official PyTorch CUDA logits, and Jaccard compares the selected label sets.
|
| 52 |
+
The full 18-row data, maximum logit errors, boundary flips, plots and commands
|
| 53 |
+
are in [benchmarks](../benchmarks/README.md).
|
| 54 |
+
|
| 55 |
+
| Model | Format | CUDA ms / baseline | Vulkan ms / baseline | CUDA logit MAE | Vulkan logit MAE | CUDA/Vulkan tag Jaccard |
|
| 56 |
+
| --- | --- | ---: | ---: | ---: | ---: | ---: |
|
| 57 |
+
| RAM | F32 | 147.396 / 0.896x | 148.056 / 0.900x | 0.001555 | 0.000232 | 1.000 / 1.000 |
|
| 58 |
+
| RAM | F16 | 141.132 / 0.858x | 138.818 / 0.844x | 0.001729 | 0.002391 | 0.944 / 0.944 |
|
| 59 |
+
| RAM | Q8_0 | 138.580 / 0.842x | 140.634 / 0.855x | 0.008774 | 0.003757 | 1.000 / 1.000 |
|
| 60 |
+
| RAM++ | F32 | 154.126 / 0.905x | 164.735 / 0.968x | 0.001874 | 0.000300 | 1.000 / 1.000 |
|
| 61 |
+
| RAM++ | F16 | 142.537 / 0.837x | 149.898 / 0.881x | 0.001836 | 0.002124 | 1.000 / 1.000 |
|
| 62 |
+
| RAM++ | Q8_0 | 141.725 / 0.833x | 148.772 / 0.874x | 0.008734 | 0.003130 | 1.000 / 1.000 |
|
| 63 |
+
| Tag2Text | F32 | 123.535 / 0.889x | 123.881 / 0.892x | 0.000718 | 0.000491 | 1.000 / 1.000 |
|
| 64 |
+
| Tag2Text | F16 | 120.063 / 0.864x | 120.650 / 0.869x | 0.000681 | 0.000965 | 1.000 / 1.000 |
|
| 65 |
+
| Tag2Text | Q8_0 | 120.286 / 0.866x | 118.945 / 0.856x | 0.004062 | 0.003055 | 1.000 / 1.000 |
|
| 66 |
+
|
| 67 |
+
RAM F16's 0.944 Jaccard is one label flipped at the official cutoff; the
|
| 68 |
+
validator classifies this as an ambiguous threshold-boundary case. All 18
|
| 69 |
+
combinations had zero flips outside a 0.01 probability margin. All 18 tagging
|
| 70 |
+
pipeline combinations were faster than PyTorch CUDA in this sample. These are
|
| 71 |
+
measurements, not universal throughput claims. The [caption comparison](../benchmarks/caption_readme.md)
|
| 72 |
+
records end-to-end caption speed and wording separately.
|
| 73 |
+
|
| 74 |
+
## Measured Tag2Text caption metrics
|
| 75 |
+
|
| 76 |
+
The following image-to-caption timings use the same RTX 3060, five warmups and
|
| 77 |
+
ten measured runs per engine. PyTorch CUDA took 384.654 ms on `demo1.jpg` and
|
| 78 |
+
279.765 ms on `demo2.jpg`. The table includes image preprocessing, tagging,
|
| 79 |
+
text encoding and parallel three-beam decoding; model loading is excluded.
|
| 80 |
+
"Exact" means the generated sentence matches the official Python output
|
| 81 |
+
byte-for-byte.
|
| 82 |
+
|
| 83 |
+
| Format | CUDA demo1/demo2 ms | Vulkan demo1/demo2 ms | CUDA caption exact | Vulkan caption exact | Use case |
|
| 84 |
+
| --- | ---: | ---: | --- | --- | --- |
|
| 85 |
+
| F32 | 296.382 / 203.245 | 346.557 / 239.606 | yes / yes | yes / yes | Numerical reference, ample memory |
|
| 86 |
+
| F16 | 241.187 / 164.446 | 289.751 / 203.374 | yes / yes | yes / yes | Default-tag caption speed/size balance |
|
| 87 |
+
| Q8_0 | 219.413 / 148.833 | 325.888 / 208.556 | yes / yes | no / yes | Smallest download; tolerate wording changes |
|
| 88 |
+
|
| 89 |
+
All 12 combinations are faster than the paired Python runs on these two images;
|
| 90 |
+
the ratios range from 0.532x to 0.901x. Vulkan Q8_0 on `demo1.jpg` is the only
|
| 91 |
+
caption wording difference, with word F1 0.895; all exact matches have F1
|
| 92 |
+
1.000. This is sample-level evidence, not a guarantee on other images or GPUs.
|
| 93 |
+
See the [caption chart, text outputs and raw measurements](../benchmarks/caption_readme.md).
|
| 94 |
+
The [specified-tag comparison](../benchmarks/specified_tags/caption_readme.md)
|
| 95 |
+
shows that even F32/F16 can choose a different sentence for `dog, sofa` on
|
| 96 |
+
`demo1.jpg`, so no format guarantees exact caption wording on arbitrary inputs.
|
| 97 |
+
|
| 98 |
+
No dataset-level mAP, AP, caption BLEU/CIDEr, power draw or peak VRAM figure
|
| 99 |
+
has been established for these exported formats. The tagging table measures
|
| 100 |
+
one demo image and the caption table measures two; neither evaluates the
|
| 101 |
+
official datasets. Users choosing a deployment format should run
|
| 102 |
+
`cpp_ggml/scripts/validate.py` and the benchmark scripts on their target
|
| 103 |
+
device and image distribution.
|
| 104 |
+
|
| 105 |
+
## Reproduce and verify
|
| 106 |
+
|
| 107 |
+
```bash
|
| 108 |
+
cd cpp_ggml/models/gguf
|
| 109 |
+
sha256sum -c ../SHA256SUMS
|
| 110 |
+
cd ../../..
|
| 111 |
+
python3 cpp_ggml/benchmarks/run.py --warmup 10 --repeat 20
|
| 112 |
+
python3 cpp_ggml/benchmarks/caption.py --warmup 5 --repeat 10
|
| 113 |
+
python3 run_inference.py --engine cpp --model tag2text --task caption --backend cuda --dtype f16
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
Use the GGUF asset together with this repository's runtime, tag metadata and
|
| 117 |
+
BERT vocabulary. The [official checkpoint provenance](pytorch/README.md)
|
| 118 |
+
and export script document the conversion path.
|