Asher-1 commited on
Commit
fa832ed
·
verified ·
1 Parent(s): b0a5ba1

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +118 -0
README.md ADDED
@@ -0,0 +1,118 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Github: https://github.com/Asher-1/recognize-anything-ggml
2
+
3
+ # RAM family GGUF model card
4
+
5
+ ## Source, intended use and download
6
+
7
+ These nine files are exported directly from this repository's three official
8
+ inference checkpoints. The GGUF filename retains the checkpoint stem. All
9
+ downloads are hosted at the [RAM release](https://github.com/Asher-1/cloudViewer_downloads/releases/tag/RAM).
10
+ The release assets' byte counts and SHA-256 digests were checked against the
11
+ local exports on 2026-09-29; [SHA256SUMS](SHA256SUMS) lists the exact hashes.
12
+
13
+ | Source model | Format | Size (MiB) | Relative size | Input/output | Download |
14
+ | --- | --- | ---: | ---: | --- | --- |
15
+ | RAM Swin-L | F32 | 1941.5 | 100% | 384 px RGB / 4585 tags | [ram_swin_large_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-f32.gguf) |
16
+ | RAM Swin-L | F16 | 970.8 | 50.0% | 384 px RGB / 4585 tags | [ram_swin_large_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-f16.gguf) |
17
+ | RAM Swin-L | Q8_0 | 521.4 | 26.9% | 384 px RGB / 4585 tags | [ram_swin_large_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_swin_large_14m-q8_0.gguf) |
18
+ | RAM++ Swin-L | F32 | 1268.0 | 100% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-f32.gguf) |
19
+ | RAM++ Swin-L | F16 | 634.0 | 50.0% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-f16.gguf) |
20
+ | RAM++ Swin-L | Q8_0 | 447.0 | 35.3% | 384 px RGB / 4585 tags | [ram_plus_swin_large_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/ram_plus_swin_large_14m-q8_0.gguf) |
21
+ | Tag2Text Swin-B | F32 | 1576.9 | 100% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-f32.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-f32.gguf) |
22
+ | Tag2Text Swin-B | F16 | 788.5 | 50.0% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-f16.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-f16.gguf) |
23
+ | Tag2Text Swin-B | Q8_0 | 427.0 | 27.1% | 384 px RGB / 3429 tags + caption | [tag2text_swin_14m-q8_0.gguf](https://github.com/Asher-1/cloudViewer_downloads/releases/download/RAM/tag2text_swin_14m-q8_0.gguf) |
24
+
25
+ F32 is the useful reference for investigating numerical differences and has
26
+ the highest weight-storage cost. F16 halves the weight file and preserved the
27
+ official caption wording on the four measured default-tag images, making it
28
+ the first choice when caption wording matters on those samples. Q8_0 minimizes
29
+ download/storage needs while maintaining strong tagging agreement on the
30
+ measured image; it can change
31
+ caption wording. RAM and RAM++ are tagging models. Tag2Text adds a 12-layer
32
+ tag-text encoder, BERT WordPiece tokenizer and 12-layer autoregressive caption
33
+ decoder. The [BERT uncased vocabulary](bert-base-uncased-vocab.txt) is a
34
+ separate runtime file and is **not** embedded in GGUF. The tag names and
35
+ thresholds come from the repository's canonical `ram/data/` files.
36
+
37
+ Q8_0 applies block quantization only to eligible 2D weight matrices. Other
38
+ weights remain F16, including the label embeddings excluded by the exporter.
39
+ File-size ratios are therefore model-dependent. A smaller file does not prove
40
+ proportional runtime VRAM or speed savings: activations, graph buffers, and
41
+ backend fallback operations also consume memory and time. All variants use
42
+ ggml v0.21.0 via a pinned submodule; CUDA and Vulkan builds do not use cuDNN.
43
+
44
+ ## Measured inference metrics
45
+
46
+ The following numbers are for `images/demo/demo1.jpg` on one RTX 3060 12 GB,
47
+ driver 550.144.03, 10 warmups and 20 measured runs. The timing scope includes
48
+ image decoding, resize/normalization, transfer, inference, readback and label
49
+ selection; model loading is excluded. PyTorch CUDA baselines were RAM 164.507
50
+ ms, RAM++ 170.223 ms and Tag2Text tags 138.882 ms. Logit MAE is against the
51
+ official PyTorch CUDA logits, and Jaccard compares the selected label sets.
52
+ The full 18-row data, maximum logit errors, boundary flips, plots and commands
53
+ are in [benchmarks](../benchmarks/README.md).
54
+
55
+ | Model | Format | CUDA ms / baseline | Vulkan ms / baseline | CUDA logit MAE | Vulkan logit MAE | CUDA/Vulkan tag Jaccard |
56
+ | --- | --- | ---: | ---: | ---: | ---: | ---: |
57
+ | RAM | F32 | 147.396 / 0.896x | 148.056 / 0.900x | 0.001555 | 0.000232 | 1.000 / 1.000 |
58
+ | RAM | F16 | 141.132 / 0.858x | 138.818 / 0.844x | 0.001729 | 0.002391 | 0.944 / 0.944 |
59
+ | RAM | Q8_0 | 138.580 / 0.842x | 140.634 / 0.855x | 0.008774 | 0.003757 | 1.000 / 1.000 |
60
+ | RAM++ | F32 | 154.126 / 0.905x | 164.735 / 0.968x | 0.001874 | 0.000300 | 1.000 / 1.000 |
61
+ | RAM++ | F16 | 142.537 / 0.837x | 149.898 / 0.881x | 0.001836 | 0.002124 | 1.000 / 1.000 |
62
+ | RAM++ | Q8_0 | 141.725 / 0.833x | 148.772 / 0.874x | 0.008734 | 0.003130 | 1.000 / 1.000 |
63
+ | Tag2Text | F32 | 123.535 / 0.889x | 123.881 / 0.892x | 0.000718 | 0.000491 | 1.000 / 1.000 |
64
+ | Tag2Text | F16 | 120.063 / 0.864x | 120.650 / 0.869x | 0.000681 | 0.000965 | 1.000 / 1.000 |
65
+ | Tag2Text | Q8_0 | 120.286 / 0.866x | 118.945 / 0.856x | 0.004062 | 0.003055 | 1.000 / 1.000 |
66
+
67
+ RAM F16's 0.944 Jaccard is one label flipped at the official cutoff; the
68
+ validator classifies this as an ambiguous threshold-boundary case. All 18
69
+ combinations had zero flips outside a 0.01 probability margin. All 18 tagging
70
+ pipeline combinations were faster than PyTorch CUDA in this sample. These are
71
+ measurements, not universal throughput claims. The [caption comparison](../benchmarks/caption_readme.md)
72
+ records end-to-end caption speed and wording separately.
73
+
74
+ ## Measured Tag2Text caption metrics
75
+
76
+ The following image-to-caption timings use the same RTX 3060, five warmups and
77
+ ten measured runs per engine. PyTorch CUDA took 384.654 ms on `demo1.jpg` and
78
+ 279.765 ms on `demo2.jpg`. The table includes image preprocessing, tagging,
79
+ text encoding and parallel three-beam decoding; model loading is excluded.
80
+ "Exact" means the generated sentence matches the official Python output
81
+ byte-for-byte.
82
+
83
+ | Format | CUDA demo1/demo2 ms | Vulkan demo1/demo2 ms | CUDA caption exact | Vulkan caption exact | Use case |
84
+ | --- | ---: | ---: | --- | --- | --- |
85
+ | F32 | 296.382 / 203.245 | 346.557 / 239.606 | yes / yes | yes / yes | Numerical reference, ample memory |
86
+ | F16 | 241.187 / 164.446 | 289.751 / 203.374 | yes / yes | yes / yes | Default-tag caption speed/size balance |
87
+ | Q8_0 | 219.413 / 148.833 | 325.888 / 208.556 | yes / yes | no / yes | Smallest download; tolerate wording changes |
88
+
89
+ All 12 combinations are faster than the paired Python runs on these two images;
90
+ the ratios range from 0.532x to 0.901x. Vulkan Q8_0 on `demo1.jpg` is the only
91
+ caption wording difference, with word F1 0.895; all exact matches have F1
92
+ 1.000. This is sample-level evidence, not a guarantee on other images or GPUs.
93
+ See the [caption chart, text outputs and raw measurements](../benchmarks/caption_readme.md).
94
+ The [specified-tag comparison](../benchmarks/specified_tags/caption_readme.md)
95
+ shows that even F32/F16 can choose a different sentence for `dog, sofa` on
96
+ `demo1.jpg`, so no format guarantees exact caption wording on arbitrary inputs.
97
+
98
+ No dataset-level mAP, AP, caption BLEU/CIDEr, power draw or peak VRAM figure
99
+ has been established for these exported formats. The tagging table measures
100
+ one demo image and the caption table measures two; neither evaluates the
101
+ official datasets. Users choosing a deployment format should run
102
+ `cpp_ggml/scripts/validate.py` and the benchmark scripts on their target
103
+ device and image distribution.
104
+
105
+ ## Reproduce and verify
106
+
107
+ ```bash
108
+ cd cpp_ggml/models/gguf
109
+ sha256sum -c ../SHA256SUMS
110
+ cd ../../..
111
+ python3 cpp_ggml/benchmarks/run.py --warmup 10 --repeat 20
112
+ python3 cpp_ggml/benchmarks/caption.py --warmup 5 --repeat 10
113
+ python3 run_inference.py --engine cpp --model tag2text --task caption --backend cuda --dtype f16
114
+ ```
115
+
116
+ Use the GGUF asset together with this repository's runtime, tag metadata and
117
+ BERT vocabulary. The [official checkpoint provenance](pytorch/README.md)
118
+ and export script document the conversion path.