Publish MIC usage, slim fixture and reviewed Transformers expectations
Browse files- .mic-repo.json +7 -0
- README.md +37 -0
- added_tokens.json +7 -0
- build.json +262 -0
- config.json +137 -0
- integration.json +103 -0
- integrations/transformers/expectations/cpu-x86_64-bf16/grounding-dino-tiny-mic.bfloat16.json +144 -0
- integrations/transformers/expectations/cpu-x86_64/grounding-dino-tiny-mic.eager.json +144 -0
- integrations/transformers/expectations/cpu-x86_64/grounding-dino-tiny-mic.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.bfloat16.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.eager.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny.bfloat16.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny.eager.json +144 -0
- integrations/transformers/expectations/cuda/grounding-dino-tiny.json +144 -0
- integrations/transformers/usage.md +93 -0
- model.safetensors +3 -0
- preprocessor_config.json +25 -0
- special_tokens_map.json +7 -0
- tokenizer.json +0 -0
- tokenizer_config.json +56 -0
- vocab.txt +0 -0
.mic-repo.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "mic-umbrella-v1",
|
| 3 |
+
"contract_files": [
|
| 4 |
+
"integration.json",
|
| 5 |
+
"integrations/transformers/usage.md"
|
| 6 |
+
]
|
| 7 |
+
}
|
README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- mic
|
| 4 |
+
- test-artifact
|
| 5 |
+
base_model: IDEA-Research/grounding-dino-tiny
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# grounding-dino-tiny-mic
|
| 9 |
+
|
| 10 |
+
Slim model weights, executable usage and framework expectations in one repository.
|
| 11 |
+
|
| 12 |
+
- [Contract manifest](integration.json)
|
| 13 |
+
- [Transformers usage](integrations/transformers/usage.md)
|
| 14 |
+
- [Transformers expectations](integrations/transformers/expectations)
|
| 15 |
+
- [Fixture provenance](build.json)
|
| 16 |
+
|
| 17 |
+
## Contract
|
| 18 |
+
|
| 19 |
+
### Grounding DINO zero-shot object detection
|
| 20 |
+
|
| 21 |
+
Runnable usage and the framework expectations directory are linked above.
|
| 22 |
+
|
| 23 |
+
## Slim model
|
| 24 |
+
|
| 25 |
+
### Slim fixture: grounding-dino-tiny-mic
|
| 26 |
+
|
| 27 |
+
A 7.7M-parameter fixture for the [GroundingDINO MIC](https://huggingface.co/huggingface-mic/grounding-dino-tiny-mic), covering [`IDEA-Research/grounding-dino-tiny`](https://huggingface.co/IDEA-Research/grounding-dino-tiny). It is a test artifact, not a usable model, tagged `test-artifact` to keep it out of candidate-model search.
|
| 28 |
+
|
| 29 |
+
- **Fixture key `32641c93e0d5`.** It covers every checkpoint whose processor files are byte-identical and whose config matches after shrinking.
|
| 30 |
+
- **Same architecture, smaller config:** `GroundingDinoForObjectDetection`, shrunk by the generic rule in [`sweep.py`](https://github.com/huggingface/transformers-mic/blob/main/experiments/sweep.py) (encoder_layers 2, decoder_layers 2, backbone num_layers 2, text num_hidden_layers 2).
|
| 31 |
+
- **Processor files copied byte for byte** from `IDEA-Research/grounding-dino-tiny`: `added_tokens.json`, `preprocessor_config.json`, `special_tokens_map.json`, `tokenizer.json`, `tokenizer_config.json`, `vocab.txt`.
|
| 32 |
+
- **Random weights (seed 17).** They passed the input-sensitivity gate: every perturbation of the contract's inputs moved the logits by at least 0.001 (weakest: `pixel_values:halve`, 16.4), so no training was needed.
|
| 33 |
+
- **Same tensor names as the real checkpoint.** The real checkpoint has every parameter, and so does the fixture.
|
| 34 |
+
- **Saved in float32**, the dtype `IDEA-Research/grounding-dino-tiny` loads in by default.
|
| 35 |
+
- **Finite in float32, bfloat16, and float16** (forward, loss, gradients).
|
| 36 |
+
|
| 37 |
+
Built by [`build_fixtures.py`](https://github.com/huggingface/transformers-mic/blob/main/example/build_fixtures.py); [`build.json`](build.json) records the pinned sources, gate results, and file hashes. This repository includes the slim weights, usage and framework expectations; rebuild the other files with `python example/build_fixtures.py --fixture grounding-dino-tiny-mic`.
|
added_tokens.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"[CLS]": 101,
|
| 3 |
+
"[MASK]": 103,
|
| 4 |
+
"[PAD]": 0,
|
| 5 |
+
"[SEP]": 102,
|
| 6 |
+
"[UNK]": 100
|
| 7 |
+
}
|
build.json
ADDED
|
@@ -0,0 +1,262 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"fixture": "huggingface-mic/grounding-dino-tiny-mic",
|
| 3 |
+
"key": "32641c93e0d5feac8a8725be8b710adc617b339ee36d1b86dec1ee01566e6c5a",
|
| 4 |
+
"key_fields": {
|
| 5 |
+
"architectures": [
|
| 6 |
+
"GroundingDinoForObjectDetection"
|
| 7 |
+
],
|
| 8 |
+
"output_hidden_states": false,
|
| 9 |
+
"return_dict": true,
|
| 10 |
+
"chunk_size_feed_forward": 0,
|
| 11 |
+
"is_encoder_decoder": true,
|
| 12 |
+
"id2label": {
|
| 13 |
+
"0": "LABEL_0",
|
| 14 |
+
"1": "LABEL_1"
|
| 15 |
+
},
|
| 16 |
+
"label2id": {
|
| 17 |
+
"LABEL_0": 0,
|
| 18 |
+
"LABEL_1": 1
|
| 19 |
+
},
|
| 20 |
+
"problem_type": null,
|
| 21 |
+
"backbone_config": {
|
| 22 |
+
"architectures": null,
|
| 23 |
+
"output_hidden_states": false,
|
| 24 |
+
"return_dict": true,
|
| 25 |
+
"dtype": null,
|
| 26 |
+
"chunk_size_feed_forward": 0,
|
| 27 |
+
"is_encoder_decoder": false,
|
| 28 |
+
"id2label": {
|
| 29 |
+
"0": "LABEL_0",
|
| 30 |
+
"1": "LABEL_1"
|
| 31 |
+
},
|
| 32 |
+
"label2id": {
|
| 33 |
+
"LABEL_0": 0,
|
| 34 |
+
"LABEL_1": 1
|
| 35 |
+
},
|
| 36 |
+
"problem_type": null,
|
| 37 |
+
"image_size": 224,
|
| 38 |
+
"patch_size": 4,
|
| 39 |
+
"num_channels": 3,
|
| 40 |
+
"embed_dim": 64,
|
| 41 |
+
"depths": [
|
| 42 |
+
1,
|
| 43 |
+
1,
|
| 44 |
+
1,
|
| 45 |
+
1
|
| 46 |
+
],
|
| 47 |
+
"num_heads": [
|
| 48 |
+
2,
|
| 49 |
+
4,
|
| 50 |
+
8,
|
| 51 |
+
16
|
| 52 |
+
],
|
| 53 |
+
"window_size": 7,
|
| 54 |
+
"mlp_ratio": 4.0,
|
| 55 |
+
"qkv_bias": true,
|
| 56 |
+
"hidden_dropout_prob": 0.0,
|
| 57 |
+
"attention_probs_dropout_prob": 0.0,
|
| 58 |
+
"drop_path_rate": 0.1,
|
| 59 |
+
"hidden_act": "gelu",
|
| 60 |
+
"use_absolute_embeddings": false,
|
| 61 |
+
"initializer_range": 0.02,
|
| 62 |
+
"layer_norm_eps": 1e-05,
|
| 63 |
+
"encoder_stride": 32,
|
| 64 |
+
"num_layers": 2,
|
| 65 |
+
"hidden_size": 64,
|
| 66 |
+
"stage_names": [
|
| 67 |
+
"stem",
|
| 68 |
+
"stage1",
|
| 69 |
+
"stage2",
|
| 70 |
+
"stage3",
|
| 71 |
+
"stage4"
|
| 72 |
+
],
|
| 73 |
+
"_name_or_path": "",
|
| 74 |
+
"model_type": "swin",
|
| 75 |
+
"output_attentions": false,
|
| 76 |
+
"out_features": [
|
| 77 |
+
"stage2",
|
| 78 |
+
"stage3",
|
| 79 |
+
"stage4"
|
| 80 |
+
],
|
| 81 |
+
"out_indices": [
|
| 82 |
+
2,
|
| 83 |
+
3,
|
| 84 |
+
4
|
| 85 |
+
]
|
| 86 |
+
},
|
| 87 |
+
"text_config": {
|
| 88 |
+
"architectures": null,
|
| 89 |
+
"output_hidden_states": false,
|
| 90 |
+
"return_dict": true,
|
| 91 |
+
"dtype": null,
|
| 92 |
+
"chunk_size_feed_forward": 0,
|
| 93 |
+
"is_encoder_decoder": false,
|
| 94 |
+
"id2label": {
|
| 95 |
+
"0": "LABEL_0",
|
| 96 |
+
"1": "LABEL_1"
|
| 97 |
+
},
|
| 98 |
+
"label2id": {
|
| 99 |
+
"LABEL_0": 0,
|
| 100 |
+
"LABEL_1": 1
|
| 101 |
+
},
|
| 102 |
+
"problem_type": null,
|
| 103 |
+
"vocab_size": 30522,
|
| 104 |
+
"hidden_size": 64,
|
| 105 |
+
"num_hidden_layers": 2,
|
| 106 |
+
"num_attention_heads": 4,
|
| 107 |
+
"intermediate_size": 128,
|
| 108 |
+
"hidden_act": "gelu",
|
| 109 |
+
"hidden_dropout_prob": 0.1,
|
| 110 |
+
"attention_probs_dropout_prob": 0.1,
|
| 111 |
+
"max_position_embeddings": 512,
|
| 112 |
+
"type_vocab_size": 2,
|
| 113 |
+
"initializer_range": 0.02,
|
| 114 |
+
"layer_norm_eps": 1e-12,
|
| 115 |
+
"pad_token_id": 0,
|
| 116 |
+
"use_cache": true,
|
| 117 |
+
"classifier_dropout": null,
|
| 118 |
+
"is_decoder": false,
|
| 119 |
+
"add_cross_attention": false,
|
| 120 |
+
"bos_token_id": null,
|
| 121 |
+
"eos_token_id": null,
|
| 122 |
+
"tie_word_embeddings": true,
|
| 123 |
+
"_name_or_path": "",
|
| 124 |
+
"model_type": "bert",
|
| 125 |
+
"output_attentions": false
|
| 126 |
+
},
|
| 127 |
+
"num_queries": 900,
|
| 128 |
+
"encoder_layers": 2,
|
| 129 |
+
"encoder_ffn_dim": 128,
|
| 130 |
+
"encoder_attention_heads": 4,
|
| 131 |
+
"decoder_layers": 2,
|
| 132 |
+
"decoder_ffn_dim": 128,
|
| 133 |
+
"decoder_attention_heads": 4,
|
| 134 |
+
"activation_function": "relu",
|
| 135 |
+
"d_model": 64,
|
| 136 |
+
"dropout": 0.1,
|
| 137 |
+
"attention_dropout": 0.0,
|
| 138 |
+
"activation_dropout": 0.0,
|
| 139 |
+
"auxiliary_loss": false,
|
| 140 |
+
"position_embedding_type": "sine",
|
| 141 |
+
"num_feature_levels": 4,
|
| 142 |
+
"encoder_n_points": 4,
|
| 143 |
+
"decoder_n_points": 4,
|
| 144 |
+
"two_stage": true,
|
| 145 |
+
"class_cost": 1.0,
|
| 146 |
+
"bbox_cost": 5.0,
|
| 147 |
+
"giou_cost": 2.0,
|
| 148 |
+
"bbox_loss_coefficient": 5.0,
|
| 149 |
+
"giou_loss_coefficient": 2.0,
|
| 150 |
+
"focal_alpha": 0.25,
|
| 151 |
+
"disable_custom_kernels": false,
|
| 152 |
+
"max_text_len": 256,
|
| 153 |
+
"text_enhancer_dropout": 0.0,
|
| 154 |
+
"fusion_droppath": 0.1,
|
| 155 |
+
"fusion_dropout": 0.0,
|
| 156 |
+
"embedding_init_target": true,
|
| 157 |
+
"query_dim": 4,
|
| 158 |
+
"decoder_bbox_embed_share": true,
|
| 159 |
+
"two_stage_bbox_embed_share": false,
|
| 160 |
+
"positional_embedding_temperature": 20,
|
| 161 |
+
"init_std": 0.02,
|
| 162 |
+
"layer_norm_eps": 1e-05,
|
| 163 |
+
"tie_word_embeddings": true,
|
| 164 |
+
"backbone": null,
|
| 165 |
+
"model_type": "grounding-dino",
|
| 166 |
+
"output_attentions": false
|
| 167 |
+
},
|
| 168 |
+
"covers": {
|
| 169 |
+
"IDEA-Research/grounding-dino-tiny": "a2bb814dd30d776dcf7e30523b00659f4f141c71"
|
| 170 |
+
},
|
| 171 |
+
"built_from": {
|
| 172 |
+
"repo": "IDEA-Research/grounding-dino-tiny",
|
| 173 |
+
"revision": "a2bb814dd30d776dcf7e30523b00659f4f141c71"
|
| 174 |
+
},
|
| 175 |
+
"seed": 17,
|
| 176 |
+
"dtype": "float32",
|
| 177 |
+
"builder": "example/build_fixtures.py",
|
| 178 |
+
"model_class": "GroundingDinoForObjectDetection",
|
| 179 |
+
"parameters": 7714390,
|
| 180 |
+
"missing_parameters": [],
|
| 181 |
+
"checkpoint_inspection": "parameter headers/indexes; bounded binary fallback",
|
| 182 |
+
"numeric_health_scope": "CPU float32/bfloat16/float16; autocast for low-precision losses",
|
| 183 |
+
"captured": {
|
| 184 |
+
"call": "training step",
|
| 185 |
+
"inputs": [
|
| 186 |
+
"attention_mask",
|
| 187 |
+
"input_ids",
|
| 188 |
+
"labels",
|
| 189 |
+
"pixel_mask",
|
| 190 |
+
"pixel_values",
|
| 191 |
+
"token_type_ids"
|
| 192 |
+
],
|
| 193 |
+
"class": "GroundingDinoForObjectDetection"
|
| 194 |
+
},
|
| 195 |
+
"weights": "random",
|
| 196 |
+
"gate": {
|
| 197 |
+
"random": {
|
| 198 |
+
"pass": true,
|
| 199 |
+
"accepted_insensitive": {},
|
| 200 |
+
"threshold": 0.001,
|
| 201 |
+
"deltas": {
|
| 202 |
+
"input_ids:roll_batch": 32.26580810546875,
|
| 203 |
+
"input_ids:reverse": 34.78125762939453,
|
| 204 |
+
"token_type_ids:all_ones": 23.121896743774414,
|
| 205 |
+
"attention_mask:all_ones": Infinity,
|
| 206 |
+
"pixel_values:roll_batch": 18.798337936401367,
|
| 207 |
+
"pixel_values:swap_channels": 20.410287857055664,
|
| 208 |
+
"pixel_values:reverse": 19.802776336669922,
|
| 209 |
+
"pixel_values:halve": 16.438833236694336
|
| 210 |
+
},
|
| 211 |
+
"weakest": "pixel_values:halve",
|
| 212 |
+
"no_perturbation_defined": [
|
| 213 |
+
"pixel_mask"
|
| 214 |
+
]
|
| 215 |
+
}
|
| 216 |
+
},
|
| 217 |
+
"numeric_health": {
|
| 218 |
+
"float32": {
|
| 219 |
+
"outputs_finite": true,
|
| 220 |
+
"loss_finite": true,
|
| 221 |
+
"gradients_finite": true,
|
| 222 |
+
"finite": true
|
| 223 |
+
},
|
| 224 |
+
"bfloat16": {
|
| 225 |
+
"outputs_finite": true,
|
| 226 |
+
"loss_finite": true,
|
| 227 |
+
"gradients_finite": true,
|
| 228 |
+
"finite": true
|
| 229 |
+
},
|
| 230 |
+
"float16": {
|
| 231 |
+
"outputs_finite": true,
|
| 232 |
+
"loss_finite": true,
|
| 233 |
+
"unscaled_gradients_finite": false,
|
| 234 |
+
"finite": true
|
| 235 |
+
}
|
| 236 |
+
},
|
| 237 |
+
"copied_unchanged": [
|
| 238 |
+
"added_tokens.json",
|
| 239 |
+
"preprocessor_config.json",
|
| 240 |
+
"special_tokens_map.json",
|
| 241 |
+
"tokenizer.json",
|
| 242 |
+
"tokenizer_config.json",
|
| 243 |
+
"vocab.txt"
|
| 244 |
+
],
|
| 245 |
+
"environment": {
|
| 246 |
+
"python": "3.11.14",
|
| 247 |
+
"machine": "arm64",
|
| 248 |
+
"torch": "2.14.1",
|
| 249 |
+
"transformers": "5.20.1",
|
| 250 |
+
"tokenizers": "0.23.3"
|
| 251 |
+
},
|
| 252 |
+
"sha256": {
|
| 253 |
+
"added_tokens.json": "909e96cb32d92ce728a01bc99850cbba26196d74115c17ebeb019275412588f2",
|
| 254 |
+
"config.json": "df3efb7ce17851c59fc75b4e2b47f46c62ceaebc42d196bedeb1b8854527052c",
|
| 255 |
+
"model.safetensors": "fb032709d5b0c9747aa9ba8462d5adab40f26452d99d119c36ea5ad9260bcd3a",
|
| 256 |
+
"preprocessor_config.json": "8454179ba95e2ad22947835aad7b45862a601fc0055ab88bf1ee70892d3aea60",
|
| 257 |
+
"special_tokens_map.json": "b6d346be366a7d1d48332dbc9fdf3bf8960b5d879522b7799ddba59e76237ee3",
|
| 258 |
+
"tokenizer.json": "d241a60d5e8f04cc1b2b3e9ef7a4921b27bf526d9f6050ab90f9267a1f9e5c66",
|
| 259 |
+
"tokenizer_config.json": "d40ab645b68211910b9170d22433d43186a6ec8ee6fd10ba170524b25bf4fb56",
|
| 260 |
+
"vocab.txt": "07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3"
|
| 261 |
+
}
|
| 262 |
+
}
|
config.json
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"activation_dropout": 0.0,
|
| 3 |
+
"activation_function": "relu",
|
| 4 |
+
"architectures": [
|
| 5 |
+
"GroundingDinoForObjectDetection"
|
| 6 |
+
],
|
| 7 |
+
"attention_dropout": 0.0,
|
| 8 |
+
"auxiliary_loss": false,
|
| 9 |
+
"backbone": null,
|
| 10 |
+
"backbone_config": {
|
| 11 |
+
"_name_or_path": "",
|
| 12 |
+
"architectures": null,
|
| 13 |
+
"attention_probs_dropout_prob": 0.0,
|
| 14 |
+
"chunk_size_feed_forward": 0,
|
| 15 |
+
"depths": [
|
| 16 |
+
1,
|
| 17 |
+
1,
|
| 18 |
+
1,
|
| 19 |
+
1
|
| 20 |
+
],
|
| 21 |
+
"drop_path_rate": 0.1,
|
| 22 |
+
"dtype": "float32",
|
| 23 |
+
"embed_dim": 64,
|
| 24 |
+
"encoder_stride": 32,
|
| 25 |
+
"hidden_act": "gelu",
|
| 26 |
+
"hidden_dropout_prob": 0.0,
|
| 27 |
+
"hidden_size": 64,
|
| 28 |
+
"id2label": {
|
| 29 |
+
"0": "LABEL_0",
|
| 30 |
+
"1": "LABEL_1"
|
| 31 |
+
},
|
| 32 |
+
"image_size": 224,
|
| 33 |
+
"initializer_range": 0.02,
|
| 34 |
+
"is_encoder_decoder": false,
|
| 35 |
+
"label2id": {
|
| 36 |
+
"LABEL_0": 0,
|
| 37 |
+
"LABEL_1": 1
|
| 38 |
+
},
|
| 39 |
+
"layer_norm_eps": 1e-05,
|
| 40 |
+
"mlp_ratio": 4.0,
|
| 41 |
+
"model_type": "swin",
|
| 42 |
+
"num_channels": 3,
|
| 43 |
+
"num_heads": [
|
| 44 |
+
2,
|
| 45 |
+
4,
|
| 46 |
+
8,
|
| 47 |
+
16
|
| 48 |
+
],
|
| 49 |
+
"num_layers": 2,
|
| 50 |
+
"out_features": [
|
| 51 |
+
"stage2",
|
| 52 |
+
"stage3",
|
| 53 |
+
"stage4"
|
| 54 |
+
],
|
| 55 |
+
"out_indices": [
|
| 56 |
+
2,
|
| 57 |
+
3,
|
| 58 |
+
4
|
| 59 |
+
],
|
| 60 |
+
"output_attentions": false,
|
| 61 |
+
"output_hidden_states": false,
|
| 62 |
+
"patch_size": 4,
|
| 63 |
+
"problem_type": null,
|
| 64 |
+
"qkv_bias": true,
|
| 65 |
+
"return_dict": true,
|
| 66 |
+
"stage_names": [
|
| 67 |
+
"stem",
|
| 68 |
+
"stage1",
|
| 69 |
+
"stage2",
|
| 70 |
+
"stage3",
|
| 71 |
+
"stage4"
|
| 72 |
+
],
|
| 73 |
+
"use_absolute_embeddings": false,
|
| 74 |
+
"window_size": 7
|
| 75 |
+
},
|
| 76 |
+
"bbox_cost": 5.0,
|
| 77 |
+
"bbox_loss_coefficient": 5.0,
|
| 78 |
+
"class_cost": 1.0,
|
| 79 |
+
"d_model": 64,
|
| 80 |
+
"decoder_attention_heads": 4,
|
| 81 |
+
"decoder_bbox_embed_share": true,
|
| 82 |
+
"decoder_ffn_dim": 128,
|
| 83 |
+
"decoder_layers": 2,
|
| 84 |
+
"decoder_n_points": 4,
|
| 85 |
+
"disable_custom_kernels": false,
|
| 86 |
+
"dropout": 0.1,
|
| 87 |
+
"dtype": "float32",
|
| 88 |
+
"embedding_init_target": true,
|
| 89 |
+
"encoder_attention_heads": 4,
|
| 90 |
+
"encoder_ffn_dim": 128,
|
| 91 |
+
"encoder_layers": 2,
|
| 92 |
+
"encoder_n_points": 4,
|
| 93 |
+
"focal_alpha": 0.25,
|
| 94 |
+
"fusion_dropout": 0.0,
|
| 95 |
+
"fusion_droppath": 0.1,
|
| 96 |
+
"giou_cost": 2.0,
|
| 97 |
+
"giou_loss_coefficient": 2.0,
|
| 98 |
+
"init_std": 0.02,
|
| 99 |
+
"is_encoder_decoder": true,
|
| 100 |
+
"layer_norm_eps": 1e-05,
|
| 101 |
+
"max_text_len": 256,
|
| 102 |
+
"model_type": "grounding-dino",
|
| 103 |
+
"num_feature_levels": 4,
|
| 104 |
+
"num_queries": 900,
|
| 105 |
+
"position_embedding_type": "sine",
|
| 106 |
+
"positional_embedding_temperature": 20,
|
| 107 |
+
"query_dim": 4,
|
| 108 |
+
"text_config": {
|
| 109 |
+
"add_cross_attention": false,
|
| 110 |
+
"attention_probs_dropout_prob": 0.1,
|
| 111 |
+
"bos_token_id": null,
|
| 112 |
+
"classifier_dropout": null,
|
| 113 |
+
"dtype": "float32",
|
| 114 |
+
"eos_token_id": null,
|
| 115 |
+
"hidden_act": "gelu",
|
| 116 |
+
"hidden_dropout_prob": 0.1,
|
| 117 |
+
"hidden_size": 64,
|
| 118 |
+
"initializer_range": 0.02,
|
| 119 |
+
"intermediate_size": 128,
|
| 120 |
+
"is_decoder": false,
|
| 121 |
+
"layer_norm_eps": 1e-12,
|
| 122 |
+
"max_position_embeddings": 512,
|
| 123 |
+
"model_type": "bert",
|
| 124 |
+
"num_attention_heads": 4,
|
| 125 |
+
"num_hidden_layers": 2,
|
| 126 |
+
"pad_token_id": 0,
|
| 127 |
+
"tie_word_embeddings": true,
|
| 128 |
+
"type_vocab_size": 2,
|
| 129 |
+
"use_cache": true,
|
| 130 |
+
"vocab_size": 30522
|
| 131 |
+
},
|
| 132 |
+
"text_enhancer_dropout": 0.0,
|
| 133 |
+
"tie_word_embeddings": true,
|
| 134 |
+
"transformers_version": "5.20.1",
|
| 135 |
+
"two_stage": true,
|
| 136 |
+
"two_stage_bbox_embed_share": false
|
| 137 |
+
}
|
integration.json
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"mic": "0.3",
|
| 3 |
+
"name": "Grounding DINO zero-shot object detection",
|
| 4 |
+
"usage": "AutoProcessor + AutoModelForZeroShotObjectDetection, batched inference with Hungarian-matcher training step",
|
| 5 |
+
"architecture": "GroundingDinoForObjectDetection",
|
| 6 |
+
"checkpoints": {
|
| 7 |
+
"IDEA-Research/grounding-dino-tiny": {
|
| 8 |
+
"fixture": "huggingface-mic/grounding-dino-tiny-mic"
|
| 9 |
+
}
|
| 10 |
+
},
|
| 11 |
+
"fixtures": {
|
| 12 |
+
"huggingface-mic/grounding-dino-tiny-mic": {
|
| 13 |
+
"built_from": "IDEA-Research/grounding-dino-tiny",
|
| 14 |
+
"covers": [
|
| 15 |
+
"IDEA-Research/grounding-dino-tiny"
|
| 16 |
+
],
|
| 17 |
+
"build": {
|
| 18 |
+
"seed": 17,
|
| 19 |
+
"config_overrides": {
|
| 20 |
+
"fields": {
|
| 21 |
+
"backbone_config.num_heads": [
|
| 22 |
+
2,
|
| 23 |
+
4,
|
| 24 |
+
8,
|
| 25 |
+
16
|
| 26 |
+
]
|
| 27 |
+
},
|
| 28 |
+
"reason": "The generic rule shrinks the Swin backbone's embed_dim to 64 (stage widths 64/128/256/512) but keeps num_heads [3,6,12,24]; 64 channels over 3 heads gives a 21-wide head and the window attention reshape fails. [2,4,8,16] keeps the per-stage head doubling with a 32-wide head."
|
| 29 |
+
}
|
| 30 |
+
}
|
| 31 |
+
}
|
| 32 |
+
},
|
| 33 |
+
"integrations": {
|
| 34 |
+
"transformers": {
|
| 35 |
+
"entry_point": "integrations/transformers/usage.md",
|
| 36 |
+
"format": "runnable-markdown",
|
| 37 |
+
"requires": [
|
| 38 |
+
"transformers>=5",
|
| 39 |
+
"torch"
|
| 40 |
+
],
|
| 41 |
+
"targets": [
|
| 42 |
+
"fixtures",
|
| 43 |
+
"checkpoints"
|
| 44 |
+
],
|
| 45 |
+
"variants": {
|
| 46 |
+
"default": {},
|
| 47 |
+
"eager": {
|
| 48 |
+
"attn_implementation": "eager"
|
| 49 |
+
},
|
| 50 |
+
"bfloat16": {
|
| 51 |
+
"dtype": "bfloat16"
|
| 52 |
+
}
|
| 53 |
+
},
|
| 54 |
+
"training": true,
|
| 55 |
+
"resources": {
|
| 56 |
+
"fixture": {
|
| 57 |
+
"memory_gb": 1,
|
| 58 |
+
"download_mb": 50
|
| 59 |
+
},
|
| 60 |
+
"IDEA-Research/grounding-dino-tiny": {
|
| 61 |
+
"memory_gb": 4,
|
| 62 |
+
"download_mb": 770,
|
| 63 |
+
"gpu_memory_gb": 2,
|
| 64 |
+
"variants": {
|
| 65 |
+
"bfloat16": {
|
| 66 |
+
"memory_gb": 2,
|
| 67 |
+
"gpu_memory_gb": 1
|
| 68 |
+
}
|
| 69 |
+
},
|
| 70 |
+
"devices": [
|
| 71 |
+
"cuda"
|
| 72 |
+
]
|
| 73 |
+
}
|
| 74 |
+
}
|
| 75 |
+
}
|
| 76 |
+
},
|
| 77 |
+
"outputs": {
|
| 78 |
+
"runtime": {
|
| 79 |
+
"compare": "exact"
|
| 80 |
+
},
|
| 81 |
+
"class_scores": {
|
| 82 |
+
"compare": "allclose",
|
| 83 |
+
"atol": 0.0001,
|
| 84 |
+
"rtol": 0.0001
|
| 85 |
+
},
|
| 86 |
+
"box_preds": {
|
| 87 |
+
"compare": "allclose",
|
| 88 |
+
"atol": 0.0001,
|
| 89 |
+
"rtol": 0.0001
|
| 90 |
+
},
|
| 91 |
+
"training.loss": {
|
| 92 |
+
"compare": "allclose",
|
| 93 |
+
"atol": 0.0001,
|
| 94 |
+
"rtol": 0.0001
|
| 95 |
+
},
|
| 96 |
+
"training.gradients": {
|
| 97 |
+
"compare": "all_true"
|
| 98 |
+
},
|
| 99 |
+
"training.updated": {
|
| 100 |
+
"compare": "true"
|
| 101 |
+
}
|
| 102 |
+
}
|
| 103 |
+
}
|
integrations/transformers/expectations/cpu-x86_64-bf16/grounding-dino-tiny-mic.bfloat16.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "bfloat16",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.17",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.1+cpu",
|
| 16 |
+
"torchaudio": "2.11.0+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "12.3.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 9.87,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cpu (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cpu (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.3.5 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "12.3.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.15.0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cpu-x86_64-bf16"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cpu",
|
| 48 |
+
"dtype": "bfloat16",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.71875,
|
| 55 |
+
0.1455078125,
|
| 56 |
+
0.8984375,
|
| 57 |
+
4.3125
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-7.34375,
|
| 61 |
+
-11.9375,
|
| 62 |
+
6.125,
|
| 63 |
+
-3.328125
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
7.15625,
|
| 67 |
+
3.984375,
|
| 68 |
+
-11.875,
|
| 69 |
+
4.8125
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
1.8515625,
|
| 75 |
+
11.1875,
|
| 76 |
+
-1.984375,
|
| 77 |
+
10.0625
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.40625,
|
| 81 |
+
-6.65625,
|
| 82 |
+
-5.0,
|
| 83 |
+
2.15625
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
2.0625,
|
| 87 |
+
-3.9375,
|
| 88 |
+
-10.9375,
|
| 89 |
+
1.8125
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.1240234375,
|
| 97 |
+
0.89453125,
|
| 98 |
+
0.05029296875,
|
| 99 |
+
0.05029296875
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.25,
|
| 103 |
+
0.90625,
|
| 104 |
+
0.05029296875,
|
| 105 |
+
0.05029296875
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.9609375,
|
| 109 |
+
0.65625,
|
| 110 |
+
0.05029296875,
|
| 111 |
+
0.05029296875
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.85546875,
|
| 117 |
+
0.5859375,
|
| 118 |
+
0.05029296875,
|
| 119 |
+
0.05029296875
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.1240234375,
|
| 123 |
+
0.11572265625,
|
| 124 |
+
0.05029296875,
|
| 125 |
+
0.05029296875
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.8828125,
|
| 129 |
+
0.14453125,
|
| 130 |
+
0.05029296875,
|
| 131 |
+
0.05029296875
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 29818.470703125,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cpu-x86_64/grounding-dino-tiny-mic.eager.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "eager",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.17",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.1+cpu",
|
| 16 |
+
"torchaudio": "2.11.0+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "12.3.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 10.18,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cpu (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cpu (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.3.5 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "12.3.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.15.0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cpu-x86_64"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cpu",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.5759308338165283,
|
| 55 |
+
-0.16834381222724915,
|
| 56 |
+
0.743553102016449,
|
| 57 |
+
4.96151065826416
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-5.0102925300598145,
|
| 61 |
+
-7.999488830566406,
|
| 62 |
+
6.8883771896362305,
|
| 63 |
+
1.8706523180007935
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-0.12812943756580353,
|
| 67 |
+
-6.246617794036865,
|
| 68 |
+
-21.668596267700195,
|
| 69 |
+
-0.3667725622653961
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-4.3448405265808105,
|
| 75 |
+
-1.8113731145858765,
|
| 76 |
+
-3.9038443565368652,
|
| 77 |
+
-2.173214912414551
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.323267459869385,
|
| 81 |
+
-6.853495121002197,
|
| 82 |
+
-4.68533182144165,
|
| 83 |
+
2.013044834136963
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
4.921824932098389,
|
| 87 |
+
4.167385578155518,
|
| 88 |
+
-12.891257286071777,
|
| 89 |
+
6.579858779907227
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.12333332747220993,
|
| 97 |
+
0.8949999213218689,
|
| 98 |
+
0.05000000074505806,
|
| 99 |
+
0.05000000074505806
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.4033333361148834,
|
| 103 |
+
0.45500001311302185,
|
| 104 |
+
0.05000000074505806,
|
| 105 |
+
0.05000000074505806
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5633333325386047,
|
| 109 |
+
0.6449999809265137,
|
| 110 |
+
0.05000000074505806,
|
| 111 |
+
0.05000000074505806
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.3768656849861145,
|
| 117 |
+
0.14499999582767487,
|
| 118 |
+
0.05000000074505806,
|
| 119 |
+
0.05000000074505806
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.12313432991504669,
|
| 123 |
+
0.11499998718500137,
|
| 124 |
+
0.05000000074505806,
|
| 125 |
+
0.05000000074505806
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.85447758436203,
|
| 129 |
+
0.5849999785423279,
|
| 130 |
+
0.05000000074505806,
|
| 131 |
+
0.05000000074505806
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 30105.6875,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cpu-x86_64/grounding-dino-tiny-mic.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "default",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.17",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cpu",
|
| 15 |
+
"torchvision": "0.29.1+cpu",
|
| 16 |
+
"torchaudio": "2.11.0+cpu",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.3.5",
|
| 19 |
+
"pillow": "12.3.0",
|
| 20 |
+
"accelerate": "1.15.0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 10.21,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cpu (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cpu (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.3.5 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "12.3.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.15.0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cpu-x86_64"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cpu",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.46091890335083,
|
| 55 |
+
-0.2171020209789276,
|
| 56 |
+
0.7039597034454346,
|
| 57 |
+
4.9123358726501465
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-4.980783462524414,
|
| 61 |
+
-7.940901756286621,
|
| 62 |
+
6.946077823638916,
|
| 63 |
+
1.9217437505722046
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-0.2095434069633484,
|
| 67 |
+
-6.152257442474365,
|
| 68 |
+
-21.6031494140625,
|
| 69 |
+
-0.2780843675136566
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-4.256356239318848,
|
| 75 |
+
-1.8284111022949219,
|
| 76 |
+
-3.9182920455932617,
|
| 77 |
+
-2.1886417865753174
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.305556297302246,
|
| 81 |
+
-6.749832630157471,
|
| 82 |
+
-4.586904525756836,
|
| 83 |
+
2.1162455081939697
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
4.714844226837158,
|
| 87 |
+
4.21169900894165,
|
| 88 |
+
-12.847938537597656,
|
| 89 |
+
6.622302532196045
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.12333332747220993,
|
| 97 |
+
0.8949999213218689,
|
| 98 |
+
0.05000000074505806,
|
| 99 |
+
0.05000000074505806
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.4033333361148834,
|
| 103 |
+
0.45500001311302185,
|
| 104 |
+
0.05000000074505806,
|
| 105 |
+
0.05000000074505806
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5633333325386047,
|
| 109 |
+
0.6449999809265137,
|
| 110 |
+
0.05000000074505806,
|
| 111 |
+
0.05000000074505806
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.3768656849861145,
|
| 117 |
+
0.14499999582767487,
|
| 118 |
+
0.05000000074505806,
|
| 119 |
+
0.05000000074505806
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.12313432991504669,
|
| 123 |
+
0.11499998718500137,
|
| 124 |
+
0.05000000074505806,
|
| 125 |
+
0.05000000074505806
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.85447758436203,
|
| 129 |
+
0.5849999785423279,
|
| 130 |
+
0.05000000074505806,
|
| 131 |
+
0.05000000074505806
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 30066.330078125,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.bfloat16.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "bfloat16",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.33,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "bfloat16",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.3125,
|
| 55 |
+
0.1748046875,
|
| 56 |
+
0.90234375,
|
| 57 |
+
4.53125
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-7.8125,
|
| 61 |
+
-12.375,
|
| 62 |
+
5.96875,
|
| 63 |
+
-3.703125
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-0.189453125,
|
| 67 |
+
-6.21875,
|
| 68 |
+
-21.125,
|
| 69 |
+
-0.44140625
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
1.6640625,
|
| 75 |
+
11.0,
|
| 76 |
+
-1.8125,
|
| 77 |
+
10.25
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.5,
|
| 81 |
+
-6.78125,
|
| 82 |
+
-4.75,
|
| 83 |
+
1.984375
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
1.7421875,
|
| 87 |
+
-4.03125,
|
| 88 |
+
-11.375,
|
| 89 |
+
2.109375
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.1240234375,
|
| 97 |
+
0.89453125,
|
| 98 |
+
0.05029296875,
|
| 99 |
+
0.05029296875
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.2490234375,
|
| 103 |
+
0.90625,
|
| 104 |
+
0.05029296875,
|
| 105 |
+
0.05029296875
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5625,
|
| 109 |
+
0.64453125,
|
| 110 |
+
0.05029296875,
|
| 111 |
+
0.05029296875
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.85546875,
|
| 117 |
+
0.5859375,
|
| 118 |
+
0.05029296875,
|
| 119 |
+
0.05029296875
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.12353515625,
|
| 123 |
+
0.11572265625,
|
| 124 |
+
0.05029296875,
|
| 125 |
+
0.05029296875
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.8828125,
|
| 129 |
+
0.14453125,
|
| 130 |
+
0.05029296875,
|
| 131 |
+
0.05029296875
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 32160.521484375,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.eager.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "eager",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 8.24,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.575408458709717,
|
| 55 |
+
-0.16752147674560547,
|
| 56 |
+
0.7431877255439758,
|
| 57 |
+
4.961802959442139
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-5.009284019470215,
|
| 61 |
+
-7.998274326324463,
|
| 62 |
+
6.887673854827881,
|
| 63 |
+
1.870965838432312
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-0.12954922020435333,
|
| 67 |
+
-6.248267650604248,
|
| 68 |
+
-21.668914794921875,
|
| 69 |
+
-0.3670003116130829
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-4.344493865966797,
|
| 75 |
+
-1.8109407424926758,
|
| 76 |
+
-3.904677391052246,
|
| 77 |
+
-2.174267053604126
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.320732116699219,
|
| 81 |
+
-6.850736141204834,
|
| 82 |
+
-4.685490608215332,
|
| 83 |
+
2.012352705001831
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
4.922200679779053,
|
| 87 |
+
4.1673970222473145,
|
| 88 |
+
-12.8938570022583,
|
| 89 |
+
6.578829765319824
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.12333332747220993,
|
| 97 |
+
0.8949999213218689,
|
| 98 |
+
0.05000000074505806,
|
| 99 |
+
0.05000000074505806
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.4033333659172058,
|
| 103 |
+
0.45500001311302185,
|
| 104 |
+
0.05000000074505806,
|
| 105 |
+
0.05000000074505806
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5633333325386047,
|
| 109 |
+
0.6449999809265137,
|
| 110 |
+
0.05000000074505806,
|
| 111 |
+
0.05000000074505806
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.3768656849861145,
|
| 117 |
+
0.14499998092651367,
|
| 118 |
+
0.05000000074505806,
|
| 119 |
+
0.05000000074505806
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.12313432991504669,
|
| 123 |
+
0.11499994993209839,
|
| 124 |
+
0.05000000074505806,
|
| 125 |
+
0.05000000074505806
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.85447758436203,
|
| 129 |
+
0.5849999785423279,
|
| 130 |
+
0.05000000074505806,
|
| 131 |
+
0.05000000074505806
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 31617.5625,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny-mic.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "default",
|
| 8 |
+
"target": "grounding-dino-tiny-mic",
|
| 9 |
+
"kind": "fixture",
|
| 10 |
+
"revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 9.22,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-3.460416316986084,
|
| 55 |
+
-0.21634916961193085,
|
| 56 |
+
0.7037617564201355,
|
| 57 |
+
4.912403583526611
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-4.97960901260376,
|
| 61 |
+
-7.939585208892822,
|
| 62 |
+
6.945551872253418,
|
| 63 |
+
1.9220560789108276
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-0.21086834371089935,
|
| 67 |
+
-6.153865814208984,
|
| 68 |
+
-21.60309600830078,
|
| 69 |
+
-0.2782466411590576
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-4.255997657775879,
|
| 75 |
+
-1.828185796737671,
|
| 76 |
+
-3.9188520908355713,
|
| 77 |
+
-2.1895363330841064
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.302761077880859,
|
| 81 |
+
-6.747104644775391,
|
| 82 |
+
-4.58685302734375,
|
| 83 |
+
2.11592698097229
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
4.715268611907959,
|
| 87 |
+
4.211608409881592,
|
| 88 |
+
-12.850332260131836,
|
| 89 |
+
6.621493816375732
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.12333332747220993,
|
| 97 |
+
0.8949999213218689,
|
| 98 |
+
0.05000000074505806,
|
| 99 |
+
0.05000000074505806
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.4033333659172058,
|
| 103 |
+
0.45500001311302185,
|
| 104 |
+
0.05000000074505806,
|
| 105 |
+
0.05000000074505806
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5633333325386047,
|
| 109 |
+
0.6449999809265137,
|
| 110 |
+
0.05000000074505806,
|
| 111 |
+
0.05000000074505806
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.3768656849861145,
|
| 117 |
+
0.14499998092651367,
|
| 118 |
+
0.05000000074505806,
|
| 119 |
+
0.05000000074505806
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.12313432991504669,
|
| 123 |
+
0.11499994993209839,
|
| 124 |
+
0.05000000074505806,
|
| 125 |
+
0.05000000074505806
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.85447758436203,
|
| 129 |
+
0.5849999785423279,
|
| 130 |
+
0.05000000074505806,
|
| 131 |
+
0.05000000074505806
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 32203.548828125,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny.bfloat16.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "bfloat16",
|
| 8 |
+
"target": "grounding-dino-tiny",
|
| 9 |
+
"kind": "checkpoint",
|
| 10 |
+
"revision": "a2bb814dd30d776dcf7e30523b00659f4f141c71",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 9.89,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "bfloat16",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-5.15625,
|
| 55 |
+
-1.703125,
|
| 56 |
+
-1.15625,
|
| 57 |
+
-5.34375
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-5.25,
|
| 61 |
+
0.00927734375,
|
| 62 |
+
-0.5078125,
|
| 63 |
+
-5.0
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-4.9375,
|
| 67 |
+
-1.4609375,
|
| 68 |
+
-1.5703125,
|
| 69 |
+
-5.40625
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-5.5,
|
| 75 |
+
-0.6015625,
|
| 76 |
+
-1.296875,
|
| 77 |
+
-4.46875
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.1875,
|
| 81 |
+
-0.875,
|
| 82 |
+
-0.62109375,
|
| 83 |
+
-5.28125
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
-4.5625,
|
| 87 |
+
-1.890625,
|
| 88 |
+
-1.59375,
|
| 89 |
+
-4.71875
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.5,
|
| 97 |
+
0.498046875,
|
| 98 |
+
0.98828125,
|
| 99 |
+
0.98828125
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.5,
|
| 103 |
+
0.5,
|
| 104 |
+
0.98828125,
|
| 105 |
+
0.984375
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5,
|
| 109 |
+
0.5,
|
| 110 |
+
0.98828125,
|
| 111 |
+
0.98828125
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.498046875,
|
| 117 |
+
0.5,
|
| 118 |
+
1.0,
|
| 119 |
+
1.0
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.5,
|
| 123 |
+
0.5,
|
| 124 |
+
1.0,
|
| 125 |
+
1.0
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.5,
|
| 129 |
+
0.5,
|
| 130 |
+
1.0,
|
| 131 |
+
1.0
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 99895.8203125,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny.eager.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "eager",
|
| 8 |
+
"target": "grounding-dino-tiny",
|
| 9 |
+
"kind": "checkpoint",
|
| 10 |
+
"revision": "a2bb814dd30d776dcf7e30523b00659f4f141c71",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 10.37,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-6.1692609786987305,
|
| 55 |
+
-3.689127206802368,
|
| 56 |
+
-3.5943362712860107,
|
| 57 |
+
-6.244291305541992
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-6.173044681549072,
|
| 61 |
+
-3.7207190990448,
|
| 62 |
+
-3.629878044128418,
|
| 63 |
+
-6.249083995819092
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-6.6102375984191895,
|
| 67 |
+
-3.400014877319336,
|
| 68 |
+
-3.4965169429779053,
|
| 69 |
+
-6.724451065063477
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-6.359555721282959,
|
| 75 |
+
-3.978100061416626,
|
| 76 |
+
-3.9349894523620605,
|
| 77 |
+
-6.351863384246826
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-6.3501296043396,
|
| 81 |
+
-3.893519878387451,
|
| 82 |
+
-3.8278846740722656,
|
| 83 |
+
-6.319794178009033
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
-6.280929088592529,
|
| 87 |
+
-3.7683141231536865,
|
| 88 |
+
-3.6899869441986084,
|
| 89 |
+
-6.225790500640869
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.5006168484687805,
|
| 97 |
+
0.5066757202148438,
|
| 98 |
+
0.9682874083518982,
|
| 99 |
+
0.9596894383430481
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.5014756917953491,
|
| 103 |
+
0.5066835880279541,
|
| 104 |
+
0.9662667512893677,
|
| 105 |
+
0.9586573243141174
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.8563729524612427,
|
| 109 |
+
0.9486265778541565,
|
| 110 |
+
0.07604724913835526,
|
| 111 |
+
0.09199947863817215
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.6176005005836487,
|
| 117 |
+
0.9725996851921082,
|
| 118 |
+
0.1449270099401474,
|
| 119 |
+
0.05400890111923218
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.22222259640693665,
|
| 123 |
+
0.9715834856033325,
|
| 124 |
+
0.16715911030769348,
|
| 125 |
+
0.05648788437247276
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.2188555747270584,
|
| 129 |
+
0.9811375141143799,
|
| 130 |
+
0.13297609984874725,
|
| 131 |
+
0.03684752807021141
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 108446.890625,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/expectations/cuda/grounding-dino-tiny.json
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"recorded_with": {
|
| 3 |
+
"contract": "huggingface-mic/grounding-dino-tiny-mic",
|
| 4 |
+
"contract_name": "GroundingDINO",
|
| 5 |
+
"contract_revision": "1a76dd53b4308ec7777f54d1872f1b1ecb534f01",
|
| 6 |
+
"framework": "transformers",
|
| 7 |
+
"variant": "default",
|
| 8 |
+
"target": "grounding-dino-tiny",
|
| 9 |
+
"kind": "checkpoint",
|
| 10 |
+
"revision": "a2bb814dd30d776dcf7e30523b00659f4f141c71",
|
| 11 |
+
"environment": {
|
| 12 |
+
"python": "3.11.18",
|
| 13 |
+
"transformers": "5.21.0.dev0",
|
| 14 |
+
"torch": "2.14.0+cu130",
|
| 15 |
+
"torchvision": "0.29.1+cu130",
|
| 16 |
+
"torchaudio": "2.11.0+cu130",
|
| 17 |
+
"tokenizers": "0.23.3",
|
| 18 |
+
"numpy": "2.4.6",
|
| 19 |
+
"pillow": "13.0.0",
|
| 20 |
+
"accelerate": "1.16.0.dev0",
|
| 21 |
+
"safetensors": "0.8.0",
|
| 22 |
+
"huggingface-hub": "2.3.0",
|
| 23 |
+
"sentencepiece": "0.2.2",
|
| 24 |
+
"protobuf": "7.36.2",
|
| 25 |
+
"librosa": "0.11.0",
|
| 26 |
+
"soundfile": "0.14.0",
|
| 27 |
+
"scipy": "1.17.1"
|
| 28 |
+
},
|
| 29 |
+
"seconds": 9.56,
|
| 30 |
+
"environment_differs": {
|
| 31 |
+
"torch": "2.14.0+cu130 (pinned 2.11.0)",
|
| 32 |
+
"torchvision": "0.29.1+cu130 (pinned 0.26.0)",
|
| 33 |
+
"tokenizers": "0.23.3 (pinned 0.23.1)",
|
| 34 |
+
"numpy": "2.4.6 (pinned 1.26.4)",
|
| 35 |
+
"pillow": "13.0.0 (pinned 12.2.0)",
|
| 36 |
+
"accelerate": "1.16.0.dev0 (pinned 1.13.0)",
|
| 37 |
+
"huggingface-hub": "2.3.0 (pinned 1.32.0)",
|
| 38 |
+
"sentencepiece": "0.2.2 (pinned 0.2.1)",
|
| 39 |
+
"protobuf": "7.36.2 (pinned 6.33.6)",
|
| 40 |
+
"soundfile": "0.14.0 (pinned 0.13.1)",
|
| 41 |
+
"scipy": "1.17.1 (pinned 1.15.3)"
|
| 42 |
+
},
|
| 43 |
+
"platform": "cuda"
|
| 44 |
+
},
|
| 45 |
+
"outputs": {
|
| 46 |
+
"runtime": {
|
| 47 |
+
"device": "cuda",
|
| 48 |
+
"dtype": "float32",
|
| 49 |
+
"attn_implementation": "eager"
|
| 50 |
+
},
|
| 51 |
+
"class_scores": [
|
| 52 |
+
[
|
| 53 |
+
[
|
| 54 |
+
-5.174252986907959,
|
| 55 |
+
-1.6470273733139038,
|
| 56 |
+
-1.1409422159194946,
|
| 57 |
+
-5.34854793548584
|
| 58 |
+
],
|
| 59 |
+
[
|
| 60 |
+
-5.3730058670043945,
|
| 61 |
+
-0.12157917022705078,
|
| 62 |
+
-0.6255021095275879,
|
| 63 |
+
-5.086806774139404
|
| 64 |
+
],
|
| 65 |
+
[
|
| 66 |
+
-5.0644917488098145,
|
| 67 |
+
-1.600938081741333,
|
| 68 |
+
-1.7390649318695068,
|
| 69 |
+
-5.462441444396973
|
| 70 |
+
]
|
| 71 |
+
],
|
| 72 |
+
[
|
| 73 |
+
[
|
| 74 |
+
-5.531043529510498,
|
| 75 |
+
-0.7425479292869568,
|
| 76 |
+
-1.4596993923187256,
|
| 77 |
+
-4.517359733581543
|
| 78 |
+
],
|
| 79 |
+
[
|
| 80 |
+
-5.097198963165283,
|
| 81 |
+
-0.8415680527687073,
|
| 82 |
+
-0.521429717540741,
|
| 83 |
+
-5.341023921966553
|
| 84 |
+
],
|
| 85 |
+
[
|
| 86 |
+
-4.5713276863098145,
|
| 87 |
+
-1.9387574195861816,
|
| 88 |
+
-1.5383539199829102,
|
| 89 |
+
-4.809746265411377
|
| 90 |
+
]
|
| 91 |
+
]
|
| 92 |
+
],
|
| 93 |
+
"box_preds": [
|
| 94 |
+
[
|
| 95 |
+
[
|
| 96 |
+
0.49990472197532654,
|
| 97 |
+
0.49890732765197754,
|
| 98 |
+
0.9873766899108887,
|
| 99 |
+
0.9869816899299622
|
| 100 |
+
],
|
| 101 |
+
[
|
| 102 |
+
0.5002824068069458,
|
| 103 |
+
0.49952933192253113,
|
| 104 |
+
0.9878783822059631,
|
| 105 |
+
0.9865610003471375
|
| 106 |
+
],
|
| 107 |
+
[
|
| 108 |
+
0.5000370144844055,
|
| 109 |
+
0.4988521635532379,
|
| 110 |
+
0.9886914491653442,
|
| 111 |
+
0.9869803786277771
|
| 112 |
+
]
|
| 113 |
+
],
|
| 114 |
+
[
|
| 115 |
+
[
|
| 116 |
+
0.499994158744812,
|
| 117 |
+
0.4999725818634033,
|
| 118 |
+
1.0,
|
| 119 |
+
0.999998927116394
|
| 120 |
+
],
|
| 121 |
+
[
|
| 122 |
+
0.5002192258834839,
|
| 123 |
+
0.5003373026847839,
|
| 124 |
+
1.0,
|
| 125 |
+
1.0
|
| 126 |
+
],
|
| 127 |
+
[
|
| 128 |
+
0.49988260865211487,
|
| 129 |
+
0.49989035725593567,
|
| 130 |
+
0.9999996423721313,
|
| 131 |
+
0.9999899864196777
|
| 132 |
+
]
|
| 133 |
+
]
|
| 134 |
+
],
|
| 135 |
+
"training": {
|
| 136 |
+
"loss": 101029.6484375,
|
| 137 |
+
"gradients": {
|
| 138 |
+
"model.text_backbone.embeddings.word_embeddings.weight": true,
|
| 139 |
+
"model.decoder.layers.0.self_attn.query.weight": true
|
| 140 |
+
},
|
| 141 |
+
"updated": true
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
}
|
integrations/transformers/usage.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Grounding DINO: zero-shot object detection with Transformers
|
| 2 |
+
|
| 3 |
+
This page is the Transformers contract of the Grounding DINO MIC and also a usage example. [doc-builder](https://github.com/huggingface/doc-builder/blob/main/docs/runnable-code-blocks.md) renders the `runnable` blocks below as documentation, and its pytest plugin runs them as written against `IDEA-Research/grounding-dino-tiny`. The MIC runner runs the same blocks against each slim fixture and each checkpoint in [`integration.json`](../../integration.json). Lines marked `# doc-builder: hide` only matter there: they swap in the pinned checkpoint or variant and print the result record.
|
| 4 |
+
|
| 5 |
+
## Detect objects from text prompts
|
| 6 |
+
|
| 7 |
+
```py runnable:grounding_dino
|
| 8 |
+
import torch
|
| 9 |
+
from PIL import Image
|
| 10 |
+
import numpy as np
|
| 11 |
+
from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection, set_seed
|
| 12 |
+
import json # doc-builder: hide
|
| 13 |
+
import os # doc-builder: hide
|
| 14 |
+
|
| 15 |
+
model_id = "IDEA-Research/grounding-dino-tiny"
|
| 16 |
+
model_id = os.environ.get("MIC_MODEL", model_id) # doc-builder: hide
|
| 17 |
+
variant = json.loads(os.environ.get("MIC_MODEL_KWARGS", "{}")) # doc-builder: hide
|
| 18 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 19 |
+
set_seed(0)
|
| 20 |
+
|
| 21 |
+
processor = AutoProcessor.from_pretrained(model_id)
|
| 22 |
+
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)
|
| 23 |
+
if variant: # doc-builder: hide
|
| 24 |
+
del model # doc-builder: hide
|
| 25 |
+
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id, **variant).to(device) # doc-builder: hide
|
| 26 |
+
|
| 27 |
+
# Two synthetic images with different text prompts (lowercase phrases, each ending with a dot)
|
| 28 |
+
rng = np.random.default_rng(0)
|
| 29 |
+
image1 = Image.fromarray(rng.integers(0, 255, (320, 480, 3), dtype=np.uint8))
|
| 30 |
+
image2 = Image.fromarray(rng.integers(0, 255, (240, 320, 3), dtype=np.uint8))
|
| 31 |
+
text1 = "a cat. a remote control."
|
| 32 |
+
text2 = "a person. a dog."
|
| 33 |
+
|
| 34 |
+
inputs = processor(images=[image1, image2], text=[text1, text2], return_tensors="pt", padding=True)
|
| 35 |
+
inputs = inputs.to(device, dtype=model.dtype) # casts only floating tensors (pixel_values)
|
| 36 |
+
|
| 37 |
+
with torch.no_grad():
|
| 38 |
+
outputs = model(**inputs)
|
| 39 |
+
|
| 40 |
+
# Compact probes for both images: first 3 queries, their first 4 token scores and boxes
|
| 41 |
+
class_scores = outputs.logits[:, :3, :4].float().tolist()
|
| 42 |
+
box_preds = outputs.pred_boxes[:, :3].float().tolist()
|
| 43 |
+
|
| 44 |
+
results = processor.post_process_grounded_object_detection(
|
| 45 |
+
outputs, inputs.input_ids, threshold=0.3, text_threshold=0.25, target_sizes=[image.size[::-1] for image in (image1, image2)]
|
| 46 |
+
)
|
| 47 |
+
print(f"detections per image: {[len(result['boxes']) for result in results]}")
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
## One training step
|
| 51 |
+
|
| 52 |
+
```py runnable:grounding_dino:2
|
| 53 |
+
# Labels: class_labels index the phrases of each prompt; boxes are normalized (cx, cy, w, h)
|
| 54 |
+
labels = [
|
| 55 |
+
{
|
| 56 |
+
"class_labels": torch.tensor([0], dtype=torch.long, device=device), # "a cat"
|
| 57 |
+
"boxes": torch.tensor([[0.5, 0.5, 0.2, 0.3]], dtype=torch.float32, device=device),
|
| 58 |
+
},
|
| 59 |
+
{
|
| 60 |
+
"class_labels": torch.tensor([1], dtype=torch.long, device=device), # "a dog"
|
| 61 |
+
"boxes": torch.tensor([[0.6, 0.6, 0.3, 0.3]], dtype=torch.float32, device=device),
|
| 62 |
+
},
|
| 63 |
+
]
|
| 64 |
+
|
| 65 |
+
model.train()
|
| 66 |
+
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
|
| 67 |
+
# The Hungarian matcher uses torch.cdist, which has no bfloat16 kernel: autocast runs it in float32
|
| 68 |
+
with torch.autocast(device, dtype=model.dtype, enabled=model.dtype != torch.float32):
|
| 69 |
+
loss = model(**inputs, labels=labels).loss
|
| 70 |
+
loss.backward()
|
| 71 |
+
before = {n: p.detach().cpu().clone() for n, p in model.named_parameters() if p.requires_grad} # doc-builder: hide
|
| 72 |
+
optimizer.step()
|
| 73 |
+
print(f"loss: {loss.item():.4f}")
|
| 74 |
+
|
| 75 |
+
if os.environ.get("MIC_RUNNER"): # doc-builder: hide
|
| 76 |
+
named = dict(model.named_parameters())
|
| 77 |
+
checked = ("model.text_backbone.embeddings.word_embeddings.weight", "model.decoder.layers.0.self_attn.query.weight")
|
| 78 |
+
result = {
|
| 79 |
+
"runtime": {
|
| 80 |
+
"device": model.device.type,
|
| 81 |
+
"dtype": str(model.dtype).removeprefix("torch."),
|
| 82 |
+
"attn_implementation": model.config._attn_implementation,
|
| 83 |
+
},
|
| 84 |
+
"class_scores": class_scores,
|
| 85 |
+
"box_preds": box_preds,
|
| 86 |
+
"training": {
|
| 87 |
+
"loss": loss.item(),
|
| 88 |
+
"gradients": {n: bool(named[n].grad is not None and torch.count_nonzero(named[n].grad) > 0 and torch.isfinite(named[n].grad).all()) for n in checked},
|
| 89 |
+
"updated": any(not torch.equal(p.detach().cpu(), before[n]) for n, p in model.named_parameters() if n in before),
|
| 90 |
+
},
|
| 91 |
+
}
|
| 92 |
+
print(json.dumps(result))
|
| 93 |
+
```
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb032709d5b0c9747aa9ba8462d5adab40f26452d99d119c36ea5ad9260bcd3a
|
| 3 |
+
size 30901064
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_normalize": true,
|
| 3 |
+
"do_pad": true,
|
| 4 |
+
"do_rescale": true,
|
| 5 |
+
"do_resize": true,
|
| 6 |
+
"format": "coco_detection",
|
| 7 |
+
"image_mean": [
|
| 8 |
+
0.485,
|
| 9 |
+
0.456,
|
| 10 |
+
0.406
|
| 11 |
+
],
|
| 12 |
+
"image_processor_type": "GroundingDinoImageProcessor",
|
| 13 |
+
"image_std": [
|
| 14 |
+
0.229,
|
| 15 |
+
0.224,
|
| 16 |
+
0.225
|
| 17 |
+
],
|
| 18 |
+
"processor_class": "GroundingDinoProcessor",
|
| 19 |
+
"resample": 2,
|
| 20 |
+
"rescale_factor": 0.00392156862745098,
|
| 21 |
+
"size": {
|
| 22 |
+
"longest_edge": 1333,
|
| 23 |
+
"shortest_edge": 800
|
| 24 |
+
}
|
| 25 |
+
}
|
special_tokens_map.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cls_token": "[CLS]",
|
| 3 |
+
"mask_token": "[MASK]",
|
| 4 |
+
"pad_token": "[PAD]",
|
| 5 |
+
"sep_token": "[SEP]",
|
| 6 |
+
"unk_token": "[UNK]"
|
| 7 |
+
}
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"added_tokens_decoder": {
|
| 3 |
+
"0": {
|
| 4 |
+
"content": "[PAD]",
|
| 5 |
+
"lstrip": false,
|
| 6 |
+
"normalized": false,
|
| 7 |
+
"rstrip": false,
|
| 8 |
+
"single_word": false,
|
| 9 |
+
"special": true
|
| 10 |
+
},
|
| 11 |
+
"100": {
|
| 12 |
+
"content": "[UNK]",
|
| 13 |
+
"lstrip": false,
|
| 14 |
+
"normalized": false,
|
| 15 |
+
"rstrip": false,
|
| 16 |
+
"single_word": false,
|
| 17 |
+
"special": true
|
| 18 |
+
},
|
| 19 |
+
"101": {
|
| 20 |
+
"content": "[CLS]",
|
| 21 |
+
"lstrip": false,
|
| 22 |
+
"normalized": false,
|
| 23 |
+
"rstrip": false,
|
| 24 |
+
"single_word": false,
|
| 25 |
+
"special": true
|
| 26 |
+
},
|
| 27 |
+
"102": {
|
| 28 |
+
"content": "[SEP]",
|
| 29 |
+
"lstrip": false,
|
| 30 |
+
"normalized": false,
|
| 31 |
+
"rstrip": false,
|
| 32 |
+
"single_word": false,
|
| 33 |
+
"special": true
|
| 34 |
+
},
|
| 35 |
+
"103": {
|
| 36 |
+
"content": "[MASK]",
|
| 37 |
+
"lstrip": false,
|
| 38 |
+
"normalized": false,
|
| 39 |
+
"rstrip": false,
|
| 40 |
+
"single_word": false,
|
| 41 |
+
"special": true
|
| 42 |
+
}
|
| 43 |
+
},
|
| 44 |
+
"clean_up_tokenization_spaces": true,
|
| 45 |
+
"cls_token": "[CLS]",
|
| 46 |
+
"do_lower_case": true,
|
| 47 |
+
"mask_token": "[MASK]",
|
| 48 |
+
"model_max_length": 512,
|
| 49 |
+
"pad_token": "[PAD]",
|
| 50 |
+
"processor_class": "GroundingDinoProcessor",
|
| 51 |
+
"sep_token": "[SEP]",
|
| 52 |
+
"strip_accents": null,
|
| 53 |
+
"tokenize_chinese_chars": true,
|
| 54 |
+
"tokenizer_class": "BertTokenizer",
|
| 55 |
+
"unk_token": "[UNK]"
|
| 56 |
+
}
|
vocab.txt
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|