Buckets:
| { | |
| "description": "Where the entity object sits in the rendered image, in both pixel space and VLM merged-image-token space. All 100 images in images/ share this exact geometry (single fixed template render: 336x336 canvas, entity segmented and fitted inside a 112x112 box placed at (112,112)) -- one shared config, not per-image, because only the entity's pixel content varies between images, never its box position or size. Aspect ratio is PRESERVED inside the box rather than stretched to fill it -- so some entity-box cells are background pixels for a given item even though the box itself is identical across all items. Box shrunk from the original 168x168 @ (84,84) by scripts/recrop_object.py -- the old box put entity_ring1 at 44.4% of the image's merged tokens, which was saturating DAS cause/iso accuracy near-ceiling from the first swept layer; see git history for the pre-shrink object_location.json / images.", | |
| "canvas_px": { | |
| "width": 336, | |
| "height": 336 | |
| }, | |
| "object_bbox_px": { | |
| "x": 112, | |
| "y": 112, | |
| "width": 112, | |
| "height": 112, | |
| "note": "top-left origin, [x, x+width) x [y, y+height) half-open. Longest side of the segmented animal is fit inside this box; remainder padded with the canvas's neutral gray (128,128,128), not stretched." | |
| }, | |
| "vlm_token_grid": { | |
| "model": "Qwen/Qwen2.5-VL-7B-Instruct", | |
| "patch_size_px": 14, | |
| "merge_size": 2, | |
| "merged_patch_size_px": 28, | |
| "grid_rows": 12, | |
| "grid_cols": 12, | |
| "n_tokens": 144, | |
| "flat_index_formula": "row * grid_cols + col", | |
| "note": "object_bbox_px falls exactly on merged-patch-grid boundaries (84/28=3, 252/28=9), so the token sets below are an exact, non-partial cover of the object's box -- verified by assertions in scripts/render_config.py." | |
| }, | |
| "object_token_indices": { | |
| "entity_only": { | |
| "description": "The 16 merged tokens the entity box occupies (rows 4-7, cols 4-7).", | |
| "rows": [ | |
| 4, | |
| 5, | |
| 6, | |
| 7 | |
| ], | |
| "cols": [ | |
| 4, | |
| 5, | |
| 6, | |
| 7 | |
| ], | |
| "flat": [ | |
| 52, | |
| 53, | |
| 54, | |
| 55, | |
| 64, | |
| 65, | |
| 66, | |
| 67, | |
| 76, | |
| 77, | |
| 78, | |
| 79, | |
| 88, | |
| 89, | |
| 90, | |
| 91 | |
| ] | |
| }, | |
| "entity_ring1": { | |
| "description": "entity_only dilated by one merged-token cell in every direction (rows 3-8, cols 3-8) -- entity_only is a strict subset.", | |
| "rows": [ | |
| 3, | |
| 4, | |
| 5, | |
| 6, | |
| 7, | |
| 8 | |
| ], | |
| "cols": [ | |
| 3, | |
| 4, | |
| 5, | |
| 6, | |
| 7, | |
| 8 | |
| ], | |
| "flat": [ | |
| 39, | |
| 40, | |
| 41, | |
| 42, | |
| 43, | |
| 44, | |
| 51, | |
| 52, | |
| 53, | |
| 54, | |
| 55, | |
| 56, | |
| 63, | |
| 64, | |
| 65, | |
| 66, | |
| 67, | |
| 68, | |
| 75, | |
| 76, | |
| 77, | |
| 78, | |
| 79, | |
| 80, | |
| 87, | |
| 88, | |
| 89, | |
| 90, | |
| 91, | |
| 92, | |
| 99, | |
| 100, | |
| 101, | |
| 102, | |
| 103, | |
| 104 | |
| ] | |
| } | |
| }, | |
| "models": { | |
| "InternVL3-8B-hf": { | |
| "vlm_token_grid": { | |
| "model": "OpenGVLab/InternVL3-8B-hf", | |
| "patch_size_px": 14, | |
| "merge_size": 2, | |
| "merged_patch_size_px": 28, | |
| "grid_rows": 16, | |
| "grid_cols": 16, | |
| "n_tokens": 256, | |
| "flat_index_formula": "row * grid_cols + col", | |
| "note": "Unlike Qwen2.5-VL, this grid is a MODEL constant, not a function of the entity's 336x336 canvas: InternVL3-8B-hf's image processor resizes every image to one fixed 448x448 tile (crop_to_patches=False, confirmed against this checkpoint's actual preprocessor config, not just the processor class's own documented default) before patchifying at patch_size=14 with a 2x2 pixel-shuffle merge (downsample_ratio=0.5), giving a fixed 16x16/256-token grid regardless of canvas size. patch_size_px/merged_patch_size_px above are in RESIZED-448px space, not the original 336px canvas -- they don't relate to object_bbox_px the way Qwen2.5-VL's do." | |
| }, | |
| "object_token_indices": { | |
| "entity_ring1": { | |
| "description": "Smallest whole-cell 16x16-grid box CONTAINING object_bbox_px after the 336->448 resize (rows 5-10, cols 5-10) -- NOT a pixel-exact cover the way Qwen2.5-VL's entity_only/entity_ring1 are. 336 and 448 don't share a common alignment at this grid's 28px-in-resized-space cell stride (448/336 resize factor is 4/3, so the bbox's resized edges at 149.33px/298.67px fall mid-cell, not on a cell boundary), so any whole-cell box necessarily includes some background margin -- verified by resizing two 336x336 synthetic images differing only inside object_bbox_px through the real InternVL3-8B-hf image processor (crop_to_patches confirmed off) and diffing pixel_values: changed pixels span [147,300]x[147,300] in the resized 448x448 image, matching the predicted [149.33,298.67) up to the bicubic resample filter's edge blur. No separate entity_only variant is defined for this model -- sub-cell precision isn't achievable at this resize ratio, so this repo's only intervention position set for InternVL3-8B-hf on this entity is this single containing box (deliberately given the SAME name, entity_ring1, as Qwen2.5-VL's dilated set, so --positions entity_ring1 replicates the same experimental CHOICE across models even though the underlying box isn't derived the same way).", | |
| "rows": [5, 6, 7, 8, 9, 10], | |
| "cols": [5, 6, 7, 8, 9, 10], | |
| "flat": [85, 86, 87, 88, 89, 90, 101, 102, 103, 104, 105, 106, 117, 118, 119, 120, 121, 122, | |
| 133, 134, 135, 136, 137, 138, 149, 150, 151, 152, 153, 154, 165, 166, 167, 168, 169, 170] | |
| } | |
| } | |
| } | |
| }, | |
| "blank_control_image": "images/_BLANK.png" | |
| } |
Xet Storage Details
- Size:
- 5.81 kB
- Xet hash:
- 70e234025028494b5898488d2ec2f90ad54a0cdd1c165f894e966028ea650b47
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.