SolusOps/fatima / data /visual_exploration /numbered_records.json
SolusOps's picture
download
raw
7.87 kB
[
{
"instruction": "put the cream cheese in the bowl",
"prompt": "Image1 is the agent camera; image2 is the wrist camera of the SAME robot scene. Each white numeric label has a yellow line ending on its candidate object. The same ID means the same object in both views. Inspect BOTH images to select IDs; never guess IDs from object names in the instruction. Labels are arbitrary and do not name objects. Some candidates may be visible in only one view.\nIdentify the requested operations in order. For each operation choose the ID of the object to manipulate and the destination OBJECT ID, if relocation is requested. Describe the requested destination surface/interior separately. Do not provide pixel coordinates. Use null if the relevant object is not visible in either image, or if no destination applies. Do not invent a hidden interior: its visible parent can still be selected, while its visibility should be described accurately.\nAllowed candidate IDs: 1, 2, 3, 4, 5, 6, 7.\nReturn only JSON with this structure:\n{\"steps\":[{\"action\":\"...\",\"object_name\":\"...\",\"target_id\":0,\"destination_id\":0,\"relationship\":\"...\",\"destination_component\":\"...\"}],\"appearance\":{\"target\":\"brief actual visible appearance\",\"destination\":\"brief actual visible appearance\"},\"coarse_locations\":{\"target\":\"brief coarse region or visible-object relationship\",\"destination\":\"brief coarse region or visible-object relationship\"}}\naction: open, close, place, push, pick_up, turn_on, turn_off, other.\nobject_name: a short English name; this name alone is NOT visual evidence.\nrelationship: none, on_top_of, inside, in_front_of, other.\ndestination_component: none, interior, support_surface, drawer_front, control, other.\nInstruction: put the cream cheese in the bowl",
"response": "{\n \"steps\": [\n {\n \"action\": \"pick_up\",\n \"object_name\": \"cream cheese\",\n \"target_id\": 5,\n \"destination_id\": 6,\n \"relationship\": \"on_top_of\",\n \"destination_component\": \"support_surface\"\n }\n ],\n \"appearance\": {\n \"target\": \"a small, round, white object with a yellow line pointing to it\",\n \"destination\": \"a dark, round bowl with a yellow line pointing to it\"\n },\n \"coarse_locations\": {\n \"target\": \"on the table, near a black object\",\n \"destination\": \"on the table, in front of a black object\"\n }\n}",
"language": "english",
"image_files": [
"images/numbered/agentview.png",
"images/numbered/wrist.png"
]
},
{
"instruction": "क्रीम चीज़ को बाउल में रखो।",
"prompt": "Image1 is the agent camera; image2 is the wrist camera of the SAME robot scene. Each white numeric label has a yellow line ending on its candidate object. The same ID means the same object in both views. Inspect BOTH images to select IDs; never guess IDs from object names in the instruction. Labels are arbitrary and do not name objects. Some candidates may be visible in only one view.\nIdentify the requested operations in order. For each operation choose the ID of the object to manipulate and the destination OBJECT ID, if relocation is requested. Describe the requested destination surface/interior separately. Do not provide pixel coordinates. Use null if the relevant object is not visible in either image, or if no destination applies. Do not invent a hidden interior: its visible parent can still be selected, while its visibility should be described accurately.\nAllowed candidate IDs: 1, 2, 3, 4, 5, 6, 7.\nReturn only JSON with this structure:\n{\"steps\":[{\"action\":\"...\",\"object_name\":\"...\",\"target_id\":0,\"destination_id\":0,\"relationship\":\"...\",\"destination_component\":\"...\"}],\"appearance\":{\"target\":\"brief actual visible appearance\",\"destination\":\"brief actual visible appearance\"},\"coarse_locations\":{\"target\":\"brief coarse region or visible-object relationship\",\"destination\":\"brief coarse region or visible-object relationship\"}}\naction: open, close, place, push, pick_up, turn_on, turn_off, other.\nobject_name: a short English name; this name alone is NOT visual evidence.\nrelationship: none, on_top_of, inside, in_front_of, other.\ndestination_component: none, interior, support_surface, drawer_front, control, other.\nInstruction: क्रीम चीज़ को बाउल में रखो।",
"response": "{\n \"steps\": [\n {\n \"action\": \"pick_up\",\n \"object_name\": \"cream cheese\",\n \"target_id\": 5,\n \"destination_id\": 6,\n \"relationship\": \"on_top_of\",\n \"destination_component\": \"support_surface\"\n }\n ],\n \"appearance\": {\n \"target\": \"a small, round, white object with a yellow line pointing to it, located on a dark surface\",\n \"destination\": \"a dark, round bowl with a yellow line pointing to it, located on a light-colored surface\"\n },\n \"coarse_locations\": {\n \"target\": \"cream cheese is on a dark surface, near a black object\",\n \"destination\": \"bowl is on a light-colored surface, near a black object\"\n }\n}",
"language": "hindi",
"image_files": [
"images/numbered/agentview.png",
"images/numbered/wrist.png"
]
},
{
"instruction": "Cream cheese ko bowl ke andar rakho.",
"prompt": "Image1 is the agent camera; image2 is the wrist camera of the SAME robot scene. Each white numeric label has a yellow line ending on its candidate object. The same ID means the same object in both views. Inspect BOTH images to select IDs; never guess IDs from object names in the instruction. Labels are arbitrary and do not name objects. Some candidates may be visible in only one view.\nIdentify the requested operations in order. For each operation choose the ID of the object to manipulate and the destination OBJECT ID, if relocation is requested. Describe the requested destination surface/interior separately. Do not provide pixel coordinates. Use null if the relevant object is not visible in either image, or if no destination applies. Do not invent a hidden interior: its visible parent can still be selected, while its visibility should be described accurately.\nAllowed candidate IDs: 1, 2, 3, 4, 5, 6, 7.\nReturn only JSON with this structure:\n{\"steps\":[{\"action\":\"...\",\"object_name\":\"...\",\"target_id\":0,\"destination_id\":0,\"relationship\":\"...\",\"destination_component\":\"...\"}],\"appearance\":{\"target\":\"brief actual visible appearance\",\"destination\":\"brief actual visible appearance\"},\"coarse_locations\":{\"target\":\"brief coarse region or visible-object relationship\",\"destination\":\"brief coarse region or visible-object relationship\"}}\naction: open, close, place, push, pick_up, turn_on, turn_off, other.\nobject_name: a short English name; this name alone is NOT visual evidence.\nrelationship: none, on_top_of, inside, in_front_of, other.\ndestination_component: none, interior, support_surface, drawer_front, control, other.\nInstruction: Cream cheese ko bowl ke andar rakho.",
"response": "{\n \"steps\": [\n {\n \"action\": \"pick_up\",\n \"object_name\": \"cream cheese\",\n \"target_id\": 5,\n \"destination_id\": 6,\n \"relationship\": \"on_top_of\",\n \"destination_component\": \"support_surface\"\n }\n ],\n \"appearance\": {\n \"target\": \"a small, round, white container with a lid, placed on a wooden surface\",\n \"destination\": \"a dark, round bowl with a textured surface, placed on a wooden surface\"\n },\n \"coarse_locations\": {\n \"target\": \"on a wooden table\",\n \"destination\": \"on a wooden table\"\n }\n}",
"language": "hinglish",
"image_files": [
"images/numbered/agentview.png",
"images/numbered/wrist.png"
]
}
]

Xet Storage Details

Size:
7.87 kB
·
Xet hash:
f74bf5a2b44f85ec458d0cd39576c4debc3baef3cdc2679309a5afcbab74c7ac

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.