Buckets:
BluStatic/DARK-MATTER / datasets /shared /Model-Training /Intermediate /20_vision_and_graphical_interfaces /validation.jsonl
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/43d59ca701b23c88afed6f6863942e7ea4d63bfff77b52f27df26e8039f52a1f.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What happens when you tap position <ref>(33,30) on the screen?"]},{"role":"assistant","content":"{\"answer\": \"This element serves as a navigational tool for users to access the article associated with the headline \\\"Mexican voters about to elect first woman president\\\". It provides users with direct access to the article's content and contextual information, enhancing their understanding of the news event.\", \"bounding_box\": [38.9375, 195.0, 805.5078125, 235.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"43d59ca701b23c88afed6f6863942e7ea4d63bfff77b52f27df26e8039f52a1f","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/43d59ca701b23c88afed6f6863942e7ea4d63bfff77b52f27df26e8039f52a1f.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/052b23c6e580eb9604ef888d0aa54c0cc8133ac290c725057f09165b55a2c7d8.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: On this webpage screenshot, I will point out an element; please predict their exact coordinates (with point). This element provides access to a personalized eBay dashboard for the user."]},{"role":"assistant","content":"{\"answer\": \"(83,2)\", \"bounding_box\": [1027.3125, 3.25, 1107.0, 27.25], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"052b23c6e580eb9604ef888d0aa54c0cc8133ac290c725057f09165b55a2c7d8","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/052b23c6e580eb9604ef888d0aa54c0cc8133ac290c725057f09165b55a2c7d8.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/ee189421863005b12ff6830cddb897a6b29c05fed11b48b57b57f6a2f30575a2.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: I'll provide textual descriptions of the element in this webpage screenshot. Can you find their coordinates (with point)? This element triggers a navigational experience that provides further details about the Copilot feature, enhancing user understanding and allowing them to make an informed decision about utilizing it."]},{"role":"assistant","content":"{\"answer\": \"(50,36)\", \"bounding_box\": [155.875, 247.1953125, 272.125, 287.1953125], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"ee189421863005b12ff6830cddb897a6b29c05fed11b48b57b57f6a2f30575a2","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/ee189421863005b12ff6830cddb897a6b29c05fed11b48b57b57f6a2f30575a2.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/e95e53a2ee404ac3bfb54802a7ca93ccf90289cdf22cf2b4b84f64f1fc7b4dbc.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Based on this screenshot, I'll describe an element. Please pinpoint their exact locations (with point). This element serves to provide the date when the latest reply in the given topic was posted, allowing users to understand the activity level and the freshness of the conversation."]},{"role":"assistant","content":"{\"answer\": \"(90,49)\", \"bounding_box\": [357.109375, 355.46875, 417.28125, 373.46875], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"e95e53a2ee404ac3bfb54802a7ca93ccf90289cdf22cf2b4b84f64f1fc7b4dbc","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/e95e53a2ee404ac3bfb54802a7ca93ccf90289cdf22cf2b4b84f64f1fc7b4dbc.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/ec07485ae104cb9480b795debb20c1a55eae81cb45385885a87e459b92e1f638.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What feature is accessed by selecting the location at <ref>(7,18)?"]},{"role":"assistant","content":"{\"answer\": \"This element provides access to additional navigation options, allowing users to explore related categories and refine their search.\", \"bounding_box\": [72.6875, 121.0, 96.6875, 145.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"ec07485ae104cb9480b795debb20c1a55eae81cb45385885a87e459b92e1f638","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/ec07485ae104cb9480b795debb20c1a55eae81cb45385885a87e459b92e1f638.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/36a16db0cab7449d02255d1fe8fd5dbaffd849bcec02b0c116a6b65d31c104bf.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Based on my descriptions, find the locations of the mentioned element in this webpage screenshot (with point). This element triggers a screen transition to manage unrestricted data access for various apps, allowing users to configure which apps are allowed to use unrestricted data."]},{"role":"assistant","content":"{\"answer\": \"(50,64)\", \"bounding_box\": [0.0, 413.0, 1280.0, 519.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"36a16db0cab7449d02255d1fe8fd5dbaffd849bcec02b0c116a6b65d31c104bf","device":"mobile","source_dataset":"Android Emulator","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/36a16db0cab7449d02255d1fe8fd5dbaffd849bcec02b0c116a6b65d31c104bf.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/55499c400b186fd3141f6decbabea6aefce303cf6a169ef7fe2c856e3549e198.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: From this webpage image, I will describe a specific element. Please predict their exact locations (with point). This element triggers a navigation to a different section or webpage, allowing users to opt for switching to the Bing service and accessing GPT-4."]},{"role":"assistant","content":"{\"answer\": \"(74,95)\", \"bounding_box\": [263.5, 692.0, 367.0, 732.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"55499c400b186fd3141f6decbabea6aefce303cf6a169ef7fe2c856e3549e198","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/55499c400b186fd3141f6decbabea6aefce303cf6a169ef7fe2c856e3549e198.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/94e1abcfb7b0f8fcf5bddc932643a3abc08cd56158ba7c8488025903a8fb3383.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What is the significance of the element located at <ref>(97,39) in the application?"]},{"role":"assistant","content":"{\"answer\": \"This element enables users to zoom in on a specific location on the map, allowing for a more detailed view.\", \"bounding_box\": [1224.0, 262.0, 1264.0, 302.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"94e1abcfb7b0f8fcf5bddc932643a3abc08cd56158ba7c8488025903a8fb3383","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/94e1abcfb7b0f8fcf5bddc932643a3abc08cd56158ba7c8488025903a8fb3383.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/5bbf4055bf78a31a39bb8b4e5bfb03095c261ca255f983c0e9bece48e1c89784.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Describe the function of the element at <ref>(4,7) on the screen."]},{"role":"assistant","content":"{\"answer\": \"This element acts as a navigation link, directing users to the main or starting page of the website when clicked.\", \"bounding_box\": [20.0, 16.0, 84.0, 80.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"5bbf4055bf78a31a39bb8b4e5bfb03095c261ca255f983c0e9bece48e1c89784","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/5bbf4055bf78a31a39bb8b4e5bfb03095c261ca255f983c0e9bece48e1c89784.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/2081a0d33f2a8502c0c0819f788fdd8ba503002d64713fd722575705e5ede1c0.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the image of this webpage, can you determine the coordinates of the element I describe (with point)? This element triggers a search query, allowing users to find specific content within the platform."]},{"role":"assistant","content":"{\"answer\": \"(97,22)\", \"bounding_box\": [1234.0, 148.1953125, 1250.0, 163.1953125], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"2081a0d33f2a8502c0c0819f788fdd8ba503002d64713fd722575705e5ede1c0","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/2081a0d33f2a8502c0c0819f788fdd8ba503002d64713fd722575705e5ede1c0.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/78a4cc8eb95b57551aa4b644dba7239302843add49a3a2386475d03e7a540e8b.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What feature is accessed by selecting the location at <ref>(8,70)?"]},{"role":"assistant","content":"{\"answer\": \"This element provides access to information about intellectual property rights and copyright policies.\", \"bounding_box\": [40.0, 496.0, 155.6328125, 517.5], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"78a4cc8eb95b57551aa4b644dba7239302843add49a3a2386475d03e7a540e8b","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/78a4cc8eb95b57551aa4b644dba7239302843add49a3a2386475d03e7a540e8b.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/fcf439fd1a330228d5e5323c4c3013d323747d65f3f806068872c2c3870b3591.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the screenshot of this webpage, identify the coordinates of the element I describe (with point). This element advances to the next tutorial slide."]},{"role":"assistant","content":"{\"answer\": \"(93,96)\", \"bounding_box\": [948.0, 2137.0, 1080.0, 2269.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"fcf439fd1a330228d5e5323c4c3013d323747d65f3f806068872c2c3870b3591","device":"mobile","source_dataset":"Android Emulator","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/fcf439fd1a330228d5e5323c4c3013d323747d65f3f806068872c2c3870b3591.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/5deae00e268861b1019ecfb58f01f363b74f2d4ea9a14dd0d313b43571c008c2.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the screenshot of this webpage, identify the coordinates of the element I describe (with point). This element serves as a navigation tool, directing users to a particular thread within the 'Practice Advice' section of the community forums."]},{"role":"assistant","content":"{\"answer\": \"(25,99)\", \"bounding_box\": [86.0, 736.09375, 124.65625, 745.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"5deae00e268861b1019ecfb58f01f363b74f2d4ea9a14dd0d313b43571c008c2","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/5deae00e268861b1019ecfb58f01f363b74f2d4ea9a14dd0d313b43571c008c2.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/5111d51f60cafe5bd9c298b8403589f7dc0fac1551ea6377469f9b078c555d19.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What is the function of the element at <ref>(90,18) on the UI?"]},{"role":"assistant","content":"{\"answer\": \"This element triggers a live streaming service for users to watch ongoing sports events or programming, providing immediate access to live sports content.\", \"bounding_box\": [1049.9921875, 84.0546875, 1249.9921875, 174.0546875], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"5111d51f60cafe5bd9c298b8403589f7dc0fac1551ea6377469f9b078c555d19","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/5111d51f60cafe5bd9c298b8403589f7dc0fac1551ea6377469f9b078c555d19.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/9b5fcb7cd2d9ad51d4c8bedc82eee0c90df5044f3c70b9f7410d00cb976debe7.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What kind of input or interaction is expected at the point marked (50,10)?"]},{"role":"assistant","content":"{\"answer\": \"This element expands the search interface for searching contacts and places, with advanced features to allow location access for place-based results.\", \"bounding_box\": [21.0, 43.0, 1259.0, 107.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"9b5fcb7cd2d9ad51d4c8bedc82eee0c90df5044f3c70b9f7410d00cb976debe7","device":"mobile","source_dataset":"Android Emulator","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/9b5fcb7cd2d9ad51d4c8bedc82eee0c90df5044f3c70b9f7410d00cb976debe7.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/b353b471c593593056769d7b3f767a991a944d111608e8c5c1380062c431448c.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What happens when you click point <ref>(22,96) on the screen?"]},{"role":"assistant","content":"{\"answer\": \"This element allows users to access the privacy policy of the website, providing them with information about data protection, user rights, and the website's privacy practices.\", \"bounding_box\": [72.671875, 706.0, 112.015625, 719.5], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"b353b471c593593056769d7b3f767a991a944d111608e8c5c1380062c431448c","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/b353b471c593593056769d7b3f767a991a944d111608e8c5c1380062c431448c.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/ee05d914ae6c41e1c7580be4c951868b81b5f5309b4283e72a2cba8ca78cb840.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the provided screenshot, I'll describe a webpage element for you to locate (with point). This element triggers a primary action related to the game, potentially allowing users to engage with the game's core functionality."]},{"role":"assistant","content":"{\"answer\": \"(38,3)\", \"bounding_box\": [469.6640625, 6.0, 502.6640625, 34.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"ee05d914ae6c41e1c7580be4c951868b81b5f5309b4283e72a2cba8ca78cb840","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/ee05d914ae6c41e1c7580be4c951868b81b5f5309b4283e72a2cba8ca78cb840.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/827436aa8694068812625db50f3260637975582ed8d77600ecf381ffad70e9fd.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Please determine the location of the element I describe in this webpage screenshot (with point). This element allows users to access and view a specific user's profile page on Reddit, displaying their content, profile information, and interaction options."]},{"role":"assistant","content":"{\"answer\": \"(27,80)\", \"bounding_box\": [300.0, 561.0, 395.3203125, 589.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"827436aa8694068812625db50f3260637975582ed8d77600ecf381ffad70e9fd","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/827436aa8694068812625db50f3260637975582ed8d77600ecf381ffad70e9fd.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/dda9e3f7507d223e62d1564cd248482b9054e517789f338fb6f1ee76f792a9ec.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What action is triggered by interacting with the area at <ref>(46,18)?"]},{"role":"assistant","content":"{\"answer\": \"This element serves as the label for one of the radio buttons, helping users understand the purpose of each option in the given context. When a user interacts with this radio button, it allows them to express their intention to transfer a domain name on the given webpage.\", \"bounding_box\": [163.3203125, 124.0, 234.234375, 148.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"dda9e3f7507d223e62d1564cd248482b9054e517789f338fb6f1ee76f792a9ec","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/dda9e3f7507d223e62d1564cd248482b9054e517789f338fb6f1ee76f792a9ec.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/934e704f79632d7eb805f71989fae798da3798b3d6a7e4b98cb1ebe62ce20eae.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the attached screenshot of a webpage, please find the coordinates of the described element (with point). This element serves as a navigational link, providing access to a webpage dedicated to the '羅默森王子' entity, offering users more in-depth information about it."]},{"role":"assistant","content":"{\"answer\": \"(31,87)\", \"bounding_box\": [362.546875, 621.9765625, 430.046875, 636.9765625], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"934e704f79632d7eb805f71989fae798da3798b3d6a7e4b98cb1ebe62ce20eae","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/934e704f79632d7eb805f71989fae798da3798b3d6a7e4b98cb1ebe62ce20eae.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/15bec8c6700374dc552e18aa4f794e3fcf93ce31570d75638e533854a430a6e8.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the attached screenshot of a webpage, please find the coordinates of the described element (with point). This element provides access to a fashion news and information archive, allowing users to explore and stay updated on the latest fashion trends, news, and analysis, with features to filter and discover content by category, tag, or brand."]},{"role":"assistant","content":"{\"answer\": \"(5,34)\", \"bounding_box\": [47.5, 241.9921875, 91.5078125, 252.9921875], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"15bec8c6700374dc552e18aa4f794e3fcf93ce31570d75638e533854a430a6e8","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/15bec8c6700374dc552e18aa4f794e3fcf93ce31570d75638e533854a430a6e8.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/c52ef7b8f216191d37aae07f676b88391e418b8174d6556b701d331bebd68b1a.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: In this web page image, please locate the element as I describe it (with point). This element provides access to information about how the service collaborates with other platforms and services."]},{"role":"assistant","content":"{\"answer\": \"(8,69)\", \"bounding_box\": [0.0, 466.1484375, 197.2421875, 528.5390625], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"c52ef7b8f216191d37aae07f676b88391e418b8174d6556b701d331bebd68b1a","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/c52ef7b8f216191d37aae07f676b88391e418b8174d6556b701d331bebd68b1a.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/983967d498cc4f434ba50f7652f58a475f2be62863b967507cca31bf84621646.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What is the function of the element at <ref>(50,18) on the UI?"]},{"role":"assistant","content":"{\"answer\": \"This element provides access to a related resource or section of the forum.\", \"bounding_box\": [382.5, 0.0, 887.5, 260.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"983967d498cc4f434ba50f7652f58a475f2be62863b967507cca31bf84621646","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/983967d498cc4f434ba50f7652f58a475f2be62863b967507cca31bf84621646.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/b5ca07e15ad5419cac4cb9abec7584ded7f1bcf653c9dbaed47c89ccc8429d2d.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What kind of input or interaction is expected at the point marked <ref>(92,10)?"]},{"role":"assistant","content":"{\"answer\": \"This element initiates the process for users to register for the Airflow Summit 2024 or access more information about the event.\", \"bounding_box\": [382.0, 67.9609375, 408.0, 87.9609375], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"b5ca07e15ad5419cac4cb9abec7584ded7f1bcf653c9dbaed47c89ccc8429d2d","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/b5ca07e15ad5419cac4cb9abec7584ded7f1bcf653c9dbaed47c89ccc8429d2d.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/4d5db23ef29b4e218b9cb4cbc3c113868feffaefe909a77ea7df50f5bbdb22d4.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Using the provided screenshot, I'll describe a webpage element for you to locate (with point). This element provides access to a submenu of Mac-related content and resources."]},{"role":"assistant","content":"{\"answer\": \"(23,13)\", \"bounding_box\": [284.328125, 70.0, 306.328125, 114.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"4d5db23ef29b4e218b9cb4cbc3c113868feffaefe909a77ea7df50f5bbdb22d4","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/4d5db23ef29b4e218b9cb4cbc3c113868feffaefe909a77ea7df50f5bbdb22d4.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/99ce546cfab3956b8431556f5dedabac4852009aa806ee675ff060c1031408f5.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Describe the function of the element at <ref>(30,3) on the screen."]},{"role":"assistant","content":"{\"answer\": \"This element directs users to a specific section or category of the website, providing access to relevant news, articles, and information.\", \"bounding_box\": [365.9296875, 0.0, 410.9453125, 40.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"99ce546cfab3956b8431556f5dedabac4852009aa806ee675ff060c1031408f5","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/99ce546cfab3956b8431556f5dedabac4852009aa806ee675ff060c1031408f5.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/f259bb2414aed06910255a89f41f3777e713ff9094493562f4d432f875aa56b7.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What is the outcome of selecting the element at <ref>(10,12)?"]},{"role":"assistant","content":"{\"answer\": \"This element provides access to data sending functionality, allowing users to configure and manage data transmission to Sumo Logic.\", \"bounding_box\": [8.796875, 74.796875, 238.0, 101.296875], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"f259bb2414aed06910255a89f41f3777e713ff9094493562f4d432f875aa56b7","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/f259bb2414aed06910255a89f41f3777e713ff9094493562f4d432f875aa56b7.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/34be0bd489aca3cffd3bf8617300809d514757e5a74f7cfc3b1bd9ebc2d3cb0a.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Based on my descriptions, find the locations of the mentioned element in this webpage screenshot (with point). This element serves as a navigation link to the section of the news site, providing users with access to articles related to business-related news stories."]},{"role":"assistant","content":"{\"answer\": \"(52,4)\", \"bounding_box\": [623.1328125, 11.5, 702.3046875, 45.5], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"34be0bd489aca3cffd3bf8617300809d514757e5a74f7cfc3b1bd9ebc2d3cb0a","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/34be0bd489aca3cffd3bf8617300809d514757e5a74f7cfc3b1bd9ebc2d3cb0a.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/097a76b659d457def3fcf14f833c7c4d246b7df1a1c466bcb7b1785a76a7f1db.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: I will describe an element from a webpage screenshot; your role is to locate it (with point). This element triggers the display or hiding of a side navigation menu or collapsible content section, contributing to the navigation experience of the webpage."]},{"role":"assistant","content":"{\"answer\": \"(7,3)\", \"bounding_box\": [8.5546875, 0.0, 49.2734375, 40.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"097a76b659d457def3fcf14f833c7c4d246b7df1a1c466bcb7b1785a76a7f1db","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/097a76b659d457def3fcf14f833c7c4d246b7df1a1c466bcb7b1785a76a7f1db.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/61d944402d1ddf0085da1949de06ef7aa0fb2e5052d75ed218ad861152a44e6e.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: In this webpage capture, I will describe a certain element. Please locate it for me (with point). This element allows users to join or participate in a community or subreddit."]},{"role":"assistant","content":"{\"answer\": \"(67,49)\", \"bounding_box\": [838.328125, 339.0, 887.0, 363.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"61d944402d1ddf0085da1949de06ef7aa0fb2e5052d75ed218ad861152a44e6e","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/61d944402d1ddf0085da1949de06ef7aa0fb2e5052d75ed218ad861152a44e6e.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/e5c7521f57b79f7aef5b65f2ed162d39d1ca9897d3dd0d93aea5d5ad2238c01f.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: On this webpage screenshot, I will point out an element; please predict their exact coordinates (with point). This element opens a detailed configuration and information page about the Bluetooth app, showing its usage statistics, permissions, and storage information."]},{"role":"assistant","content":"{\"answer\": \"(50,52)\", \"bounding_box\": [309.0, 347.0, 971.0, 411.0], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"e5c7521f57b79f7aef5b65f2ed162d39d1ca9897d3dd0d93aea5d5ad2238c01f","device":"mobile","source_dataset":"Android Emulator","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/e5c7521f57b79f7aef5b65f2ed162d39d1ca9897d3dd0d93aea5d5ad2238c01f.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/781909939fda1323bb10600a70bb07500a7c64be5baa71cf23e61ec9379f8156.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What action is triggered by interacting with the area at <ref>(79,2)?"]},{"role":"assistant","content":"{\"answer\": \"This element serves as a navigation tool, allowing users to access their Watchlist, potentially displaying items they have previously saved or been interested in on eBay.\", \"bounding_box\": [972.078125, 3.25, 1053.3125, 27.25], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"781909939fda1323bb10600a70bb07500a7c64be5baa71cf23e61ec9379f8156","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/781909939fda1323bb10600a70bb07500a7c64be5baa71cf23e61ec9379f8156.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/e493d075dd1e0eef1686da014a7980b8e31caf01d98fd22804854ef3934cb921.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: What kind of input or interaction is expected at the point marked <ref>(12,3)?"]},{"role":"assistant","content":"{\"answer\": \"This element provides access to the main or homepage of the Apple website.\", \"bounding_box\": [134.5, 0.0, 164.5, 44.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"e493d075dd1e0eef1686da014a7980b8e31caf01d98fd22804854ef3934cb921","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/e493d075dd1e0eef1686da014a7980b8e31caf01d98fd22804854ef3934cb921.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/221dde56a58214fa7f05a213a553013854a98d31d4516683ef7232e9f3d97360.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: In the provided web page image, I'll describe a specific element. Identify their locations, please (with point). This element serves as a user input field for the user's last name in the context of the sign-up form for a free Salesforce trial. Users are required to enter their last name before proceeding with the sign-up process."]},{"role":"assistant","content":"{\"answer\": \"(50,58)\", \"bounding_box\": [36.0, 413.7890625, 392.0, 445.7890625], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"221dde56a58214fa7f05a213a553013854a98d31d4516683ef7232e9f3d97360","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/221dde56a58214fa7f05a213a553013854a98d31d4516683ef7232e9f3d97360.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/3296e47ff3fa26eaf8c72f0b80bcd1431a4a325be76d22591641bc01c035951b.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Based on this screenshot, I'll describe an element. Please pinpoint their exact locations (with point). This element is a link that, when clicked, is likely to direct users to other related content or pages on the website, possibly featuring articles or resources about systems, particularly in the context of technology, policy, strategy, or China."]},{"role":"assistant","content":"{\"answer\": \"(78,27)\", \"bounding_box\": [298.4453125, 192.7890625, 368.8671875, 216.78125], \"device\": \"mobile\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"3296e47ff3fa26eaf8c72f0b80bcd1431a4a325be76d22591641bc01c035951b","device":"mobile","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/3296e47ff3fa26eaf8c72f0b80bcd1431a4a325be76d22591641bc01c035951b.png"} | |
| {"messages":[{"role":"user","content":[{"type":"image","image":"evolution/intermediate/_assets/autogui/9471fc5e165a7fc559b047746212a46c30e8efb974f91acee18280a52cfbbe8f.png"},"Ground the requested GUI element from the screenshot and answer the instruction accurately: Based on my descriptions, find the locations of the mentioned element in this webpage screenshot (with point). This element grants access to a user's account, enabling them to perform actions that require authentication, such as changing their Netflix plan."]},{"role":"assistant","content":"{\"answer\": \"(6,5)\", \"bounding_box\": [12.0, 17.0, 132.6640625, 49.0], \"device\": \"web\"}"}],"source":"AutoGUI/AutoGUI-v1-702k","source_revision":"937c77379af0500ea36419b95473615eb0b83489","source_image_id":"9471fc5e165a7fc559b047746212a46c30e8efb974f91acee18280a52cfbbe8f","device":"web","source_dataset":"Common Crawl","domain":"intermediate_gui_grounding","language":"en","media_path":"evolution/intermediate/_assets/autogui/9471fc5e165a7fc559b047746212a46c30e8efb974f91acee18280a52cfbbe8f.png"} | |
Xet Storage Details
- Size:
- 37.8 kB
- Xet hash:
- 640d5f774c8b216fd9fa737b0e2581cb1ae8fcad222cd19d0a87b1126f5ff16b
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.