Buckets:

BluStatic
/

DARK-MATTER

Files
xet
BluStatic/DARK-MATTER / datasets /shared /Model-Training /Basic /16_autonomous_workflows /validation.jsonl
BluStatic
13 days ago
download
raw
373 kB
{"messages":[{"role":"user","content":"Available function specifications:\nSYSTEM: You are a helpful assistant with access to the following functions. Use them if required -\n{\n    \"name\": \"calculate_fibonacci_sequence\",\n    \"description\": \"Calculate the Fibonacci sequence up to a given number of terms\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"terms\": {\n                \"type\": \"integer\",\n                \"description\": \"The number of terms in the sequence\"\n            }\n        },\n        \"required\": [\n            \"terms\"\n        ]\n    }\n}\n\nUser request:\nHi, I need to calculate the Fibonacci sequence for 10 terms. Can you help me with that?"},{"role":"assistant","content":"<functioncall> {\"name\":\"calculate_fibonacci_sequence\",\"arguments\":{\"terms\":10}}"}],"source":"glaiveai/glaive-function-calling-v2","domain":"tool_calling","language":"en","pair_sha256":"2f2d99ec70c467c00065d9e1b446e9fde98f5fc1c1aa2dd10934ca988f58b9a8","source_curated":true}
{"messages":[{"role":"user","content":"An authorized FLUX workflow fails after two completed steps. What next? Context reference 5013."},{"role":"assistant","content":"Record the failure, check persisted checkpoints, reconcile previous side effects and resume only safe remaining steps."}],"domain":"autonomous_workflows","source":"STATICAI authored simulated assistant instruction scenarios","language":"en","scenario_id":"assistant-005013","pair_sha256":"0cf37652d34710fa4dc80cb0733644bff9a8a25cf41df90f1a3c738a67d76922"}
{"messages":[{"role":"user","content":"Searching for video using screen keyboard, subscribing to a channel, liking, sharing, adding to newly created playlist, setting visibility of this playlist and making a collaboration."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the virtual keyboard icon in the search bar to open the on-screen keyboard.\",\"code\":\"pyautogui.click(x=0.6219, y=0.1508)\",\"image\":\"8e845961-fda8-4353-8aac-cf273b99e70d.png\"},{\"action\":\"Click on the \\\"f\\\" key on the virtual keyboard to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.7797, y=0.8483)\",\"image\":\"ece3846a-c9fd-498e-8e92-f7a5b1cd90d7.png\"},{\"action\":\"Click on the \\\"i\\\" key on the virtual keyboard to continue typing the search query.\",\"code\":\"pyautogui.click(x=0.8568, y=0.8142)\",\"image\":\"09cff593-4f9c-4cb2-821b-96bff0bad532.png\"},{\"action\":\"Click on the search button (magnifying glass icon) to execute the search with the current query.\",\"code\":\"pyautogui.click(x=0.662, y=0.1683)\",\"image\":\"df93b0c3-7baa-4f29-8afd-f24581c0fd34.png\"},{\"action\":\"Click on the \\\"Me at the zoo\\\" video thumbnail to open and watch the video.\",\"code\":\"pyautogui.click(x=0.3995, y=0.435)\",\"image\":\"6c2cc2d1-1f92-49fa-91c1-ab8833c927b2.png\"},{\"action\":\"Click on the close button (X) in the top-right corner of the virtual keyboard to dismiss it.\",\"code\":\"pyautogui.click(x=0.9719, y=0.745)\",\"image\":\"7ec28f7b-d175-4f74-8c2e-689b04a9f46f.png\"},{\"action\":\"Click on the \\\"Subskrybuj\\\" (Subscribe) button below the video to subscribe to the jawed channel.\",\"code\":\"pyautogui.click(x=0.1922, y=0.9058)\",\"image\":\"0bf0fd6b-564b-4375-aa03-6d24ab13d140.png\"},{\"action\":\"Click on the \\\"Wszystko\\\" (Everything) option in the notification preferences dialog to select all notifications for the channel.\",\"code\":\"pyautogui.click(x=0.2276, y=0.7567)\",\"image\":\"96df2b21-8ac9-40cd-9b53-271e45b1ada9.png\"},{\"action\":\"Click on the like button (thumbs up icon) showing 16 million likes beneath the video to like the \\\"Me at the zoo\\\" video.\",\"code\":\"pyautogui.click(x=0.3344, y=0.9133)\",\"image\":\"b44b3f09-4c87-4089-b03c-5eb71b17b079.png\"},{\"action\":\"Click on the \\\"Udostępnij\\\" (Share) button below the video to open sharing options for \\\"Me at the zoo\\\".\",\"code\":\"pyautogui.click(x=0.4302, y=0.9133)\",\"image\":\"407ea19b-9a84-4c71-b683-f379261a439d.png\"},{\"action\":\"Click the \\\"Kopiuj\\\" (Copy) button to copy the video URL to clipboard.\",\"code\":\"pyautogui.click(x=0.6141, y=0.6417)\",\"image\":\"280f65ab-4782-422b-9e99-f2b7d12e04d8.png\"},{\"action\":\"Click the close (X) button in the top-right corner of the sharing dialog to dismiss it and return to the main YouTube interface.\",\"code\":\"pyautogui.click(x=0.6453, y=0.3183)\",\"image\":\"41a83fed-a99e-4c3c-b4ef-c67bcef4f424.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241017214129_f3d9d31e-69e6-4bf9-8790-8c7be61d1cfc","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"48f8ed7fe8e7dedc588293cdf0d94ede8ad43058aa0d0ae25efd62d2276d4501"}
{"messages":[{"role":"user","content":"Sign in to Google forms from Google Chrome, Create a new google form, name the google form according to the research, populate the form with questions, configure the response settings to make the questions a must answer, set the form to ask for responders email for further communications, send the form to known participants or copy the form link and publish the link."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google apps menu icon (the 3x3 grid of dots) in the upper right corner of the browser to access Google services including Google Forms.\",\"code\":\"pyautogui.click(x=0.9609, y=0.1426)\",\"image\":\"d6907867-338a-48ba-925e-39123689d8cf.png\"},{\"action\":\"Click on the Google Forms icon in the Google apps menu (the purple icon with white form lines).\",\"code\":\"pyautogui.click(x=0.951, y=0.4454)\",\"image\":\"91a4ec7d-4a43-4e6a-a038-1fc3b190c236.png\"},{\"action\":\"Click on the \\\"Blank form\\\" template to create a new empty form.\",\"code\":\"pyautogui.click(x=0.2568, y=0.2611)\",\"image\":\"8655a2e3-80e9-4b5a-bd3d-0d1bc9ee31b0.png\"},{\"action\":\"Double-click on the \\\"Untitled form\\\" title field to select all the text and prepare to rename the form according to my research purpose.\",\"code\":\"pyautogui.doubleClick(x=0.3609, y=0.2639)\",\"image\":\"b712a3ca-4c92-49de-868b-eef0ae250683.png\"},{\"action\":\"Type \\\"AgentNet study feedback.\\\" as the new title for the form.\",\"code\":\"pyautogui.write(message='AgentNet study feedback.')\",\"image\":\"7b9f57b1-605c-407e-adb2-d2505e29b7ac.png\"},{\"action\":\"Click on the \\\"Form description\\\" field to begin adding a description for the AgentNet study feedback form.\",\"code\":\"pyautogui.click(x=0.3443, y=0.3565)\",\"image\":\"6707eef7-732d-4ca1-b4ac-081c85877e91.png\"},{\"action\":\"Type \\\"We just want to know how you found the tasks and what you would like us to change.\\\" in the Form description field.\",\"code\":\"pyautogui.write(message='We just want to know how you found the tasks and what you would like us to change.')\",\"image\":\"fa9dd482-4cba-4787-9387-a7d8473a2cac.png\"},{\"action\":\"Click on the \\\"Untitled Question\\\" text field to begin editing the first question of the form.\",\"code\":\"pyautogui.click(x=0.35, y=0.4398)\",\"image\":\"606b8471-c28e-423b-9f7e-3f5e259779aa.png\"},{\"action\":\"Type \\\"How many task did you do??\\\" in the question field to create the first question of the form.\",\"code\":\"pyautogui.write(message='How many task did you do??')\",\"image\":\"2a8f91ae-d8b0-40bb-aec7-8140d2c22744.png\"},{\"action\":\"Click on the question type dropdown menu that currently displays \\\"Short answer\\\" to change the question type to a more appropriate format for the numeric question about tasks completed.\",\"code\":\"pyautogui.click(x=0.6432, y=0.4)\",\"image\":\"1ae3a0e1-a641-4407-bd68-b307b5d7cdc7.png\"},{\"action\":\"Click on the \\\"Multiple choice\\\" option in the question type dropdown menu to change the question format from short answer to multiple choice.\",\"code\":\"pyautogui.click(x=0.6, y=0.4602)\",\"image\":\"b8caa7c8-5329-4f9f-a666-87eeb8281822.png\"},{\"action\":\"Double-click on the \\\"Option 1\\\" text field to select all the text and prepare to replace it with a specific answer choice for the multiple choice question.\",\"code\":\"pyautogui.doubleClick(x=0.363, y=0.4657)\",\"image\":\"372e0637-254e-4ddd-9a43-d2d2e93cf013.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241018063017_a8aa11a7-934d-493a-a9e4-071f01fb1352","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"b64f3a13715bacb6303a3d96d439a7764cd68a996b6852e1e9c4a45e6388c1d4"}
{"messages":[{"role":"user","content":"On iHerb website, go to shopping cart page, remove items previously added which are unavailable. Click view buying options on EVLution Nutrition isolate protein chocolate peanut butter flavor 5lb. Select Double Rich cholocolate flavour. click add to cart. click proceed to checkout. select the google account to sign in."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the user account icon displaying \\\"maximbrink6@gmail...\\\" in the green header section.\",\"code\":\"pyautogui.click(x=0.4714, y=0.1439)\",\"image\":\"6fd6cc55-6db5-4d04-93ea-7b907f47c0ba.png\"},{\"action\":\"Click on the shopping cart icon in the account dropdown menu to access the shopping cart page.\",\"code\":\"pyautogui.click(x=0.4718, y=0.1489)\",\"image\":\"17a3b45d-4b05-46a2-9930-c60ea5b9da29.png\"},{\"action\":\"Scroll down on the shopping cart page to see more of the unavailable items section.\",\"code\":\"pyautogui.moveTo(x=0.2899, y=0.2945)\\npyautogui.scroll(-3)\",\"image\":\"be806d44-57d0-4df8-b596-52b17bc2f453.png\"},{\"action\":\"Click the \\\"Remove\\\" link for the EVLution Nutrition, 100% Whey Protein, Double Rich Chocolate product to remove this item from the shopping cart.\",\"code\":\"pyautogui.click(x=0.2021, y=0.8486)\",\"image\":\"966f12ee-0c3c-40d0-86e1-eb859a3197a1.png\"},{\"action\":\"Click the \\\"Remove\\\" link for the ALLMAX GOLD ALLWHEY Premium Whey Protein French Vanilla product.\",\"code\":\"pyautogui.click(x=0.2023, y=0.6519)\",\"image\":\"94bb3ac0-3e4c-482c-8f7f-e40fd28288f7.png\"},{\"action\":\"Click the \\\"Remove\\\" link next to the EVLution Nutrition, 100% Whey Protein, Vanilla Ice Cream product to remove it from the shopping cart.\",\"code\":\"pyautogui.click(x=0.2079, y=0.5493)\",\"image\":\"e3380585-c2ff-4227-b398-e0649da1fe10.png\"},{\"action\":\"Click on the \\\"View all buying options\\\" link for the EVLution Nutrition, 100% Isolate Protein, Chocolate Peanut Butter product.\",\"code\":\"pyautogui.click(x=0.145, y=0.5506)\",\"image\":\"e9e03073-e059-4793-822c-552deb7a3911.png\"},{\"action\":\"Click on the Double Rich Chocolate flavor option to select it as the alternative flavor for the EVLution Nutrition 100% Isolate Protein.\",\"code\":\"pyautogui.click(x=0.2334, y=0.542)\",\"image\":\"1c6c4d71-0f62-4e9f-bea4-958333a52d75.png\"},{\"action\":\"Click the \\\"Add to Cart\\\" button to add the EVLution Nutrition 100% Isolate Protein with Double Rich Chocolate flavor to the shopping cart.\",\"code\":\"pyautogui.click(x=0.2814, y=0.7298)\",\"image\":\"a2729ae0-f3c5-4d26-97a9-208d40a3df2a.png\"},{\"action\":\"Scroll down in the shopping cart page to view more content below the currently visible area, particularly in the area that shows shipping options and recommendations.\",\"code\":\"pyautogui.moveTo(x=0.2828, y=0.7275)\\npyautogui.scroll(14)\",\"image\":\"83aecc36-0031-4305-85db-4bd4d861d8eb.png\"},{\"action\":\"Click on the \\\"Proceed to Checkout (1)\\\" button in the order summary section to continue to the checkout process.\",\"code\":\"pyautogui.click(x=0.3837, y=0.5506)\",\"image\":\"3e441870-f09e-42d2-9775-b841fd7c23ee.png\"},{\"action\":\"Click on the \\\"Sign in with Google\\\" button to proceed with Google account authentication.\",\"code\":\"pyautogui.click(x=0.203, y=0.4852)\",\"image\":\"0c033384-43e3-49ce-83a5-926b458d3ba0.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114144753_67519b79-6c1c-4e4f-b469-ade289d4ef8a","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"0faa534a55d9c897ba9142611c76fafecad44248123f5141d54390160e69942a"}
{"messages":[{"role":"user","content":"Search on Google for instructions on changing the Docker source, follow the online tutorial to change the Docker image source"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click the new tab button (+ symbol) in the browser's tab bar to open a new tab.\",\"code\":\"pyautogui.click(x=0.4089, y=0.0176)\",\"image\":\"cee740d1-fc7a-47fc-9105-db7805d117c2.png\"},{\"action\":\"Click on the search bar in the center of the Bing homepage where it says \\\"Search the web\\\" to begin entering a search query about Docker image sources.\",\"code\":\"pyautogui.click(x=0.2125, y=0.313)\",\"image\":\"42be6514-5a87-4977-9630-f7c0ebcb94ed.png\"},{\"action\":\"Type \\\"google\\\" in the search bar and press Enter to search for Google's website.\",\"code\":\"pyautogui.write(message='google')\\npyautogui.press('enter')\",\"image\":\"e6270227-1cff-4edc-b1a1-6fd5167278d6.png\"},{\"action\":\"Scroll down on the Bing search results page to view more results about Google.\",\"code\":\"pyautogui.moveTo(x=0.2125, y=0.312)\\npyautogui.scroll(-3)\",\"image\":\"e2413749-5a6e-4ca0-a2ea-9d524637be81.png\"},{\"action\":\"Click on the \\\"Google\\\" text link that appears below the Google logo in the search results, which will take you to Google's homepage.\",\"code\":\"pyautogui.click(x=0.112, y=0.6935)\",\"image\":\"2120649a-5ed6-4107-9c8e-f6d2b02cca60.png\"},{\"action\":\"Type \\\"dockerhuanyuahuanyuan3\\\" in the Google search bar and press Enter to search for instructions on changing the Docker image source.\",\"code\":\"pyautogui.write(message='dockerhuanyuahuanyuan3')\\npyautogui.press('enter')\",\"image\":\"63c1c5bb-18f5-41e8-a7df-8830cf44655c.png\"},{\"action\":\"Click on the first search result titled \\\"docker更换国内源加速原创\\\" from CSDN to access the tutorial about changing Docker image sources.\",\"code\":\"pyautogui.click(x=0.1443, y=0.437)\",\"image\":\"939d1395-7a37-4668-87be-10aad9a4462e.png\"},{\"action\":\"Click on the Windows search box at the bottom of the screen to search for a terminal application.\",\"code\":\"pyautogui.click(x=0.3354, y=0.9778)\",\"image\":\"8ce0e419-7000-4f29-9a71-cc7be83562da.png\"},{\"action\":\"Type \\\"docker\\\" in the Windows search box to search for Docker-related applications.\",\"code\":\"pyautogui.write(message='docker')\",\"image\":\"2f481345-a7b9-440d-bf98-d01f09dd41a6.png\"},{\"action\":\"Click on the Docker Desktop application in the search results under \\\"Best match\\\".\",\"code\":\"pyautogui.click(x=0.3214, y=0.4028)\",\"image\":\"5d779b8e-2500-4e78-b330-810f53b8dc01.png\"},{\"action\":\"Click on the settings gear icon in the Docker Desktop navigation bar to access Docker's configuration settings.\",\"code\":\"pyautogui.click(x=0.6943, y=0.1667)\",\"image\":\"7cc30ce6-9b04-46e5-9658-6d7e8c0c7e12.png\"},{\"action\":\"Click on the \\\"Docker Engine\\\" option in the left sidebar menu of the Docker Desktop settings panel.\",\"code\":\"pyautogui.click(x=0.2458, y=0.4056)\",\"image\":\"1e2d7020-e2fd-466f-ac31-bba439f788d1.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240928115110_eaa4d08c-fbb3-4a9a-b8db-84dbd4465032","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c479899edcf71ae9b562eb2f0ecc4616498e07be63e116e557d38252427b26eb"}
{"messages":[{"role":"user","content":"Watching YouTube Shorts and reviewing the short, giving a like and a comment, sorting comments, changing volume, copying share link to clipboard, turning off cinema light, reading description of short, and reporting short to YouTube."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Shorts\\\" option in the left sidebar navigation menu to access the YouTube Shorts feed.\",\"code\":\"pyautogui.click(x=0.0839, y=0.2517)\",\"image\":\"00910c16-086f-487d-9302-b0330b49d7d5.png\"},{\"action\":\"Drag the volume slider knob slightly to the right to increase the volume of the YouTube Short video.\",\"code\":\"pyautogui.moveTo(x=0.5745, y=0.2333)\\npyautogui.dragTo(x=0.5964, y=0.2333, button='left')\",\"image\":\"fe7d7f09-19bc-43aa-b1cf-cdf40183d4eb.png\"},{\"action\":\"Click the like (thumbs up) button located to the right of the video player to give a like to the YouTube Short video.\",\"code\":\"pyautogui.click(x=0.725, y=0.4492)\",\"image\":\"848b98e0-f04c-4754-99c2-c7a5246e75a5.png\"},{\"action\":\"Click on the comment button (chat bubble icon) to open the comments section of the YouTube Short.\",\"code\":\"pyautogui.click(x=0.7323, y=0.6317)\",\"image\":\"be82300a-b308-40f4-9d78-cb5ad0c2c75b.png\"},{\"action\":\"Click on the \\\"Dodaj komentarz\\\" (Add comment) text field at the bottom of the comments section to begin typing a comment.\",\"code\":\"pyautogui.click(x=0.6781, y=0.8742)\",\"image\":\"a9946e5f-a6c7-4af9-81b6-2ed53997cfb4.png\"},{\"action\":\"Type \\\"komenatrz\\\" in the active comment field.\",\"code\":\"pyautogui.write(message='komenatrz')\",\"image\":\"42369acb-2411-4770-bf28-b9ce659b7755.png\"},{\"action\":\"Click the \\\"Skomentuj\\\" (Comment) button to submit the typed comment.\",\"code\":\"pyautogui.click(x=0.8672, y=0.8842)\",\"image\":\"b1ea1bb0-f334-46c2-a69e-04b061fc5f0b.png\"},{\"action\":\"Click on the comments sorting menu button (three horizontal lines icon) in the top-right corner of the comments section to access comment sorting options.\",\"code\":\"pyautogui.click(x=0.851, y=0.2275)\",\"image\":\"538c50f2-8b32-4264-a522-c32cff53d39f.png\"},{\"action\":\"Click on the \\\"Od najnowszych\\\" (Newest first) option in the comment sorting dropdown menu to sort the comments by recency.\",\"code\":\"pyautogui.click(x=0.8562, y=0.3283)\",\"image\":\"5285811e-f90e-40ad-9034-0da6c14e542c.png\"},{\"action\":\"Click the X button in the top-right corner of the comments section to close the comments panel.\",\"code\":\"pyautogui.click(x=0.8818, y=0.2242)\",\"image\":\"e66b70e0-5992-43c3-82f2-e4ba2f42233a.png\"},{\"action\":\"Click on the share button (\\\"Udostępnij\\\") with the forward arrow icon on the right side of the YouTube Shorts video to access sharing options.\",\"code\":\"pyautogui.click(x=0.737, y=0.755)\",\"image\":\"195cbaa5-076e-421c-88f0-fe38c673d4aa.png\"},{\"action\":\"Click the \\\"Kopiuj\\\" button to copy the YouTube Short's URL to the clipboard.\",\"code\":\"pyautogui.click(x=0.6365, y=0.6058)\",\"image\":\"489ffcb2-f920-48bb-8194-a6c646a06761.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241017213534_5fec49e3-c5fa-43b9-9e3c-90f6e945760d","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"2f5dcf4667f6f14cdc3427781c849e7fe749186fcbf80b3a09a3a5c34666b8e8"}
{"messages":[{"role":"user","content":"Google search about Mr Ratan Tata, a famous Indian Industrialist"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the address bar at the top of the Chrome browser window to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.2729, y=0.0796)\",\"image\":\"9ca6e4c7-52c6-4615-853c-900c9971e458.png\"},{\"action\":\"Type \\\"detailed biography about ratan tata\\\" in the address bar and press Enter to execute the search query.\",\"code\":\"pyautogui.write(message='detailed biography about ratan tata')\\npyautogui.press('enter')\",\"image\":\"724c6809-a087-4dea-b362-a956f2a85cc5.png\"},{\"action\":\"Double-click on the \\\"Ratan Tata\\\" text in the Britannica search result to open the detailed biography page.\",\"code\":\"pyautogui.doubleClick(x=0.1542, y=0.4602)\",\"image\":\"830a13f8-5dcf-4996-bf35-f2239a309523.png\"},{\"action\":\"Click the back button in the top-left corner of the browser window to return to the previous page with Google search results.\",\"code\":\"pyautogui.click(x=0.0125, y=0.0676)\",\"image\":\"ebc0239f-ca4c-46b1-a73d-9086b6006818.png\"},{\"action\":\"Click on the question \\\"Which is the best biography of Ratan Tata?\\\" in the \\\"People also ask\\\" section to expand it and view the answer.\",\"code\":\"pyautogui.click(x=0.2844, y=0.5046)\",\"image\":\"9afadffa-a161-46d0-9700-255739aa2c63.png\"},{\"action\":\"Click on the word \\\"Birth\\\" in the \\\"Ratan Tata Biography: Birth, Age, Education, Family, Wife...\\\" search result title.\",\"code\":\"pyautogui.click(x=0.2865, y=0.5343)\",\"image\":\"87554064-7099-4d9e-8535-59229635a03f.png\"},{\"action\":\"Click the back button in the top-left corner of the browser window to return to the previous page.\",\"code\":\"pyautogui.click(x=0.0104, y=0.0713)\",\"image\":\"9344e611-05e0-4d65-b9f7-725544823b33.png\"},{\"action\":\"Click the X (close) button at the top-right corner of the current browser tab to close the Google search results page about Ratan Tata.\",\"code\":\"pyautogui.click(x=0.9901, y=0.0083)\",\"image\":\"792f8a64-cb81-45bf-9da4-b2fe916165a5.png\"},{\"action\":\"Click the close (X) button in the top-right corner of the current browser tab to close the Google search results page about Ratan Tata.\",\"code\":\"pyautogui.click(x=0.987, y=0.0194)\",\"image\":\"1a3ad9b4-ce5b-4022-9840-18cb0cb79536.png\"},{\"action\":\"Terminate the task with a success status.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"ed83c9ef-cfeb-46c4-b7ac-5073853f9da1.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241015163351_5f6fdae7-94c0-4826-b6a0-109aee5ea312","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"0bb7c72a85914b98f3bae9e998a6e3752392b945aaa65c3da0528b91bfa7db61"}
{"messages":[{"role":"user","content":"Purchase a surface in microsoft.com\nNavigate to microsoft.com, click surface, set options, continue to check out"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the new tab button to open a fresh tab.\",\"code\":\"pyautogui.click(x=0.863, y=0.067)\",\"image\":\"b4ac45fe-e112-49b4-a8f3-6dcc46a618e2.png\"},{\"action\":\"Type \\\"microsoft\\\" in the Google search box and press Enter to search for Microsoft's website.\",\"code\":\"pyautogui.write(message='microsoft')\\npyautogui.press('enter')\",\"image\":\"4ee3191d-846a-4500-8023-2f5afa4b2c1f.png\"},{\"action\":\"Click on the main Microsoft search result link that reads \\\"Microsoft – AI, Cloud, Productivity, Computing, Gaming & Apps\\\" to navigate to Microsoft's official website.\",\"code\":\"pyautogui.click(x=0.362, y=0.361)\",\"image\":\"2b160133-271f-4072-bd23-42c95efd7c94.png\"},{\"action\":\"Click on the \\\"Surface\\\" link in the top navigation menu, which is located between \\\"Windows\\\" and \\\"Xbox\\\".\",\"code\":\"pyautogui.click(x=0.361, y=0.154)\",\"image\":\"3529b6d6-c6d2-41da-80fa-ffcee5e43536.png\"},{\"action\":\"Scroll down on the Microsoft Surface page to view more product options and information.\",\"code\":\"pyautogui.moveTo(x=0.422, y=0.539)\\npyautogui.scroll(10)\",\"image\":\"5bbc7012-f8c7-4212-a779-60df3880a5a9.png\"},{\"action\":\"Click on the \\\"Meet the new Surface Pro\\\" button to view Surface Pro product details and purchase options.\",\"code\":\"pyautogui.click(x=0.446, y=0.472)\",\"image\":\"0ab6c5c5-62f1-4fc6-b080-1d7989136c61.png\"},{\"action\":\"Click the \\\"Start building yours\\\" blue button under the Surface Pro model on the left side of the screen.\",\"code\":\"pyautogui.click(x=0.306, y=0.754)\",\"image\":\"2391e2ac-95e7-4b2d-905c-9ee8d9290709.png\"},{\"action\":\"Click on the Snapdragon X Plus (10 Core) with LCD display option to select this processor configuration.\",\"code\":\"pyautogui.click(x=0.717, y=0.755)\",\"image\":\"f39b2c49-e256-4109-ae48-6960e4be65c2.png\"},{\"action\":\"Click on the \\\"Wi-Fi\\\" option in the Connectivity section to select this as the preferred connectivity method for the Surface Pro.\",\"code\":\"pyautogui.click(x=0.743, y=0.612)\",\"image\":\"accd1927-adeb-47c3-9267-00a2c0e4de23.png\"},{\"action\":\"Scroll down slightly in the Specifications section to view more configuration options for the Surface Pro.\",\"code\":\"pyautogui.moveTo(x=0.743, y=0.691)\\npyautogui.scroll(-10)\",\"image\":\"76671ed5-2d6f-479f-8232-1f3d5a81ed3d.png\"},{\"action\":\"Click the \\\"Continue to add-ons\\\" button to proceed with the Surface Pro purchase process and view additional accessory options.\",\"code\":\"pyautogui.click(x=0.756, y=0.518)\",\"image\":\"844bdb54-8d38-433b-abf3-0cc18c76f962.png\"},{\"action\":\"Click on the burgundy/red color option to select this color for the Surface Pro.\",\"code\":\"pyautogui.click(x=0.697, y=0.397)\",\"image\":\"237eaf4d-783f-4b5e-bddc-54a9c2bdd9ce.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240906134801_fc15712d-1fe5-40ed-b8e9-7d17b2f79824","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"5c1c5218e79ea1517afa75664343945e043b67eea928e893db3f248c3d8023a3"}
{"messages":[{"role":"user","content":"n blender perform the rigging/skinning step of binding an armature to a mesh. In this case we edit the position of the joints a human rig so that it aligns with the joint position of the human object"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Press the Tab key to switch from Object Mode to Edit Mode.\",\"code\":\"pyautogui.press('tab')\",\"image\":\"025f7919-e354-4f5c-ac6a-e9945774403e.png\"},{\"action\":\"Click on the Move tool in the left toolbar to enable object translation/movement capabilities.\",\"code\":\"pyautogui.click(x=0.015, y=0.163)\",\"image\":\"4c8bec74-1b79-4fe1-bd17-efa92b031cd3.png\"},{\"action\":\"Click on the \\\"X-Axis Mirror\\\" option in the Move tool settings panel to enable symmetrical editing for the rigging process.\",\"code\":\"pyautogui.click(x=0.846, y=0.468)\",\"image\":\"b90cb718-eb25-4087-87c4-9f57fbd465f0.png\"},{\"action\":\"Scroll the mouse wheel at the current position in the 3D viewport to adjust the view of the armature and human model.\",\"code\":\"pyautogui.moveTo(x=0.419, y=0.475)\\npyautogui.scroll(10)\",\"image\":\"3bee0597-470f-406a-86c6-35cd1cd63614.png\"},{\"action\":\"Click on the joint connection point in the arm's armature to select it for repositioning.\",\"code\":\"pyautogui.click(x=0.416, y=0.426)\",\"image\":\"c9974f6f-232e-4a8e-bab8-5de700829ca3.png\"},{\"action\":\"Drag the selected armature joint from its current position to a new position slightly upward and to the right to better align it with the anatomical elbow joint position of the human mesh.\",\"code\":\"pyautogui.moveTo(x=0.416, y=0.426)\\npyautogui.dragTo(x=0.435, y=0.376, button='left')\",\"image\":\"f508f7f1-eb6a-4fb6-ba27-74d530bf0582.png\"},{\"action\":\"Click on a joint position in the viewport and drag it to a new position to better align it with the human mesh's anatomy.\",\"code\":\"pyautogui.moveTo(x=0.469, y=0.503)\\npyautogui.dragTo(x=0.428, y=0.499, button='left')\",\"image\":\"7d556d1e-03d4-4095-8120-fbefa11a94bb.png\"},{\"action\":\"Click on the joint in the upper arm/shoulder area of the armature to select it for repositioning.\",\"code\":\"pyautogui.click(x=0.155, y=0.311)\",\"image\":\"bbebb23a-0265-48da-bc83-024d37c538f5.png\"},{\"action\":\"Drag the selected joint in the armature (indicated by the white cross with colored axes) horizontally to the right to better align it with the corresponding anatomical position on the human mesh model.\",\"code\":\"pyautogui.moveTo(x=0.090, y=0.371)\\npyautogui.dragTo(x=0.184, y=0.371, button='left')\",\"image\":\"d9207b8f-6030-4b08-9f64-b49994df5332.png\"},{\"action\":\"Scroll the mouse wheel down in the 3D viewport to zoom out slightly and get a better view of the armature and human mesh.\",\"code\":\"pyautogui.moveTo(x=0.473, y=0.459)\\npyautogui.scroll(-10)\",\"image\":\"22cf8774-84bc-4d36-b61c-f46f7f38c128.png\"},{\"action\":\"Click on an empty area in the 3D viewport and drag to a position near a joint that needs adjustment in the armature.\",\"code\":\"pyautogui.moveTo(x=0.500, y=0.463)\\npyautogui.dragTo(x=0.406, y=0.474, button='left')\",\"image\":\"c661b88b-bf46-4f3c-94e8-ecb3d76acd63.png\"},{\"action\":\"Move the cursor to the upper chest/torso area of the human mesh and scroll the mouse wheel up to zoom in on that region.\",\"code\":\"pyautogui.moveTo(x=0.313, y=0.280)\\npyautogui.scroll(10)\",\"image\":\"c87e32b7-0d72-4d97-917b-1b5fdaa61b68.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240929153659_06ad9d13-e611-4d5d-a4a2-0416d437020d","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"216fee5a4ba4fa42e1070776b42fecb921b02e77da0245d8a0d62be9f10eaf12"}
{"messages":[{"role":"user","content":"Create a new rule in Outlook mail, the rule marks all the emails where I am the only recipient to be important, set an exception to ignore emails where I am on the to list, apply the rule"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Microsoft Outlook icon in the dock to open the application.\",\"code\":\"pyautogui.click(x=0.3105, y=0.9758)\",\"image\":\"3b66f510-f816-44e2-97a3-0d6d396eecf5.png\"},{\"action\":\"Click on the Help menu in the top menu bar to access help resources and search for information about creating mail rules.\",\"code\":\"pyautogui.click(x=0.5572, y=0.0117)\",\"image\":\"d7d369cf-673e-4b2e-8e0d-c1f88d880dbe.png\"},{\"action\":\"Click on the \\\"Rules...\\\" option in the Help menu to access the mail rules configuration interface.\",\"code\":\"pyautogui.click(x=0.5928, y=0.0648)\",\"image\":\"c75f9a32-4cc9-4d07-a5ae-e093fe46d435.png\"},{\"action\":\"Click on the \\\"Rules...\\\" option in the Tools menu to access the rules configuration interface.\",\"code\":\"pyautogui.click(x=0.4097, y=0.1612)\",\"image\":\"0b5fdcfa-fbd9-4cf0-81c8-4ad66ed3b943.png\"},{\"action\":\"Click the \\\"New Rule\\\" button at the bottom of the Rules window to begin creating a new mail rule.\",\"code\":\"pyautogui.click(x=0.2106, y=0.9973)\",\"image\":\"ee59d032-fc33-4eb6-a796-41bec6952563.png\"},{\"action\":\"Type \\\"MARK IMPORTANT\\\" in the \\\"Name your rule\\\" text field to give the rule a descriptive name.\",\"code\":\"pyautogui.write(message='MARK IMPORTANT')\",\"image\":\"0531a01e-b11f-44c9-ab84-5785668d4e97.png\"},{\"action\":\"Click on the \\\"Select a condition\\\" dropdown to display the available conditions for the rule.\",\"code\":\"pyautogui.click(x=0.2846, y=0.4666)\",\"image\":\"13df9db0-8b98-4194-9e01-a9ceabb254bb.png\"},{\"action\":\"Click on the \\\"I'm the only recipient\\\" option in the condition dropdown menu to select this as the rule condition.\",\"code\":\"pyautogui.click(x=0.4909, y=0.7995)\",\"image\":\"ccf5d2c6-0e80-4892-9dfc-0f8427c5b616.png\"},{\"action\":\"Click on the \\\"Select an action\\\" dropdown to view available actions for the rule.\",\"code\":\"pyautogui.click(x=0.3295, y=0.6082)\",\"image\":\"38c48f61-63a2-4760-a206-b56ce0da1465.png\"},{\"action\":\"Click on the \\\"Mark with importance\\\" option in the dropdown menu to select it as the action for the rule.\",\"code\":\"pyautogui.click(x=0.5117, y=0.7144)\",\"image\":\"5a980139-3249-42f2-8f94-6236ec338265.png\"},{\"action\":\"Click on the \\\"Select an option\\\" dropdown to reveal importance level options.\",\"code\":\"pyautogui.click(x=0.5173, y=0.6081)\",\"image\":\"b87b4a11-d6c7-418a-ad1c-e68bdba93d7b.png\"},{\"action\":\"Click on the \\\"High\\\" option in the importance level dropdown menu.\",\"code\":\"pyautogui.click(x=0.5081, y=0.6582)\",\"image\":\"045a4808-feb8-4812-b707-ad120c274f22.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240930041922_fde05c69-509b-46d8-a606-88b4671e20e9","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"9d2f6f48562d7926befba97ccdbc05fad3bb65aba47ab1b8fcd786aa0b737473"}
{"messages":[{"role":"user","content":"In Google Finance, check the news on the top right, check COMP, SPX and DJIA, scroll down, click view more, copy and share link. Return to the homepage, check most active, copy and share link."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Chrome browser icon in the dock to open or focus the Chrome web browser.\",\"code\":\"pyautogui.click(x=0.312, y=0.948)\",\"image\":\"756c07ae-db98-4209-b4f7-e7d719298ceb.png\"},{\"action\":\"Click on the news headline \\\"Stock Market Today: S&P 500, Nasdaq finish at record...\\\" in the navigation bar to read the full article.\",\"code\":\"pyautogui.click(x=0.655, y=0.194)\",\"image\":\"6ff18e63-2773-44ce-803f-8d2a99812554.png\"},{\"action\":\"Click on the COMP data point on the performance graph to view more detailed information about the Nasdaq Composite index.\",\"code\":\"pyautogui.click(x=0.679, y=0.798)\",\"image\":\"c2eccf6e-d10e-4cb3-a31b-13bfc3263347.png\"},{\"action\":\"Scroll down on the MarketWatch article page to view more content below the performance graph.\",\"code\":\"pyautogui.moveTo(x=0.593, y=0.748)\\npyautogui.scroll(-10)\",\"image\":\"fe70ecdb-61c6-4864-bfc2-e9871d8286da.png\"},{\"action\":\"Click on the SPX (S&P 500) data point on the performance graph to view more detailed information about the S&P 500 index.\",\"code\":\"pyautogui.click(x=0.693, y=0.700)\",\"image\":\"890bcdf1-02a1-4267-925d-998c56314cdd.png\"},{\"action\":\"Click on the DJIA data point on the performance graph to view more detailed information about the Dow Jones Industrial Average index.\",\"code\":\"pyautogui.click(x=0.686, y=0.762)\",\"image\":\"fb74af96-deb8-4201-abaa-618c27c85943.png\"},{\"action\":\"Click on the \\\"View More\\\" button to expand additional content.\",\"code\":\"pyautogui.click(x=0.373, y=0.642)\",\"image\":\"54585037-5e1a-4904-b0b3-549fe45775da.png\"},{\"action\":\"Click on the \\\"Share\\\" button located to the right of the market data summary to access sharing options for the article.\",\"code\":\"pyautogui.click(x=0.365, y=0.619)\",\"image\":\"460d1cb5-a1c4-4ca3-8d92-0c298272f5f2.png\"},{\"action\":\"Click on the \\\"Copy Link\\\" option in the sharing menu to copy the article URL to the clipboard.\",\"code\":\"pyautogui.click(x=0.599, y=0.576)\",\"image\":\"47fe5d1d-60e8-4c80-9a11-f329c0323202.png\"},{\"action\":\"Scroll down on the MarketWatch page to view more content about oil futures.\",\"code\":\"pyautogui.moveTo(x=0.234, y=0.718)\\npyautogui.scroll(10)\",\"image\":\"e54ba83a-d866-4724-961a-04b0f5415244.png\"},{\"action\":\"Click on the \\\"Most active\\\" tab in the Market trends section to view the most actively traded stocks.\",\"code\":\"pyautogui.click(x=0.705, y=0.549)\",\"image\":\"06f6d7e5-6e3b-4431-b6b2-8b9066d460fa.png\"},{\"action\":\"Click on the Share button located to the right of the \\\"Explore market trends\\\" heading to access sharing options for the most active stocks page.\",\"code\":\"pyautogui.click(x=0.754, y=0.306)\",\"image\":\"71d85379-ecef-4b27-852a-2506777c9b7c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241206021306_d9f5355f-077a-4a47-947e-2ea9576064cf","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"bf3d38621aeeee7ceb8c038179ae5f0b4d1151b35b760cfec035ef06b2164649"}
{"messages":[{"role":"user","content":"Search on Kayak for business class flights from Dallas to Los Angeles with 2 checked bags allowed, depart flights on Nov 27, return flights on Nov 30, filter the results by take-off from DFW after 7:34am and Wi-Fi only, view the USD 1381 deal"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"From?\\\" input field to begin entering the departure location.\",\"code\":\"pyautogui.click(x=0.2023, y=0.4294)\",\"image\":\"ecc7b712-97d8-463d-b2c2-96cae6cacd0f.png\"},{\"action\":\"Type \\\"dallas\\\" in the \\\"From?\\\" field to specify the departure location for the flight search.\",\"code\":\"pyautogui.write(message='dallas')\",\"image\":\"d5da3bef-5b34-4fc2-8876-6f9ffaf9b0c5.png\"},{\"action\":\"Click on the \\\"Dallas, Texas, United States DFW All airports\\\" option in the dropdown menu to select it as the departure location.\",\"code\":\"pyautogui.click(x=0.2039, y=0.5363)\",\"image\":\"f1965d5f-eac2-4851-9789-e44738e8740e.png\"},{\"action\":\"Click on the \\\"To?\\\" input field to begin entering the destination location.\",\"code\":\"pyautogui.click(x=0.4023, y=0.425)\",\"image\":\"1f59fe75-9bf4-45da-b5ce-80a7bdf9af41.png\"},{\"action\":\"Type \\\"lax\\\" in the destination field to search for Los Angeles.\",\"code\":\"pyautogui.write(message='lax')\",\"image\":\"6b7b6368-b68d-4f47-a889-73b6614e66ae.png\"},{\"action\":\"Click on the \\\"Los Angeles, California, United States LAX\\\" option in the destination dropdown menu to select it as the destination for the flight search.\",\"code\":\"pyautogui.click(x=0.4145, y=0.5563)\",\"image\":\"9657d64a-7d94-49c4-a6c4-96e148e2929a.png\"},{\"action\":\"Click on the Economy dropdown menu to change the cabin class.\",\"code\":\"pyautogui.click(x=0.2992, y=0.3681)\",\"image\":\"4e7effd2-afa8-4d2a-9334-2fa6a2db0d45.png\"},{\"action\":\"Click on the \\\"Business\\\" option in the cabin class dropdown menu to select business class for the flight search.\",\"code\":\"pyautogui.click(x=0.2945, y=0.4888)\",\"image\":\"39f38012-7d61-4a32-a48d-63042df75d36.png\"},{\"action\":\"Click on the \\\"0 bags\\\" dropdown menu to change the baggage allowance.\",\"code\":\"pyautogui.click(x=0.3617, y=0.3563)\",\"image\":\"b6930e57-44a9-44c5-bf9f-02dfd3ca77d7.png\"},{\"action\":\"Click the plus button next to the checked bag counter to increase the number of checked bags from 1 to 2.\",\"code\":\"pyautogui.click(x=0.6535, y=0.2944)\",\"image\":\"1f8152ab-adac-42bb-af2e-3a906d731347.png\"},{\"action\":\"Click on the departure date field that displays \\\"Tue 12/3\\\" to begin changing the departure date to November 27.\",\"code\":\"pyautogui.click(x=0.5578, y=0.4288)\",\"image\":\"cb39934c-9131-4cdd-b9d8-d742773b6474.png\"},{\"action\":\"Click the left arrow button next to the \\\"DATES\\\" tab to navigate to the previous month (November) in the calendar.\",\"code\":\"pyautogui.click(x=0.3699, y=0.4612)\",\"image\":\"bdd6d2c5-bf38-4f3c-a3d3-9e6c3a41df2a.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241119161922_6ea65fde-a1d8-4591-9d2d-bc035e60317a","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8b88de3b4e45d806f194c1ec9f9b49d24ca790667bcc2f5a5479dd48bb5ca2bf"}
{"messages":[{"role":"user","content":"take a screenshot of the letter sent by the HK ImmD in Gmail, send that screenshot to Wen Yunjie in WeChat"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search field at the top of the Finder window to begin searching for a web browser.\",\"code\":\"pyautogui.click(x=0.359, y=0.403)\",\"image\":\"258f6d7a-5679-4c0d-ab7a-6d6a731bb0eb.png\"},{\"action\":\"Click on the Chrome icon in the dock to launch the web browser.\",\"code\":\"pyautogui.click(x=0.402, y=0.983)\",\"image\":\"bd281426-b9d1-4f69-b868-f41df4da04bb.png\"},{\"action\":\"Click on the Gmail icon in the browser's toolbar in the upper-right corner of the window to directly access Gmail.\",\"code\":\"pyautogui.click(x=0.810, y=0.020)\",\"image\":\"ff6f98c5-7a94-4f70-a612-0d140cd8ce28.png\"},{\"action\":\"Scroll down in the PDF document to view more content of the Immigration Department letter.\",\"code\":\"pyautogui.moveTo(x=0.508, y=0.456)\\npyautogui.scroll(-10)\",\"image\":\"368c4c6a-b462-4498-9aa0-e0f2e4bd72fd.png\"},{\"action\":\"Click on the pen/annotation tool in the PDF viewer toolbar to begin marking or selecting content for screenshot.\",\"code\":\"pyautogui.click(x=0.487, y=0.567)\",\"image\":\"5e62fb54-b0c6-4d96-825a-d9ab55354e72.png\"},{\"action\":\"Drag from the center of the document to the right to create a highlighting annotation across important text information about the electronic visa services.\",\"code\":\"pyautogui.moveTo(x=0.471, y=0.457)\\npyautogui.dragTo(x=0.586, y=0.464, button='left')\",\"image\":\"55717ed2-83d5-4879-80ae-dbc40f7089ba.png\"},{\"action\":\"Create a small highlight by dragging from the current position to a point slightly to the right in the document to mark another important piece of information in the electronic visa services section.\",\"code\":\"pyautogui.moveTo(x=0.389, y=0.472)\\npyautogui.dragTo(x=0.413, y=0.472, button='left')\",\"image\":\"6f1773df-f9f3-41fa-a640-a693fce4a89f.png\"},{\"action\":\"Click on the document navigation control in the PDF toolbar to finalize the annotations or switch view modes.\",\"code\":\"pyautogui.click(x=0.640, y=0.565)\",\"image\":\"357b3d14-7c27-4fa7-8303-83dfd3da08d9.png\"},{\"action\":\"Press Command+V to paste the clipboard content (the screenshot of the Immigration Department letter) into the WeChat conversation.\",\"code\":\"pyautogui.hotkey(['cmd', 'v'])\",\"image\":\"63958712-daa7-49bb-b23b-48add086e774.png\"},{\"action\":\"Press the Enter key to send the pasted screenshot to Wen Yunjie in the WeChat conversation.\",\"code\":\"pyautogui.press('enter')\",\"image\":\"168d5128-4e85-4fd5-bd56-aac5901581aa.png\"},{\"action\":\"Terminate the task with a success status as all goals have been accomplished.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"b2ab9c35-36a5-40b3-9432-2e1905e6de29.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241028144945_3398963c-d858-4154-ad96-151977060166","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"398e10c2f2bbe52716124c6369673a3dbac6aa1ecb7a8d8998fd3a16e07b27b1"}
{"messages":[{"role":"user","content":"On the MS Bing website, click More, click Online Games, click Thief Puzzle Game, and click Start Passing"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"More\\\" option (three dots) in the Bing navigation bar, which is located to the right of the \\\"News\\\" option.\",\"code\":\"pyautogui.click(x=0.474, y=0.092)\",\"image\":\"08ec92ff-9438-488b-8032-0ad5769b237f.png\"},{\"action\":\"Click on \\\"Online Games\\\" in the dropdown menu.\",\"code\":\"pyautogui.click(x=0.495, y=0.343)\",\"image\":\"f0e6ab02-5570-41c1-a3f8-7d29341ab1f8.png\"},{\"action\":\"Click on the Thief Puzzle Game tile featuring a stick figure character with a bag\",\"code\":\"pyautogui.click(x=0.363, y=0.288)\",\"image\":\"ce22b314-a7e0-4adf-9523-d13fcd566164.png\"},{\"action\":\"Click on the play button (circular button with the play triangle icon) in the center of the screen to start the Thief Puzzle game.\",\"code\":\"pyautogui.click(x=0.402, y=0.597)\",\"image\":\"4a293503-6756-45ba-bad4-b8d3735cd812.png\"},{\"action\":\"Click on the thief character's hand that appears to be reaching toward the other character on the bench to perform a stealing action.\",\"code\":\"pyautogui.click(x=0.273, y=0.602)\",\"image\":\"495afa23-9189-4982-ac37-78ddfdc93f32.png\"},{\"action\":\"Drag from the thief's hand to the brown bag, following the green trajectory line to complete the stealing action.\",\"code\":\"pyautogui.moveTo(x=0.277, y=0.602)\\npyautogui.dragTo(x=0.415, y=0.277, button='left')\",\"image\":\"adb7d8f5-a291-42e2-97c5-be79ae558f3d.png\"},{\"action\":\"Click the retry/restart button (circular arrow icon) at the bottom right of the game area to restart Level 2 after the failed attempt.\",\"code\":\"pyautogui.click(x=0.430, y=0.770)\",\"image\":\"21e2b284-9a1d-4a37-b50d-07032c587123.png\"},{\"action\":\"Click on the retry/restart button (circular arrow icon) on the left side of the control panel at the bottom of the game area.\",\"code\":\"pyautogui.click(x=0.437, y=0.766)\",\"image\":\"ebfea5c2-b00c-4aef-baf1-d1961ae91aa6.png\"},{\"action\":\"Click on Level 2 tile (the yellow square with the number \\\"2\\\" and a question mark icon) to enter Level 2 of the Thief Puzzle game.\",\"code\":\"pyautogui.click(x=0.408, y=0.380)\",\"image\":\"0d0258c0-44ae-4b8e-b31f-7f93e21e3d26.png\"},{\"action\":\"Drag from the thief character's hand (near the bottom left of the scene) to the triangular object hanging above in the middle of the screen, following the green dotted trajectory line.\",\"code\":\"pyautogui.moveTo(x=0.272, y=0.607)\\npyautogui.dragTo(x=0.441, y=0.188, button='left')\",\"image\":\"3368b1b3-9119-4cdf-9192-c5cf0978bf78.png\"},{\"action\":\"Click on the purple menu button with horizontal lines located at the bottom right of the game control panel.\",\"code\":\"pyautogui.click(x=0.359, y=0.760)\",\"image\":\"ed31c298-bae6-42c5-be93-8ef2b28feb4e.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114163747_52135a3b-7b97-485b-95fd-0ba9bfa18642","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c0df342c6dd57f1ca569996f026520178de39cab23cbee6770dcfc50dbbdb844"}
{"messages":[{"role":"user","content":"In RStudio, click File, create a new SQL script file, open the browser, copy the code to RStudio and click Preview. It will prompt whether to install the required package. Click Yes. After downloading, the running results will be displayed."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the File menu at the top-left corner of the RStudio interface to access file operations.\",\"code\":\"pyautogui.click(x=0.0098, y=0.0437)\",\"image\":\"bca753bc-6fb5-4de4-a38f-e85a365f2ded.png\"},{\"action\":\"Click on the \\\"New File...\\\" option in the File dropdown menu to open the submenu of file types that can be created.\",\"code\":\"pyautogui.click(x=0.0551, y=0.0674)\",\"image\":\"10d3ed95-6e80-4bf3-8c4a-ffd7b5554b23.png\"},{\"action\":\"Click on the \\\"SQL Script\\\" option in the New File submenu to create a new SQL script file.\",\"code\":\"pyautogui.click(x=0.3148, y=0.791)\",\"image\":\"828313a7-a125-4462-bae8-2918b384dcb8.png\"},{\"action\":\"Click the \\\"Yes\\\" button in the \\\"Install Required Packages\\\" dialog box to allow RStudio to install the DBI and RSQLite packages needed for SQL query previewing.\",\"code\":\"pyautogui.click(x=0.577, y=0.5)\",\"image\":\"f497bcd1-3e88-479b-849d-39c0107c2b1f.png\"},{\"action\":\"Click on the Chrome browser icon in the Windows taskbar to open the web browser.\",\"code\":\"pyautogui.click(x=0.475, y=0.9826)\",\"image\":\"cff0ecfe-a931-43b2-a66f-918ca226c66c.png\"},{\"action\":\"Click the \\\"复制代码\\\" (copy code) button next to the SQL script example to copy the SQL code to the clipboard.\",\"code\":\"pyautogui.click(x=0.8215, y=0.5618)\",\"image\":\"cc8282dc-bd97-4d3b-a7e0-09e38637f3ae.png\"},{\"action\":\"Click on the RStudio icon in the Windows taskbar to switch back to the RStudio application.\",\"code\":\"pyautogui.click(x=0.6594, y=0.9764)\",\"image\":\"7806912f-0ea7-4a02-8a8a-9706d8f64ada.png\"},{\"action\":\"Click the \\\"Preview\\\" button in the toolbar to run the SQL queries and display the results.\",\"code\":\"pyautogui.click(x=0.5078, y=0.15)\",\"image\":\"6254a184-d5cb-42e7-a598-f6f9461e7cb2.png\"},{\"action\":\"Click in the empty filename field to enter a name for the SQL script file.\",\"code\":\"pyautogui.click(x=0.243, y=0.391)\",\"image\":\"66d3eae5-83a5-49a3-af94-eae000b06c1a.png\"},{\"action\":\"Type \\\"sq\\\" in the filename field, press Enter, and then type \\\"_try1\\\" to complete the filename as \\\"sq_try1\\\".\",\"code\":\"pyautogui.write(message='sq')\\npyautogui.press('enter')\\npyautogui.write(message='_try1')\",\"image\":\"937600a2-1b40-44c2-8240-d43841937d72.png\"},{\"action\":\"Click the \\\"Save\\\" button to save the SQL script file with the name \\\"sq_try1.sql\\\".\",\"code\":\"pyautogui.click(x=0.4109, y=0.4889)\",\"image\":\"f4ca2e9a-9999-46ab-a046-c2992a1854bd.png\"},{\"action\":\"Click the Preview button in the SQL editor toolbar to execute the SQL script and view the results.\",\"code\":\"pyautogui.click(x=0.4934, y=0.166)\",\"image\":\"787b1173-903b-4ff8-ade3-85aa6ccf667c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241109193627_7923cd8a-701a-4d70-9390-febf1fa6a35f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"bb0e8175445a84c812d2e6f413a0f20557b292cf1d9c6430078247e5a3adccbe"}
{"messages":[{"role":"user","content":"Initiate a Tencent meeting and invite WeChat friends to join the meeting."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Double-click on the Tencent Meeting (腾讯会议) icon to launch the application.\",\"code\":\"pyautogui.doubleClick(x=0.0813, y=0.8639)\",\"image\":\"71ad234b-f214-41a6-a9ae-ee59522fa2e0.png\"},{\"action\":\"Click on the \\\"快速会议\\\" (Quick Meeting) button to start a new meeting immediately.\",\"code\":\"pyautogui.click(x=0.4339, y=0.4157)\",\"image\":\"d4b17d55-8e12-4a3b-ac8a-8afb1935829b.png\"},{\"action\":\"Click on the \\\"邀请\\\" (Invite) button in the control bar at the bottom of the meeting interface to access invitation options for adding participants to the meeting.\",\"code\":\"pyautogui.click(x=0.4005, y=0.825)\",\"image\":\"d2540718-8958-4eba-bc35-05875193e7a0.png\"},{\"action\":\"Click on the \\\"复制全部信息\\\" (Copy all information) button to copy the complete meeting details to the clipboard, which can then be shared with WeChat friends.\",\"code\":\"pyautogui.click(x=0.4307, y=0.7491)\",\"image\":\"57b99688-2b29-429b-aaed-d5c3c9e0bc8f.png\"},{\"action\":\"Click the close button (X) at the top-right corner of the meeting information panel to dismiss it.\",\"code\":\"pyautogui.click(x=0.7688, y=0.1352)\",\"image\":\"693a9b61-7bde-46c8-9ee7-722de18ad132.png\"},{\"action\":\"Click on a contact or chat entry in the left panel of the WeChat window to select the recipient for the meeting information.\",\"code\":\"pyautogui.click(x=0.2547, y=0.8806)\",\"image\":\"81fec753-2d93-4cc4-9f53-86bf9ba51fd9.png\"},{\"action\":\"Click on the video camera button in the Tencent Meeting control bar to toggle the video off.\",\"code\":\"pyautogui.click(x=0.8896, y=0.2148)\",\"image\":\"62c047a2-d864-49b3-bb09-429183a75075.png\"},{\"action\":\"Click on the text input field at the bottom of the WeChat chat window to focus it and prepare to paste the meeting invitation information.\",\"code\":\"pyautogui.click(x=0.3932, y=0.7917)\",\"image\":\"e6b1bf7c-f4cc-4374-90ba-8612149f9b4c.png\"},{\"action\":\"Right-click in the text input field at the bottom of the WeChat chat window to bring up the context menu that will include the paste option.\",\"code\":\"pyautogui.rightClick(x=0.3943, y=0.787)\",\"image\":\"78cabdc0-e110-4961-b013-c3f247e42b5b.png\"},{\"action\":\"Click on the \\\"粘贴\\\" (Paste) option in the context menu to paste the previously copied meeting invitation information into the WeChat text input field.\",\"code\":\"pyautogui.click(x=0.4224, y=0.8037)\",\"image\":\"021defad-50f6-4874-89d3-5d09a7eecb00.png\"},{\"action\":\"Click the \\\"发送(S)\\\" (Send) button at the bottom right of the text input field to send the meeting invitation to the contact.\",\"code\":\"pyautogui.click(x=0.6781, y=0.8398)\",\"image\":\"c359e5a1-e9c4-467c-ae0c-1d9112a2da51.png\"},{\"action\":\"Click the \\\"X\\\" close button in the top-right corner of the Tencent Meeting window to close the application or end the current meeting.\",\"code\":\"pyautogui.click(x=0.6984, y=0.2241)\",\"image\":\"589ce4fc-a9d1-4a76-bd1b-b08fea88e27a.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241116004628_d04e1468-bd28-4abb-830e-942f8883aab4","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8ac60169c27893141a7636338523f316b519a91f37518e9bc1d034d29b9c8586"}
{"messages":[{"role":"user","content":"Create a new page in Notion, add a title, set the page to calendar view, schedule a meeting for October 5th at 9:00 AM, add an icon to the page, and drag the new page to the home page."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the small button next to the \\\"Private\\\" label in the left sidebar.\",\"code\":\"pyautogui.click(x=0.1704, y=0.2903)\",\"image\":\"70f53749-bf6d-47e8-91c4-aeb03ea1e1f9.png\"},{\"action\":\"Type \\\"New Calendar\\\" to set the title for the new Notion page.\",\"code\":\"pyautogui.write(message='New Calendar')\",\"image\":\"a3a80620-6336-4647-b801-4b3f9f11f832.png\"},{\"action\":\"Click in the empty content area below the \\\"New Calendar\\\" title to begin adding content to the page.\",\"code\":\"pyautogui.click(x=0.4162, y=0.291)\",\"image\":\"1ff4d5fa-4e99-4d41-ae8c-81dda307a3f5.png\"},{\"action\":\"Type \\\"/calen\\\" to bring up calendar-related block options in Notion's command menu.\",\"code\":\"pyautogui.write(message='/calen')\",\"image\":\"20013896-1bb1-4fbb-b695-c6832b369df2.png\"},{\"action\":\"Click on the \\\"Calendar View\\\" option in the dropdown menu to create a calendar database view.\",\"code\":\"pyautogui.click(x=0.4505, y=0.3493)\",\"image\":\"cc65b35a-8d40-4810-a9c2-2ac93cb77fe8.png\"},{\"action\":\"Click on the \\\"New calendar\\\" option in the New view dialog to create a blank calendar database.\",\"code\":\"pyautogui.click(x=0.8278, y=0.4069)\",\"image\":\"9678cb7e-aac2-488e-bcf5-ec1139d17241.png\"},{\"action\":\"Click on the time icon next to the date field to set the meeting time.\",\"code\":\"pyautogui.click(x=0.6333, y=0.3014)\",\"image\":\"7f7040b1-4c45-4765-8e1d-8a52e6fad3e8.png\"},{\"action\":\"Click on the \\\"Include time\\\" toggle switch to enable time setting for the meeting.\",\"code\":\"pyautogui.click(x=0.5315, y=0.775)\",\"image\":\"572b922a-5d03-40a9-85d4-742d6c53f101.png\"},{\"action\":\"Click on the time field showing \\\"12:00 AM\\\" to edit the meeting time.\",\"code\":\"pyautogui.click(x=0.4741, y=0.3431)\",\"image\":\"c1362267-9ffc-4f53-947c-ef403e6142b0.png\"},{\"action\":\"Select the \\\"12\\\" in the time field by dragging from the beginning of \\\"12:00 AM\\\" to just before the colon, in order to replace it with \\\"9\\\" and set the meeting time to 9:00 AM.\",\"code\":\"pyautogui.moveTo(x=0.4704, y=0.3417)\\npyautogui.dragTo(x=0.4565, y=0.35, button='left')\",\"image\":\"0d51c7f9-8a70-4c3f-bf84-086936fc6ee5.png\"},{\"action\":\"Click on the \\\"Oct 5, 2024\\\" text at the top of the date picker dialog to confirm the date and time selection of October 5th at 9:00 AM.\",\"code\":\"pyautogui.click(x=0.5972, y=0.2938)\",\"image\":\"8b3457a9-d4e3-4e5b-92d4-120b7110529b.png\"},{\"action\":\"Click on the page customization button in the upper right corner of the content area to access options for adding an icon to the page.\",\"code\":\"pyautogui.click(x=0.9042, y=0.3236)\",\"image\":\"40a23ea4-b438-4810-9894-17e2cc9665ef.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241005110606_59cde677-1f9b-498c-9359-d8d69f2cd8ec","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"6613419b4cb27854251a9d1726112e6755d0af7d56ab08dfb679a44e85f212fe"}
{"messages":[{"role":"user","content":"Go to the official NeurIPS 2024 website to find the conference dates"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Type \\\"neurips 2024\\\" in the search bar and press Enter to search for information about the NeurIPS 2024 conference.\",\"code\":\"pyautogui.write(message='neurips 2024')\\npyautogui.press('enter')\",\"image\":\"798031b4-4436-48b4-8709-672d5f9c357b.png\"},{\"action\":\"Click on the \\\"NeurIPS 2024\\\" title link in the first search result to navigate to the official conference website.\",\"code\":\"pyautogui.click(x=0.1656, y=0.2854)\",\"image\":\"933cca24-cc5e-4b23-bb55-6c9df532dfb2.png\"},{\"action\":\"Terminate the session with success status as the goal has been achieved.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"b451edb7-2ee3-4699-b925-9f437d629b2c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241011040004_af17e338-1540-4514-b307-c5bc4a9387a9","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c2f94a6078c259361d24d5219fa530b34e44313a1512011da523140459c17a2d"}
{"messages":[{"role":"user","content":"Check my St Andrews email, 163 email, as well as Gmail, open the email related to SCS Graduate Admission and follow the instructions to open the application portal."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Email\\\" tab in the browser tab bar to switch to the email service.\",\"code\":\"pyautogui.click(x=0.1623, y=0.0526)\",\"image\":\"dbdd88ee-eade-438f-8980-4a057b7f28d6.png\"},{\"action\":\"Click the refresh button in the address bar to reload the St Andrews Outlook email page.\",\"code\":\"pyautogui.click(x=0.0764, y=0.1015)\",\"image\":\"e6be67ad-c6ca-4e3c-a7a4-1aa97b3c78c5.png\"},{\"action\":\"Click on the browser tab labeled \\\"10366\\\" to switch to that tab.\",\"code\":\"pyautogui.click(x=0.4013, y=0.0476)\",\"image\":\"6da277c3-480c-45f7-b0b6-5bb12df50f0e.png\"},{\"action\":\"Click the refresh button in the browser's navigation bar to reload the 163 email page.\",\"code\":\"pyautogui.click(x=0.0772, y=0.0922)\",\"image\":\"5f71d901-9a5e-4aa1-b0ec-198988abd837.png\"},{\"action\":\"Click the Gmail tab icon to switch to the Gmail email service.\",\"code\":\"pyautogui.click(x=0.4658, y=0.041)\",\"image\":\"61fa43b1-02fd-43a9-a9b2-0db81b151815.png\"},{\"action\":\"Click on the email from \\\"SCS Graduate Admiss.\\\" with the subject \\\"Application Submitted - MS in Computational Data Science (MCDS) Program\\\" to open and read its contents.\",\"code\":\"pyautogui.click(x=0.3097, y=0.3799)\",\"image\":\"e33aa717-ba02-41c8-9501-c138826caede.png\"},{\"action\":\"Click on the red \\\"Application Portal\\\" button in the email to access the Carnegie Mellon University application portal.\",\"code\":\"pyautogui.click(x=0.5529, y=0.4955)\",\"image\":\"0a9c26d1-856b-45b8-b194-80049dbb04b6.png\"},{\"action\":\"Type \\\"hkuyuyan@gmail.com\\\" in the Email Address field to begin the login process.\",\"code\":\"pyautogui.write(message='hkuyuyan@gmail.com')\",\"image\":\"374f697d-d14a-4138-b2c1-55b325684de2.png\"},{\"action\":\"Click on the Password field to position the cursor for entering the password.\",\"code\":\"pyautogui.click(x=0.2877, y=0.5146)\",\"image\":\"24facb5d-21c5-475c-8775-28b59ba9e3d7.png\"},{\"action\":\"Click the \\\"Login\\\" button to submit the login credentials and access the Carnegie Mellon University application portal.\",\"code\":\"pyautogui.click(x=0.1134, y=0.5719)\",\"image\":\"42c6a810-58b9-4828-afec-55219d769975.png\"},{\"action\":\"Terminate the task with a success status.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"e1d4f349-5d83-4f79-8ce8-c0ea3eaccdbb.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241118115802_a18ad956-b375-45c4-a43e-4021a4316662","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"81fc053e6846c4b4a6b4f3997eeb12678ad137f23915e759b56b8d7fc3fce35a"}
{"messages":[{"role":"user","content":"Book Flights with Shortest Layovers to Seoul on May 3, 2025, using Skyscanner website"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"From\\\" input field to begin entering the departure location for the flight search.\",\"code\":\"pyautogui.click(x=0.147, y=0.582)\",\"image\":\"2caa963f-d463-46d9-845b-9b06f0c8ddf9.png\"},{\"action\":\"Type \\\"san\\\" in the From field to begin entering the departure location.\",\"code\":\"pyautogui.write(message='san')\",\"image\":\"ca5ddb0c-bef3-48b1-926b-26e9b673eec4.png\"},{\"action\":\"Click on \\\"San Francisco International, CA (SFO)\\\" in the dropdown menu to select it as the departure location.\",\"code\":\"pyautogui.click(x=0.186, y=0.673)\",\"image\":\"5aa32de5-5243-4b0c-bd71-ec6446b1854d.png\"},{\"action\":\"Type \\\"seoul\\\" in the \\\"To\\\" field to specify the destination.\",\"code\":\"pyautogui.write(message='seoul')\",\"image\":\"4d947f4f-65a9-4d68-9afd-b865579f726b.png\"},{\"action\":\"Click on \\\"Incheon International (ICN)\\\" in the destination dropdown menu to select it as the arrival airport.\",\"code\":\"pyautogui.click(x=0.365, y=0.771)\",\"image\":\"d7cf6486-5c1e-43d2-9213-da54e7b40a92.png\"},{\"action\":\"Click on the departure date field (08/05/2025) to open the date picker and change the date.\",\"code\":\"pyautogui.click(x=0.488, y=0.570)\",\"image\":\"b9da8a5b-ffd0-4bb7-9adf-5de2460bca3a.png\"},{\"action\":\"Click on the \\\"3\\\" date in the May calendar to select May 3, 2025 as the departure date.\",\"code\":\"pyautogui.moveTo(x=0.496, y=0.534)\",\"image\":\"59486c7a-b35c-44c4-8580-3b0287a1de56.png\"},{\"action\":\"Click on the \\\"Return\\\" dropdown button to specify whether this will be a round-trip or one-way flight.\",\"code\":\"pyautogui.click(x=0.275, y=0.546)\",\"image\":\"5850569c-3c37-426c-bb2c-7133a1d6b1cf.png\"},{\"action\":\"Click on the \\\"One way\\\" option in the trip type dropdown menu to select a one-way flight.\",\"code\":\"pyautogui.click(x=0.310, y=0.613)\",\"image\":\"49e852b8-f4a1-43b2-b2d4-1cb66c5b3a7b.png\"},{\"action\":\"Scroll down slightly to see more of the flight search interface below the calendar.\",\"code\":\"pyautogui.moveTo(x=0.513, y=0.638)\\npyautogui.scroll(-10)\",\"image\":\"f6911f63-6273-4b44-93b0-963703fb1c34.png\"},{\"action\":\"Scroll down slightly in the calendar area to see more of the interface below.\",\"code\":\"pyautogui.moveTo(x=0.605, y=0.633)\\npyautogui.scroll(-10)\",\"image\":\"60fe7c2d-95f2-4f8e-bdac-b9c2e8c980e4.png\"},{\"action\":\"Click the \\\"Search one way\\\" button at the bottom of the calendar modal to confirm the date selection and execute the flight search.\",\"code\":\"pyautogui.click(x=0.808, y=0.869)\",\"image\":\"80cf135e-9428-4c37-97c3-4c75af3f8bc5.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241120133808_cc43e56f-624d-41df-91c9-c6d0775e70fc","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"f3f01282c68d6a817af224f10b6fda63b62613c191a24cfe69293b293074ae49"}
{"messages":[{"role":"user","content":"Go yo Youtube Studio, look through my dashboard, content, analytics and comments, then change the default currency to HKD"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search button in the NAVER search bar to prepare for entering a search query.\",\"code\":\"pyautogui.click(x=0.506, y=0.220)\",\"image\":\"dda4c6ae-e0b6-455b-ac49-e0e4cd297870.png\"},{\"action\":\"Click on the \\\"YouTube\\\" link at the top of the search results to navigate to the YouTube website.\",\"code\":\"pyautogui.click(x=0.148, y=0.364)\",\"image\":\"52c78d87-e47d-4882-854c-f8a1e4a4c5b9.png\"},{\"action\":\"Scroll down in the left navigation sidebar to reveal more menu options, looking for YouTube Studio or related settings.\",\"code\":\"pyautogui.moveTo(x=0.096, y=0.618)\\npyautogui.scroll(-10)\",\"image\":\"8982219c-bce6-4b3a-9b9e-31d7d7dbd1c6.png\"},{\"action\":\"Click on the \\\"YouTube Studio\\\" option in the left navigation sidebar under the \\\"More from YouTube\\\" section.\",\"code\":\"pyautogui.click(x=0.105, y=0.764)\",\"image\":\"99c1a13a-50b8-4a14-91c5-672230ba7fce.png\"},{\"action\":\"Scroll down in the left navigation sidebar to view more options, focusing on the Settings area where currency preferences would be located.\",\"code\":\"pyautogui.moveTo(x=0.376, y=0.546)\\npyautogui.scroll(-10)\",\"image\":\"f5b450c2-4ff8-472d-b361-86bd99f4fac3.png\"},{\"action\":\"Click on the \\\"Analytics\\\" option in the left navigation sidebar to access the channel analytics section.\",\"code\":\"pyautogui.click(x=0.122, y=0.597)\",\"image\":\"6bc144dd-5526-477f-9e8e-48be0f717a0c.png\"},{\"action\":\"Scroll down in the Analytics section to view more analytics data below the current view.\",\"code\":\"pyautogui.moveTo(x=0.334, y=0.440)\\npyautogui.scroll(10)\",\"image\":\"1d2f4c41-bb05-46f5-8ffc-45c79b148020.png\"},{\"action\":\"Click on the \\\"Content\\\" tab in the Channel analytics navigation bar to view content-specific analytics.\",\"code\":\"pyautogui.click(x=0.309, y=0.346)\",\"image\":\"41331ed0-87aa-4d0c-aaea-d10c38830080.png\"},{\"action\":\"Click on the \\\"Audience\\\" tab in the Channel analytics navigation bar to view audience-related analytics.\",\"code\":\"pyautogui.click(x=0.367, y=0.350)\",\"image\":\"a478d90a-294a-477b-879c-58e2dcebbb8b.png\"},{\"action\":\"Click on the \\\"Inspiration\\\" tab in the Channel analytics navigation bar to view inspiration-related analytics.\",\"code\":\"pyautogui.click(x=0.438, y=0.358)\",\"image\":\"641bb7b1-9b4e-43a0-a028-63c0d6218b76.png\"},{\"action\":\"Scroll down slightly in the left navigation sidebar to better position the mouse near the \\\"Comments\\\" option, preparing to click on it next.\",\"code\":\"pyautogui.moveTo(x=0.125, y=0.702)\\npyautogui.scroll(10)\",\"image\":\"871b3994-e5bb-4e39-a892-0e2f1ba196a5.png\"},{\"action\":\"Click on the \\\"Comments\\\" option in the left navigation sidebar, which has a red notification dot next to it.\",\"code\":\"pyautogui.click(x=0.105, y=0.659)\",\"image\":\"d2456477-6ee6-4e1a-b620-20c651d6dbd4.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241022125124_1166574d-6f1c-41f5-9bb8-bdcbbba0c26d","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"b32fc18fe653c93663edbb0f4ac0edc8165e66a0a9ce21788c777d2079a56652"}
{"messages":[{"role":"user","content":"Open Google Maps from the Run window. Go to Saved, delete the Hello list, and create a new list named Wen. Input List in the list description, then select the map location Albuquerque. Next, select Rio Puerco Ridge and save this address to the Wen list. Open the Wen list, go to Sharing options, set it to Shared, and enable Let others edit your list. Close the Sharing options."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Maps thumbnail in the taskbar to restore the minimized Google Maps window.\",\"code\":\"pyautogui.click(x=0.1688, y=0.9037)\",\"image\":\"cb7416a6-132e-47d6-b128-4ef74f9c3629.png\"},{\"action\":\"Click on the \\\"Saved\\\" icon (bookmark/flag symbol) in the left sidebar to access the saved places interface.\",\"code\":\"pyautogui.click(x=0.0214, y=0.1676)\",\"image\":\"193cffe2-0c56-4d0b-bdda-331d0d47cf32.png\"},{\"action\":\"Click on the delete/share icon next to the \\\"hello\\\" list to access list management options.\",\"code\":\"pyautogui.click(x=0.2536, y=0.2963)\",\"image\":\"c1ee8f12-32f5-49ab-9682-7d0547594857.png\"},{\"action\":\"Click on the \\\"Delete list\\\" option at the bottom of the dropdown menu to begin the process of deleting the \\\"hello\\\" list.\",\"code\":\"pyautogui.click(x=0.2865, y=0.4556)\",\"image\":\"1f970122-ae47-411c-925a-cce0fce2e00e.png\"},{\"action\":\"Click the \\\"DELETE\\\" button to confirm deletion of the \\\"hello\\\" list.\",\"code\":\"pyautogui.click(x=0.5391, y=0.5759)\",\"image\":\"dfbe44b4-41ef-41c6-a180-d8d9e52f083a.png\"},{\"action\":\"Click on the \\\"Create list\\\" button (blue \\\"+\\\" icon) next to \\\"Your Lists\\\" to begin creating a new list.\",\"code\":\"pyautogui.click(x=0.237, y=0.2296)\",\"image\":\"d205087e-5f7b-4050-a312-d5950732806b.png\"},{\"action\":\"Type \\\"wen\\\" in the list name field and press Enter to create the new list.\",\"code\":\"pyautogui.write(message='wen')\\npyautogui.press('enter')\",\"image\":\"23dd0d1a-ded1-4ef2-b59f-698c41b65f7d.png\"},{\"action\":\"Click the \\\"CREATE\\\" button to finalize the creation of the new list.\",\"code\":\"pyautogui.click(x=0.5865, y=0.5833)\",\"image\":\"7d94fb92-d13a-4964-9367-207ae1dbcef2.png\"},{\"action\":\"Click in the list description text area below the list name field to begin entering the description for the \\\"wen\\\" list.\",\"code\":\"pyautogui.click(x=0.1516, y=0.3796)\",\"image\":\"ffdbd9ff-7c0d-404b-9b79-0eb697beaca5.png\"},{\"action\":\"Click on Albuquerque on the map to select this location.\",\"code\":\"pyautogui.click(x=0.4083, y=0.5074)\",\"image\":\"5cc5f607-42c1-4b5b-b672-6e1f5ff2bd40.png\"},{\"action\":\"Click on the \\\"Rio Puerco Bridge\\\" text label on the map to select this location.\",\"code\":\"pyautogui.click(x=0.4974, y=0.5833)\",\"image\":\"ede353af-6889-4e21-a9ad-0e620e98a912.png\"},{\"action\":\"Click on the \\\"Save\\\" button (bookmark icon) to save the Rio Puerco Bridge to a list.\",\"code\":\"pyautogui.click(x=0.1115, y=0.5037)\",\"image\":\"27cdd32e-f7af-48c8-82b3-1de8cd46aaf8.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241210212821_84cb586d-55ac-4cf5-a7da-70ebb357ef80","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"3253d6a333b519da8fc3738ec939652504bcf5b16ba605cac6f00b241f0e427a"}
{"messages":[{"role":"user","content":"change the appearance of my calendar to light and change the time scale to 30 minutes"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Calendar settings button (gear icon) in the toolbar to access calendar configuration options.\",\"code\":\"pyautogui.click(x=0.7516, y=0.145)\",\"image\":\"1f7f6861-23be-4254-a113-3f3fc11aa030.png\"},{\"action\":\"Click on the \\\"General\\\" option in the Settings navigation panel to access the appearance settings, where the theme can be changed from dark to light.\",\"code\":\"pyautogui.click(x=0.1104, y=0.3183)\",\"image\":\"fed3fe18-97cd-4f30-8805-e5f4107f5b8f.png\"},{\"action\":\"Click on the \\\"Appearance\\\" option in the General settings submenu to access theme settings where we can change from dark to light mode.\",\"code\":\"pyautogui.click(x=0.2594, y=0.2175)\",\"image\":\"874ecfb2-06cc-443f-ac19-a652b0feed24.png\"},{\"action\":\"Click on the Light theme option in the Dark mode section to change the appearance from dark to light.\",\"code\":\"pyautogui.click(x=0.4313, y=0.3217)\",\"image\":\"7a53c7a5-2204-4a0a-885c-c05610492dbe.png\"},{\"action\":\"Click the \\\"Save\\\" button at the bottom of the Appearance settings panel to save the theme change.\",\"code\":\"pyautogui.click(x=0.6432, y=0.8783)\",\"image\":\"be8d631f-6f4c-4f7c-b625-379f42b6eb1c.png\"},{\"action\":\"Click on the \\\"Calendar\\\" option in the left navigation panel of the Settings window to access calendar-specific settings.\",\"code\":\"pyautogui.click(x=0.1302, y=0.3958)\",\"image\":\"0e288c29-a8ed-426d-86e5-a44d4d8cf2c3.png\"},{\"action\":\"Click on the time scale dropdown field that currently displays \\\"15 minutes\\\" to open the dropdown menu and access the time interval options.\",\"code\":\"pyautogui.click(x=0.7005, y=0.4242)\",\"image\":\"b72505f0-db2c-4823-a465-b95aad7c3218.png\"},{\"action\":\"Click on the \\\"30 minutes\\\" option in the Time scale dropdown menu to change the time scale from 15 minutes to 30 minutes.\",\"code\":\"pyautogui.click(x=0.4536, y=0.6442)\",\"image\":\"424e6ec8-3518-4cb0-95d2-bd1ac3a48a06.png\"},{\"action\":\"Click the \\\"Save\\\" button at the bottom of the Calendar settings window to confirm and apply the changes made to both the appearance and time scale settings.\",\"code\":\"pyautogui.click(x=0.6411, y=0.8775)\",\"image\":\"f4134cf2-c8f1-4503-b408-75c0039d6f39.png\"},{\"action\":\"Click the close (X) button in the top-right corner of the settings dialog to exit the settings interface.\",\"code\":\"pyautogui.click(x=0.7547, y=0.1017)\",\"image\":\"25ba4cc2-1422-43db-ad23-2268e21d2f62.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241013092959_dd4578e3-fb3d-4af8-9ee8-0c37ddf31892","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"802c501035a16b013be1b2004217e3837e7681ed724b0566b72947b5b20dfa2d"}
{"messages":[{"role":"user","content":"Create a new task 'Web Agent Task' at 10:00 AM on Nov 4 in Google Calendar, set the task to be repeated every 2 weeks on Monday, to end after 6 occurrences."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Create\\\" button in the left sidebar to start creating a new event in Google Calendar.\",\"code\":\"pyautogui.click(x=0.0359, y=0.2525)\",\"image\":\"0d59133b-ea15-4c7e-bd21-f950837a11e4.png\"},{\"action\":\"Click on the \\\"Task\\\" option in the dropdown menu to begin creating a new task in Google Calendar.\",\"code\":\"pyautogui.click(x=0.052, y=0.37)\",\"image\":\"b4affc3b-9336-4eb1-bb4d-8961f22daa8f.png\"},{\"action\":\"Type \\\"Web Agent Task\\\" in the title field of the task creation dialog.\",\"code\":\"pyautogui.write(message='Web Agent Task')\",\"image\":\"a31fed55-6f9a-4e2c-997c-dd7eaa22b2f0.png\"},{\"action\":\"Click on the date and time field showing \\\"Nov 1, 2024, 3:00pm\\\" to open the date/time settings editor.\",\"code\":\"pyautogui.click(x=0.5082, y=0.5006)\",\"image\":\"607cd01c-c10a-4248-b7cf-a77544879eec.png\"},{\"action\":\"Type \\\"Nov\\\" in the currently highlighted date field to begin entering the new date for the task.\",\"code\":\"pyautogui.write(message='Nov')\",\"image\":\"00fafc7e-17d0-4c81-a180-80feb59a2c32.png\"},{\"action\":\"Click on the number \\\"4\\\" in the November 2024 calendar to select November 4th as the task date.\",\"code\":\"pyautogui.click(x=0.4883, y=0.6438)\",\"image\":\"7046aaa0-ba5a-43cb-ae4d-1168876b248d.png\"},{\"action\":\"Click on the time field showing \\\"3:00pm\\\" to edit the time for the task.\",\"code\":\"pyautogui.click(x=0.5828, y=0.4894)\",\"image\":\"2f51e95b-a4cc-4f22-ad04-629ee944a2a4.png\"},{\"action\":\"Click on the \\\"10:00am\\\" option in the time dropdown menu to set the task time.\",\"code\":\"pyautogui.click(x=0.6012, y=0.6425)\",\"image\":\"2d7de1cd-0089-434a-abe3-e20c9ec27408.png\"},{\"action\":\"Click on the \\\"Does not repeat\\\" text to open the recurrence settings.\",\"code\":\"pyautogui.click(x=0.5891, y=0.5719)\",\"image\":\"df15d5ab-1418-49d4-aba8-cc539b7dc534.png\"},{\"action\":\"Click on the \\\"Custom...\\\" option in the recurrence dropdown menu to access detailed recurrence settings for the task.\",\"code\":\"pyautogui.click(x=0.5461, y=0.5331)\",\"image\":\"c35fe4ab-7016-4ae6-abc4-d857056272b5.png\"},{\"action\":\"Type \\\"2\\\" in the highlighted \\\"Repeat every\\\" field to set the task to repeat every 2 weeks.\",\"code\":\"pyautogui.write(message='2')\",\"image\":\"0ab63f91-f7b4-4a04-a00b-f348b4877168.png\"},{\"action\":\"Click on the \\\"After\\\" radio button in the \\\"Ends\\\" section of the Custom recurrence dialog box.\",\"code\":\"pyautogui.click(x=0.3805, y=0.7125)\",\"image\":\"705a21ac-b063-4c35-9fdd-1fc2ad8031f0.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241101171703_0502fb37-f91c-46d6-ac78-eca70148110f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"9f1b852ec709a26bd50353cab1e2f85b2b53d436cf91fb0ac280b19dfabf1c71"}
{"messages":[{"role":"user","content":"Add auto filter trash emails rule when I'm not on the To line to junk emails in my Microsoft Outlook and instantly run it"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the settings/gear icon in the top-right area of the Outlook interface to access Outlook settings.\",\"code\":\"pyautogui.click(x=0.918, y=0.1593)\",\"image\":\"a2622644-5f25-4a75-a352-cc093cfacd86.png\"},{\"action\":\"Click on the \\\"Rules\\\" option in the settings menu to access the email rules configuration page.\",\"code\":\"pyautogui.click(x=0.2498, y=0.3447)\",\"image\":\"34e29e93-62ff-4c4b-a1a4-e2c1c081a273.png\"},{\"action\":\"Click on the \\\"Add new rule\\\" button to begin creating a new email filtering rule.\",\"code\":\"pyautogui.click(x=0.4399, y=0.3345)\",\"image\":\"6ddec2cd-7fca-4e8d-998f-b854221317b4.png\"},{\"action\":\"Click on the \\\"Name your rule\\\" text field to begin entering a name for the new rule.\",\"code\":\"pyautogui.click(x=0.4746, y=0.2941)\",\"image\":\"e0dbdf3f-f57e-4c2f-bd5d-a9cff9fc8a5f.png\"},{\"action\":\"Type \\\"Filter1 TRASH email\\\" in the \\\"Name your rule\\\" text field to name the email filtering rule.\",\"code\":\"pyautogui.write(message='Filter1 TRASH email')\",\"image\":\"b04f798c-b869-4b92-9b42-4828446242c5.png\"},{\"action\":\"Click on the \\\"Select a condition\\\" dropdown to reveal the available condition options for the email rule.\",\"code\":\"pyautogui.click(x=0.4601, y=0.3925)\",\"image\":\"8dc09086-9018-4797-9b60-7b856e158397.png\"},{\"action\":\"Click on the \\\"I'm not on the To line\\\" option from the dropdown menu to select this condition for the email filtering rule.\",\"code\":\"pyautogui.click(x=0.6471, y=0.4518)\",\"image\":\"b29de92d-8623-43ad-8bb2-a5302a4295c4.png\"},{\"action\":\"Click on the \\\"Select an action\\\" dropdown to reveal the available action options for the email rule.\",\"code\":\"pyautogui.click(x=0.4968, y=0.5378)\",\"image\":\"01bf9a62-c5fe-40ad-b5c9-8254b805392e.png\"},{\"action\":\"Click on the \\\"Mark as Junk\\\" option in the dropdown menu to set this as the action for emails that match the \\\"I'm not on the To line\\\" condition.\",\"code\":\"pyautogui.click(x=0.6397, y=0.7437)\",\"image\":\"ebac0814-2ac1-4ffc-b1cc-48c70b4ebd7b.png\"},{\"action\":\"Click on the \\\"Run rule now\\\" checkbox to enable immediate processing of existing emails when the rule is saved.\",\"code\":\"pyautogui.click(x=0.3962, y=0.7161)\",\"image\":\"643fed55-64f8-4542-8a7f-2819d90a7a3b.png\"},{\"action\":\"Click the \\\"Save\\\" button to finalize and activate the email filtering rule.\",\"code\":\"pyautogui.click(x=0.8072, y=0.9138)\",\"image\":\"30fef9a8-9288-4513-aa77-a602beb949a6.png\"},{\"action\":\"Terminate the task with success status as the goal has been fully accomplished.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"4072f3d5-6d9f-407b-a532-12a5fb3eaf99.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241003145403_695fb1c5-19c0-47d2-bbd1-3f5ebefdbc79","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"ed11bf4b7524b6f918f5884d355f951e03a8829eec6ca6f47ec6ecac553f098f"}
{"messages":[{"role":"user","content":"Open IKEA in google browser, search vase, view all screening conditions, select Buyable online, adjust the size to the bottle height of 8+ \"and the price to $15.00-19.99, view all qualified products, view products and view comments, and select to view comments of other countries"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google search bar to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.467, y=0.451)\",\"image\":\"99cc3552-1498-484b-97c9-407b7a710744.png\"},{\"action\":\"Click on the \\\"ikea - Google Search\\\" suggestion in the dropdown menu.\",\"code\":\"pyautogui.click(x=0.370, y=0.519)\",\"image\":\"916b7a4c-50f1-4686-a9d2-9f5c0c94c7bc.png\"},{\"action\":\"Click on the IKEA official website link that shows the IKEA logo and \\\"https://www.ikea.com\\\" URL.\",\"code\":\"pyautogui.click(x=0.174, y=0.388)\",\"image\":\"dc979f76-e6de-4549-af60-c7e44e29d369.png\"},{\"action\":\"Click on the search bar that contains the placeholder text \\\"What are you looking for?\\\" to begin typing a search query for vases.\",\"code\":\"pyautogui.click(x=0.280, y=0.208)\",\"image\":\"30580959-4b65-498a-9e82-d118f118ba6e.png\"},{\"action\":\"Type \\\"vase\\\" in the search box and press Enter to search for vase products.\",\"code\":\"pyautogui.write(message='vase')\\npyautogui.press('enter')\",\"image\":\"3a026a03-6bbe-4d2b-a61e-7f4937aa12f5.png\"},{\"action\":\"Scroll down to view more vase products and filtering options on the search results page.\",\"code\":\"pyautogui.moveTo(x=0.423, y=0.456)\\npyautogui.scroll(-10)\",\"image\":\"f1fb0576-a89c-4153-bcfa-032c2724b36e.png\"},{\"action\":\"Click on the \\\"All filters\\\" button to access comprehensive filtering options for the vase search results.\",\"code\":\"pyautogui.click(x=0.803, y=0.313)\",\"image\":\"5c356ab5-3cc9-4152-98ed-ee828ea349c7.png\"},{\"action\":\"Click on the \\\"Buyable online\\\" option in the filter panel to enable this filter.\",\"code\":\"pyautogui.click(x=0.954, y=0.781)\",\"image\":\"9fd3745c-5de4-441d-af80-ffb52c8b7b3a.png\"},{\"action\":\"Click on the settings/edit icon next to the \\\"Size\\\" filter option to access detailed size configuration options.\",\"code\":\"pyautogui.click(x=0.950, y=0.392)\",\"image\":\"f06d7490-3c66-4654-bf3d-46e1649aed32.png\"},{\"action\":\"Scroll down in the Size filter panel to find more height options\",\"code\":\"pyautogui.moveTo(x=0.917, y=0.557)\\npyautogui.scroll(-10)\",\"image\":\"f8425682-da4c-4d5d-a265-b18ddf34a3df.png\"},{\"action\":\"Click on the checkbox next to the \\\"8+ \\\"\\\" height option to select vases with a height of 8 inches or taller.\",\"code\":\"pyautogui.click(x=0.952, y=0.670)\",\"image\":\"ea72fc64-8465-4fbe-af85-7ae47abb4146.png\"},{\"action\":\"Scroll down in the filter panel to reveal more filtering options, specifically to find the price filter.\",\"code\":\"pyautogui.moveTo(x=0.850, y=0.617)\\npyautogui.scroll(-10)\",\"image\":\"a9a346ae-ee3f-4ecf-b9f5-946d4ca48f6a.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241226142643_75ab72e6-948c-4ae4-b3e7-5121ab3e0a51","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"a5623232572e563558881a82993e9503b3166cb642dceee8356e69fb722b95a7"}
{"messages":[{"role":"user","content":"Create a new github repo called agentnet-external-manage, set it private, add a readme, select None as gitignore"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click the plus (+) button in the upper-right corner of the browser to open a new tab.\",\"code\":\"pyautogui.click(x=0.377, y=0.033)\",\"image\":\"028edf8f-4886-4782-b2a9-991b3137f200.png\"},{\"action\":\"Press the down arrow key to highlight the first GitHub search result, then press Enter to navigate to GitHub's homepage.\",\"code\":\"pyautogui.press('down')\\npyautogui.press('enter')\",\"image\":\"e0b996ed-8b01-4931-8d28-d93424a1659a.png\"},{\"action\":\"Click on the \\\"Your repositories\\\" option in the dropdown menu to navigate to the repositories page.\",\"code\":\"pyautogui.click(x=0.863, y=0.304)\",\"image\":\"21e56982-c877-45af-a8e6-d6c65fa898f9.png\"},{\"action\":\"Click on the green \\\"New\\\" button to begin creating a new repository.\",\"code\":\"pyautogui.click(x=0.929, y=0.290)\",\"image\":\"5b08d541-5c59-4486-b3dd-8890d040ef7f.png\"},{\"action\":\"Click on the repository name text field to begin entering the name for the new repository.\",\"code\":\"pyautogui.click(x=0.400, y=0.604)\",\"image\":\"26e933b2-5375-4bf0-8cbf-eb2724c2ccdd.png\"},{\"action\":\"Type \\\"agentnet-external-manage\\\" in the repository name field.\",\"code\":\"pyautogui.write(message='agentnet-external-manage')\",\"image\":\"6f621241-c09c-4c73-9051-f07caeb7b72e.png\"},{\"action\":\"Scroll down on the repository creation page to view the remaining options for repository settings, including the Private option, README checkbox, and gitignore dropdown.\",\"code\":\"pyautogui.moveTo(x=0.380, y=0.728)\\npyautogui.scroll(-10)\",\"image\":\"dd78be82-46e3-4d68-81ea-bf026282f6dd.png\"},{\"action\":\"Click on the Private radio button to set the repository visibility to private.\",\"code\":\"pyautogui.click(x=0.221, y=0.798)\",\"image\":\"b9707186-b812-468c-8a28-86abbe99841f.png\"},{\"action\":\"Click the checkbox next to \\\"Add a README file\\\" to enable README creation for the repository.\",\"code\":\"pyautogui.click(x=0.224, y=0.886)\",\"image\":\"330e97ab-eeb2-414f-97a2-e9c4eecfbaef.png\"},{\"action\":\"Click on the .gitignore template dropdown that currently shows \\\"None\\\" to confirm or select this option.\",\"code\":\"pyautogui.click(x=0.340, y=0.541)\",\"image\":\"00361985-d011-4670-a7a0-5bcccfc33624.png\"},{\"action\":\"Click on the \\\"None\\\" option in the .gitignore template dropdown menu.\",\"code\":\"pyautogui.click(x=0.350, y=0.682)\",\"image\":\"0e4f6e61-1620-4a85-af9f-44276aa77f74.png\"},{\"action\":\"Scroll down slightly on the repository creation page to fully reveal the \\\"Create repository\\\" button and any remaining options.\",\"code\":\"pyautogui.moveTo(x=0.523, y=0.764)\\npyautogui.scroll(-10)\",\"image\":\"b74882e1-f053-4c28-913f-6af9b002268f.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241013171009_699092f4-bfdd-428c-8a19-c56f35de66bc","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"5b1afe34aa32d62c268203393a61c6062273332f6f805a711df68667d3fb5622"}
{"messages":[{"role":"user","content":"Find Tik Tok official website through Google, search for Justin Bieber's song \"Peaches\" video, and like it"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome icon in the taskbar at the bottom of the screen.\",\"code\":\"pyautogui.click(x=0.5005, y=0.975)\",\"image\":\"25d4c7e9-4256-4ff9-8f6f-a2d840597307.png\"},{\"action\":\"Click on the \\\"tik tok - Google 搜索\\\" suggestion from the search history dropdown list.\",\"code\":\"pyautogui.click(x=0.4401, y=0.5722)\",\"image\":\"6c7d48b7-b69a-4850-ac1b-554adcaace6d.png\"},{\"action\":\"Click on the \\\"Watch trending videos for you\\\" link in the search results to navigate directly to TikTok's video content section.\",\"code\":\"pyautogui.click(x=0.2271, y=0.6713)\",\"image\":\"8b32f942-507d-4fe1-bf7f-ff8427d282d8.png\"},{\"action\":\"Click on the microphone icon in the search bar to activate voice search functionality.\",\"code\":\"pyautogui.click(x=0.5036, y=0.2139)\",\"image\":\"d9179b5d-e62d-4684-9dd6-7c9ae4da4df5.png\"},{\"action\":\"Type \\\"justin bieber peaches\\\" in the search bar and press Enter to search for Justin Bieber's Peaches videos on TikTok.\",\"code\":\"pyautogui.write(message='justin bieber peaches')\\npyautogui.press('enter')\",\"image\":\"b60d5d69-0e70-45b2-9399-cda2d6d94897.png\"},{\"action\":\"Click on the \\\"Video\\\" tab in the search results categories to view all video results for Justin Bieber's \\\"Peaches\\\".\",\"code\":\"pyautogui.click(x=0.4932, y=0.2935)\",\"image\":\"29a454cc-66e4-4d01-956f-92fe40c404b5.png\"},{\"action\":\"Click on the first video titled \\\"Peaches - Justin Bieber\\\" by shyne_lyrics, which displays \\\"I got my peaches\\\" text overlay.\",\"code\":\"pyautogui.click(x=0.4359, y=0.5806)\",\"image\":\"955221fe-7294-4287-b500-01e7a3c318ac.png\"},{\"action\":\"Click on the heart/like button located on the right side of the screen that shows \\\"132\\\" likes to like the Justin Bieber \\\"Peaches\\\" video.\",\"code\":\"pyautogui.click(x=0.6193, y=0.4722)\",\"image\":\"3fd72fa8-16b4-4a33-a907-df661e265da2.png\"},{\"action\":\"Terminate the task with a success status since all required steps have been completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"9eb83ef1-d5c4-49cf-85de-2cb8f9548014.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241008194202_2244e19a-9c4f-4fb9-aca1-9d7235c045f0","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"614480510ff8182b49443ef9a4747eb12e87e7d25eaf740136b1868f72aed2e7"}
{"messages":[{"role":"user","content":"Open Blender, create a cube, adjust the cube's location, rotation, and scale"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Blender application icon (orange and blue 3D logo) in the macOS dock to launch the application.\",\"code\":\"pyautogui.click(x=0.7327, y=0.9641)\",\"image\":\"32ad60bb-bfcb-42d0-9c28-0c2a2fc4c849.png\"},{\"action\":\"Click on the default cube in the center of the 3D viewport to select or interact with it before making transformations.\",\"code\":\"pyautogui.click(x=0.4249, y=0.5067)\",\"image\":\"36548691-ee7c-4ac8-becb-c4fe86ab48ef.png\"},{\"action\":\"Click on the Location X value field in the Transform properties panel to edit the cube's X-position.\",\"code\":\"pyautogui.click(x=0.9393, y=0.4818)\",\"image\":\"13ce5bd9-284d-4c93-a385-86bdcb538664.png\"},{\"action\":\"Type \\\"2\\\" in the Location X field and press Enter to change the cube's X position to 2 units.\",\"code\":\"pyautogui.write(message='2')\\npyautogui.press('enter')\",\"image\":\"1df22685-5f98-432e-8064-8d4399beb832.png\"},{\"action\":\"Click on the Location Y value field in the Transform panel to edit the cube's Y-position.\",\"code\":\"pyautogui.click(x=0.9391, y=0.5024)\",\"image\":\"c2c684c6-8445-475f-a2c7-23d96f7b4298.png\"},{\"action\":\"Type \\\"3\\\" in the Location Y field and press Enter to change the cube's Y position to 3 units.\",\"code\":\"pyautogui.write(message='3')\\npyautogui.press('enter')\",\"image\":\"b10a5190-bce4-4121-866a-8ae06dd4cca6.png\"},{\"action\":\"Click on the Location Z value field in the Transform panel to prepare for editing the cube's Z-position.\",\"code\":\"pyautogui.click(x=0.9407, y=0.5198)\",\"image\":\"90c30429-d413-49b8-b3a4-24cab74e2948.png\"},{\"action\":\"Type \\\"3\\\" in the Location Z field and press Enter to set the cube's Z position to 3 units.\",\"code\":\"pyautogui.write(message='3')\\npyautogui.press('enter')\",\"image\":\"9579efba-0f67-497d-8288-df6d03920336.png\"},{\"action\":\"Click on the Rotation X value field in the Transform panel to prepare to edit the cube's X-axis rotation.\",\"code\":\"pyautogui.click(x=0.9434, y=0.5429)\",\"image\":\"da00d253-fd11-4e93-ae5b-d5a14592ced9.png\"},{\"action\":\"Type \\\"30\\\" in the Rotation X field and press Enter to set the cube's X-axis rotation to 30 degrees.\",\"code\":\"pyautogui.write(message='30')\\npyautogui.press('enter')\",\"image\":\"8c2fea70-3ad4-45ef-a2a8-147699a3b2a0.png\"},{\"action\":\"Click on the Rotation Y value field in the Transform panel to prepare to edit the cube's Y-axis rotation.\",\"code\":\"pyautogui.click(x=0.9444, y=0.557)\",\"image\":\"90c4dca8-8fda-4dec-84e2-630cfbfc60e0.png\"},{\"action\":\"Type \\\"40\\\" in the Rotation Y field and press Enter to set the cube's Y-axis rotation to 40 degrees.\",\"code\":\"pyautogui.write(message='40')\\npyautogui.press('enter')\",\"image\":\"8961c108-e462-4e6c-9011-5e1b5eb434b2.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240928013000_b50ec668-bd4b-46bd-b5cb-215754e46e1e","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"803590f779f86315067c6f74008cb979d1b82360d0fe590f1c31df9ccf3567fb"}
{"messages":[{"role":"user","content":"Enable “Geotagging” for location-based search results"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Scroll down in the description field area to reveal more settings or options that might contain location information.\",\"code\":\"pyautogui.moveTo(x=0.4255, y=0.6889)\\npyautogui.scroll(-10)\",\"image\":\"52ab2ca2-5510-401d-bff7-4e82bc80eb36.png\"},{\"action\":\"Click on the \\\"Show more\\\" button to reveal additional video settings including options for paid promotion, tags, subtitles, and potentially location information.\",\"code\":\"pyautogui.click(x=0.2344, y=0.8111)\",\"image\":\"258fe676-9833-4942-bc02-6a85b0419d3a.png\"},{\"action\":\"Scroll down in the expanded settings area to reveal more options below the \\\"Paid promotion\\\" section.\",\"code\":\"pyautogui.moveTo(x=0.4, y=0.7204)\\npyautogui.scroll(-16)\",\"image\":\"445f7963-6953-4198-9019-362913d7593b.png\"},{\"action\":\"Click on the \\\"Video location\\\" dropdown field which currently shows \\\"None\\\" to open the location selection interface.\",\"code\":\"pyautogui.click(x=0.5198, y=0.6361)\",\"image\":\"2fe5a7e9-59c7-43be-ac81-33a0c01e4503.png\"},{\"action\":\"Type \\\"lahore\\\" in the video location search field.\",\"code\":\"pyautogui.write(message='lahore')\",\"image\":\"c3eb6619-ad5f-4926-81db-586acff4c3f5.png\"},{\"action\":\"Click on the \\\"Lahore High Court Shahrah-e-Quaid-e-Azam, Mozang Chungi, Lahore\\\" option from the dropdown menu to select this location for the video.\",\"code\":\"pyautogui.click(x=0.6318, y=0.5185)\",\"image\":\"5d0bbae8-e6e7-4bc3-b9e1-03ce67f31613.png\"},{\"action\":\"Click the \\\"Save\\\" button in the top-right corner of the page to apply the geotagging changes to the video.\",\"code\":\"pyautogui.click(x=0.9354, y=0.2491)\",\"image\":\"3faa96ee-3573-471d-b969-1fcb81af7082.png\"},{\"action\":\"Terminate the process with success status as the task has been completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"d06371a8-4ea0-4f83-8800-e4ff20ff1ffb.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241111213211_b3fabdcc-f566-4379-8358-e987d830e162","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"eac3143d57bc570ac00e76c264b85cfdda2a5b5af321d077c16ee2e1c74d300c"}
{"messages":[{"role":"user","content":"Calculate the average goals that Heungmin Son have scored from last five seasons (until 2023/2024) in Premier League website using Mac Calculator"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the voice search icon in the Google search bar.\",\"code\":\"pyautogui.click(x=0.4129, y=0.4415)\",\"image\":\"c90d0588-7104-4a69-a83b-a3128fbdb84e.png\"},{\"action\":\"Type \\\"Heungmin Son\\\" in the search field and press Enter to search for information about the player.\",\"code\":\"pyautogui.write(message='Heungmin Son')\\npyautogui.press('enter')\",\"image\":\"ce576483-95f5-4db6-9a2f-fd03a2de9f27.png\"},{\"action\":\"Click on \\\"Follow Son Heung-Min\\\" link in the Premier League search result to access Son's official player profile.\",\"code\":\"pyautogui.click(x=0.3387, y=0.441)\",\"image\":\"7d6c5b86-552c-4539-91cc-7725d36c57a0.png\"},{\"action\":\"Move the cursor to the middle of Son Heung-Min's name area, scroll slightly to the right, then scroll down to view more of his career statistics table.\",\"code\":\"pyautogui.moveTo(x=0.3387, y=0.441)\\npyautogui.hscroll(4)\\npyautogui.scroll(-28)\",\"image\":\"96d1cf0a-fde5-48e8-a43c-491f1aa86e79.png\"},{\"action\":\"Click on the Calculator app icon in the Mac dock to open the Calculator application.\",\"code\":\"pyautogui.click(x=0.8207, y=0.9624)\",\"image\":\"314b4b2d-6927-4b84-8ee6-e1663509cb60.png\"},{\"action\":\"Click on the number \\\"1\\\" button on the calculator keypad to begin entering the first value for the calculation.\",\"code\":\"pyautogui.click(x=0.2325, y=0.3954)\",\"image\":\"f55752df-9e5f-47c4-acc6-3adb6e4ea67d.png\"},{\"action\":\"Click on the number \\\"7\\\" button on the calculator keypad to complete entering the first value \\\"17\\\".\",\"code\":\"pyautogui.click(x=0.2285, y=0.3111)\",\"image\":\"8f807fd0-d42d-4e28-95ab-23c5cfb2f051.png\"},{\"action\":\"Click on the equals (=) button on the calculator keypad.\",\"code\":\"pyautogui.click(x=0.3397, y=0.4221)\",\"image\":\"0287ab82-3c02-49af-9ecb-65958dcf0e9b.png\"},{\"action\":\"Click on the number \\\"1\\\" button on the calculator keypad to begin entering the second value for the calculation.\",\"code\":\"pyautogui.click(x=0.2352, y=0.4012)\",\"image\":\"893b1132-7fb1-4742-8b77-a1426165bd53.png\"},{\"action\":\"Click on the number \\\"0\\\" button on the calculator keypad to complete entering the second value \\\"10\\\" for the calculation.\",\"code\":\"pyautogui.click(x=0.2339, y=0.4572)\",\"image\":\"141bbcbd-e898-4d2c-b848-c2eba882a778.png\"},{\"action\":\"Click the addition (+) button on the calculator to add the first two goal values together.\",\"code\":\"pyautogui.click(x=0.3442, y=0.3943)\",\"image\":\"92342b06-6916-4c47-a0d0-aa272e6332a8.png\"},{\"action\":\"Click on the number \\\"3\\\" button on the calculator keypad to complete entering the value \\\"23\\\" for the 2021/22 season's goals.\",\"code\":\"pyautogui.click(x=0.3028, y=0.4037)\",\"image\":\"f503226f-3a34-4fbd-8da3-60873abba003.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240926005328_388a62d8-58f6-460f-8d62-ed1d079e487a","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"83f26b96c57921e9d5a89e68256f6def6ab9d8afc006b80a8f01538e69f3f887"}
{"messages":[{"role":"user","content":"Search for flights for 3 adults from Beijing to San Francisco, departing on 14 November 2024, returning on 19 November 2024, filter results by same depart/return airport only, and view the China Southern deal."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Flights\\\" button in the main navigation menu to access the flight search functionality.\",\"code\":\"pyautogui.click(x=0.1742, y=0.2544)\",\"image\":\"4f5bdbea-84c9-4a57-ae7d-c208b63b6860.png\"},{\"action\":\"Click on the \\\"x\\\" button next to \\\"Taipei City (TPE)\\\" to clear the current departure city input.\",\"code\":\"pyautogui.click(x=0.1648, y=0.63)\",\"image\":\"49ce5b10-f886-4b53-9fdb-31b1e422a3dd.png\"},{\"action\":\"Click on the \\\"From?\\\" input field to focus it and prepare to enter the departure city.\",\"code\":\"pyautogui.click(x=0.157, y=0.63)\",\"image\":\"480d14d0-1cc0-406b-a497-137c933a277c.png\"},{\"action\":\"Type \\\"beijing\\\" in the currently focused \\\"From?\\\" input field to specify the departure city for the flight search.\",\"code\":\"pyautogui.write(message='beijing')\",\"image\":\"ddcfc036-06c9-40d8-af1a-69772c4f128c.png\"},{\"action\":\"Click on \\\"Beijing, China (BJS)\\\" in the dropdown menu to select all Beijing airports as the departure location.\",\"code\":\"pyautogui.click(x=0.3984, y=0.62)\",\"image\":\"88108772-3706-4f41-8ab1-4baa9568fa5c.png\"},{\"action\":\"Click on the \\\"To?\\\" input field to focus it and prepare to enter the destination city.\",\"code\":\"pyautogui.click(x=0.3816, y=0.5188)\",\"image\":\"92ec98d6-cd6b-46b7-870d-a2102049d01c.png\"},{\"action\":\"Type \\\"san f\\\" in the destination field to begin entering San Francisco as the destination city.\",\"code\":\"pyautogui.write(message='san f')\",\"image\":\"e9d25b8f-1728-4f41-930d-f00238a0c9f6.png\"},{\"action\":\"Click on \\\"San Francisco, California, United States\\\" in the dropdown menu to select it as the destination city.\",\"code\":\"pyautogui.click(x=0.4242, y=0.6156)\",\"image\":\"8228d64a-5b34-4409-a72f-b91e247f5284.png\"},{\"action\":\"Click on the departure date field showing \\\"Tue 3/11\\\" to open the date picker calendar, which will allow me to select November 14, 2024 as the departure date.\",\"code\":\"pyautogui.click(x=0.5441, y=0.5181)\",\"image\":\"c44d254e-8ff7-406b-a141-c36e4f9478f2.png\"},{\"action\":\"Click on the left-facing arrow button in the calendar interface to navigate to the previous month, which should display November 2024.\",\"code\":\"pyautogui.click(x=0.4512, y=0.365)\",\"image\":\"e079d27c-7009-4d7a-9790-992c940ec9fb.png\"},{\"action\":\"Click on November 14, 2024 in the calendar to select it as the departure date.\",\"code\":\"pyautogui.click(x=0.5625, y=0.6212)\",\"image\":\"d06660b2-8308-4b7e-8122-ee1f56a142b0.png\"},{\"action\":\"Click on date \\\"19\\\" in the November 2024 calendar to select it as the return date for the flight.\",\"code\":\"pyautogui.click(x=0.4922, y=0.6781)\",\"image\":\"578102d4-2a56-43e5-bf36-1bb0ebea5998.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241104155927_07dfabfe-eada-4bdc-9b53-d6e81a680193","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8357d97166316ac60766a4c408e8b7e9fa1e4f9de3d19a5b3ea9412f08133bcf"}
{"messages":[{"role":"user","content":"create a repository on github website and upload the local project in the repository"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the address/search bar to prepare for entering the GitHub URL.\",\"code\":\"pyautogui.click(x=0.2755, y=0.0694)\",\"image\":\"960a1e3c-364d-477e-8ea7-755c3d02e278.png\"},{\"action\":\"Type \\\"gi\\\" in the address bar to begin entering GitHub's URL.\",\"code\":\"pyautogui.write(message='gi')\",\"image\":\"cee4c27f-4522-42b0-807c-32322278f345.png\"},{\"action\":\"Click on the word \\\"build\\\" within the GitHub search result title \\\"GitHub: Let's build from here · GitHub\\\".\",\"code\":\"pyautogui.click(x=0.226, y=0.7944)\",\"image\":\"c7a3deb2-2977-4f58-ac71-ad930e5919e5.png\"},{\"action\":\"Click on the \\\"Your repositories\\\" option in the user dropdown menu.\",\"code\":\"pyautogui.click(x=0.8599, y=0.2694)\",\"image\":\"76b3814c-c1e8-4b3b-aa1c-2a5ac91f8283.png\"},{\"action\":\"Click on the \\\"+\\\" icon in the top navigation bar to access options for creating new items.\",\"code\":\"pyautogui.click(x=0.8495, y=0.1204)\",\"image\":\"f105c49d-0024-47f9-abeb-f4ec8887f072.png\"},{\"action\":\"Click on \\\"New repository\\\" in the dropdown menu to create a new GitHub repository.\",\"code\":\"pyautogui.click(x=0.8646, y=0.1787)\",\"image\":\"48610ec3-4a92-45c3-95aa-0780a83193ed.png\"},{\"action\":\"Click on the Repository name text field to start entering a name for the new repository.\",\"code\":\"pyautogui.click(x=0.3865, y=0.4028)\",\"image\":\"d04509e1-6f77-4156-bc0d-1452d481ea04.png\"},{\"action\":\"Type \\\"task1\\\" in the Repository name field to name the new GitHub repository.\",\"code\":\"pyautogui.write(message='task1')\",\"image\":\"72b19c63-d4b6-4513-a3eb-2252d2406a1b.png\"},{\"action\":\"Click on the \\\"Private\\\" radio button to change the repository visibility setting from public to private.\",\"code\":\"pyautogui.click(x=0.2708, y=0.688)\",\"image\":\"57e4976d-01ba-4a84-a9a6-bf8d1a01bce2.png\"},{\"action\":\"Click the checkbox for \\\"Add a README file\\\" under the repository initialization options to include a README.md in the new repository.\",\"code\":\"pyautogui.click(x=0.2661, y=0.5806)\",\"image\":\"9cbc3c43-b48d-451d-b9f9-6683ad3a79b0.png\"},{\"action\":\"Click the \\\"Create repository\\\" button to finalize the creation of the new GitHub repository.\",\"code\":\"pyautogui.click(x=0.6589, y=0.7194)\",\"image\":\"73634163-3ab9-480a-b2db-915348c0fb5e.png\"},{\"action\":\"Click on the \\\"Add file\\\" button to open the file addition options menu.\",\"code\":\"pyautogui.click(x=0.5823, y=0.3269)\",\"image\":\"ac7104f2-de94-4e4d-97fe-2d18107120ac.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241008212859_aaf41f7a-8b4a-4eb2-80a9-26e4586aba39","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c61488b1e079cc7f8d9f54ff74c79a37e8d48eab07feb877127caa949d794578"}
{"messages":[{"role":"user","content":"go to bilibili, go to view my personal website and view the data of my uploaded video"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome icon in the dock to focus on or launch the browser.\",\"code\":\"pyautogui.click(x=0.580, y=0.962)\",\"image\":\"928383f1-6d9c-4454-b884-593968ab15fb.png\"},{\"action\":\"Click on the plus (+) button in the Chrome tab bar to create a new tab.\",\"code\":\"pyautogui.click(x=0.751, y=0.096)\",\"image\":\"04dbb662-02af-46d0-a975-61951b59727e.png\"},{\"action\":\"Type \\\"bilibili\\\" in the address bar and press Enter twice to navigate to the Bilibili website.\",\"code\":\"pyautogui.write(message='bilibili')\\npyautogui.press('enter', presses=2)\",\"image\":\"24c9ef0f-6098-4673-b7b9-15cfaebe102f.png\"},{\"action\":\"Click on the \\\"bilibili\\\" text link in the first search result titled \\\"哔哩哔哩(゜-゜)つロ干杯～bilibili\\\"\",\"code\":\"pyautogui.click(x=0.225, y=0.475)\",\"image\":\"dade3be7-35bf-4c30-a5f1-e3bd6159b323.png\"},{\"action\":\"Click on the profile/avatar icon in the top-right area of the Bilibili interface to access the account section.\",\"code\":\"pyautogui.click(x=0.580, y=0.242)\",\"image\":\"0dd40cc6-86b1-4c12-bacf-de2b04ddb693.png\"},{\"action\":\"Click on the \\\"投稿\\\" (Uploads) tab to view uploaded video content and statistics.\",\"code\":\"pyautogui.click(x=0.153, y=0.549)\",\"image\":\"b5c919d1-7bc3-4870-8207-4bc4a9c238a2.png\"},{\"action\":\"Scroll down in the uploads section to view the uploaded video content and its statistics. This should reveal the video data that isn't currently visible in the initial view of the page.\",\"code\":\"pyautogui.moveTo(x=0.153, y=0.549)\\npyautogui.scroll(-10)\",\"image\":\"21e956bc-6f9b-4c33-8763-0e9253da9387.png\"},{\"action\":\"Click on the video thumbnail showing a bird with the title \\\"可怜的小鸟 好像受伤了\\\" (Poor little bird seems injured) to view the detailed video page and its statistics.\",\"code\":\"pyautogui.click(x=0.216, y=0.655)\",\"image\":\"f451a025-58af-46a8-8366-933590724fff.png\"},{\"action\":\"Click on the share button (with arrow icon) below the video player, located in the row of interaction buttons next to the engagement statistics.\",\"code\":\"pyautogui.click(x=0.421, y=0.764)\",\"image\":\"d4ca2892-8e3e-40dd-a056-6a28f1abbb22.png\"},{\"action\":\"Scroll down in the video analytics area to view more detailed statistics in the \\\"核心大屏数据\\\" (Core Screen Data) section that appears below the currently visible portion of the page.\",\"code\":\"pyautogui.moveTo(x=0.467, y=0.597)\\npyautogui.scroll(-10)\",\"image\":\"103e01a9-51b7-44ba-bf1b-bb5b550d0351.png\"},{\"action\":\"Click on the \\\"数据中心\\\" (Data Center) option in the left navigation menu to access the comprehensive analytics dashboard.\",\"code\":\"pyautogui.click(x=0.092, y=0.623)\",\"image\":\"15276403-ffc9-4fb8-8efa-898aaf8fb1f2.png\"},{\"action\":\"Scroll down in the Data Center dashboard to view additional analytics information below the currently visible metrics.\",\"code\":\"pyautogui.moveTo(x=0.402, y=0.526)\\npyautogui.scroll(10)\",\"image\":\"2d58bbec-fd60-4ee6-80de-24cfff4bf5e6.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240920225005_e25883db-29ba-497c-abbb-9f1e824cab15","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"0449d9cc08a7a85d17908b26540909e8c33c19409c7b6eddd9ca65cfbee671c6"}
{"messages":[{"role":"user","content":"View the Page Analysis in last 30 days of \"The University of Hong Kong\" in Microsoft SharePoint"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the SharePoint application tile to open SharePoint, which will allow access to the University of Hong Kong site and its page analysis features.\",\"code\":\"pyautogui.click(x=0.4131, y=0.6057)\",\"image\":\"b34a4634-8da2-4200-b66f-d7bb0ea866a1.png\"},{\"action\":\"Click on the SharePoint tile to open the SharePoint application.\",\"code\":\"pyautogui.click(x=0.3852, y=0.6029)\",\"image\":\"f285da09-3b60-4dda-bafe-79852752baa7.png\"},{\"action\":\"Click on \\\"The University of Hong Kong\\\" tile in the Frequent sites section to access the university's SharePoint site.\",\"code\":\"pyautogui.click(x=0.3378, y=0.7403)\",\"image\":\"ed0127df-aa67-4405-bd65-9ddff3dd06c1.png\"},{\"action\":\"Click on the \\\"Analytics\\\" button in the top navigation menu to access the page analysis data for The University of Hong Kong SharePoint site.\",\"code\":\"pyautogui.click(x=0.3782, y=0.2928)\",\"image\":\"cf4dedc1-194c-4b96-a928-ca18183fe17d.png\"},{\"action\":\"Click on the \\\"30\\\" days option in the time period selector of the Page Analytics panel to change the view from 7 days to 30 days.\",\"code\":\"pyautogui.click(x=0.7299, y=0.2823)\",\"image\":\"6e095fc5-bcf8-4f3f-a9e5-af397eeb9a35.png\"},{\"action\":\"Scroll down in the Page Analytics panel to view more of the analytics data that extends beyond the currently visible area.\",\"code\":\"pyautogui.moveTo(x=0.7733, y=0.6247)\\npyautogui.scroll(-14)\",\"image\":\"76edfc6c-5c42-4b9a-bc49-7c956cd8a7ea.png\"},{\"action\":\"Terminate the task with success status as we have successfully viewed the Page Analysis for the last 30 days of \\\"The University of Hong Kong\\\" in Microsoft SharePoint.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"e7abce8e-ca25-42b0-9588-b734dae83542.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241006223753_fbace8a0-941e-4029-a7b1-1372ac95ebd1","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"7ed99bc9c7784e0f04e59bb7a4246b505871d40dac65946eaf11b98ebcfc6ae9"}
{"messages":[{"role":"user","content":"Click Draw on the MS Bing website, ZC's Notebook page to select colors and strokes."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the more options button (three dots) in the top navigation bar, which is located after the \\\"News\\\" option.\",\"code\":\"pyautogui.click(x=0.478, y=0.091)\",\"image\":\"7819c9e3-a48c-4464-b8f9-d25a992d0fe5.png\"},{\"action\":\"Click on \\\"ZC's Notebook\\\" in the notebooks list to open it.\",\"code\":\"pyautogui.click(x=0.225, y=0.300)\",\"image\":\"0b5d6a36-8b42-4137-9414-17c18d6776ec.png\"},{\"action\":\"Click on the \\\"Draw\\\" tab in the top navigation ribbon, located between the \\\"Insert\\\" and \\\"View\\\" tabs.\",\"code\":\"pyautogui.click(x=0.104, y=0.132)\",\"image\":\"943663a5-8847-49e0-83e1-662da0c995be.png\"},{\"action\":\"Click on the blue pen tool in the drawing toolbar to select it as the active drawing instrument.\",\"code\":\"pyautogui.click(x=0.172, y=0.163)\",\"image\":\"a4e3abcb-6631-46a3-ac76-56e08971deb8.png\"},{\"action\":\"Click on the red pen tool in the drawing toolbar to select it as the active drawing instrument.\",\"code\":\"pyautogui.click(x=0.121, y=0.164)\",\"image\":\"0c32c6e5-af35-4fc4-9915-ea4e0d8d3ddd.png\"},{\"action\":\"Draw a line by dragging from the left side of the notebook page toward the right side using the currently selected red pen tool.\",\"code\":\"pyautogui.moveTo(x=0.286, y=0.418)\\npyautogui.dragTo(x=0.405, y=0.531, button='left')\",\"image\":\"6fee64c3-3776-451d-9b4a-74bd76512475.png\"},{\"action\":\"Click on the black calligraphy pen tool in the drawing toolbar, located to the right of the colored pen options.\",\"code\":\"pyautogui.click(x=0.237, y=0.165)\",\"image\":\"a32e1d6f-6f25-42f5-acdf-be3b9230a916.png\"},{\"action\":\"Click on the burgundy/dark red color square in the color palette to change the pen color.\",\"code\":\"pyautogui.click(x=0.228, y=0.199)\",\"image\":\"5654183d-8d4e-45a0-a89d-4abab4f44629.png\"},{\"action\":\"Draw a line with the burgundy/dark red calligraphy pen by dragging from a point near the middle of the page downward and slightly to the left.\",\"code\":\"pyautogui.moveTo(x=0.342, y=0.329)\\npyautogui.dragTo(x=0.283, y=0.420, button='left')\",\"image\":\"ebf774f7-d853-4f92-8209-f67c102b10d3.png\"},{\"action\":\"Terminate the task as completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"4a063f6f-3d92-4313-9f3d-72001d370d28.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241115193242_527fb14f-6037-400e-bf6d-2520a2e9265c","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"174e05a0ecf7a06a9380a1c4940a15640486cc58def9f7110ea9f0df69c29400"}
{"messages":[{"role":"user","content":"In Tableau 10.5, open the instance data ‘示例超市’. Rename the ‘订单’ column to ‘production’ and hide the ‘类别’ column，show the hidden objects. Search for ‘销售额’ and move it to the ‘Rows’，Move ‘地区’ to the ‘Columns’, change the data type of the ‘子类别’ column to numeric."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Tableau icon in the Windows taskbar to launch the Tableau application.\",\"code\":\"pyautogui.click(x=0.5458, y=0.9769)\",\"image\":\"0dc64d4f-c6b6-4b66-aacd-3c5c82005347.png\"},{\"action\":\"Click on the \\\"More...\\\" option under the \\\"To a File\\\" section to explore additional file connection options.\",\"code\":\"pyautogui.click(x=0.0625, y=0.6148)\",\"image\":\"63d3bd2f-a7b2-4599-842b-d769fbc30ae9.png\"},{\"action\":\"Click on the \\\"示例超市\\\" (Sample Supermarket) file in the file list to select it.\",\"code\":\"pyautogui.click(x=0.1812, y=0.25)\",\"image\":\"1e69e6f6-6468-4653-a07c-5538b9a61dc6.png\"},{\"action\":\"Click the \\\"打开(O)\\\" (Open) button in the file dialog to open the selected \\\"示例超市\\\" (Sample Supermarket) file.\",\"code\":\"pyautogui.click(x=0.3505, y=0.6463)\",\"image\":\"45d2d39a-e8bf-4451-abba-4ea53c6319c0.png\"},{\"action\":\"Click the \\\"OK\\\" button in the Import Data Source dialog to confirm the selection of the \\\"示例 - 超市\\\" data source.\",\"code\":\"pyautogui.click(x=0.4766, y=0.5935)\",\"image\":\"0f37b5be-5588-40b7-b82f-f0b935d689ab.png\"},{\"action\":\"Click on the \\\"订单\\\" (Orders) item in the Tables section of the data pane to select it.\",\"code\":\"pyautogui.click(x=0.0078, y=0.2898)\",\"image\":\"fdb59bc2-fffe-4037-9a3e-61a41cb970cf.png\"},{\"action\":\"Right-click on the \\\"产品\\\" field in the data pane to open its context menu.\",\"code\":\"pyautogui.click(x=0.0474, y=0.3204)\",\"image\":\"4d46f6b7-5654-457e-ab36-31d39376eb3a.png\"},{\"action\":\"Click on the \\\"订单\\\" (Orders) table name in the data pane to select the entire table.\",\"code\":\"pyautogui.click(x=0.1391, y=0.3167)\",\"image\":\"ac2ce689-74a2-4fd6-8b3e-4701c918ef47.png\"},{\"action\":\"Delete the current table name by pressing the backspace key twice, type \\\"production\\\" as the new name, and press enter to confirm the change.\",\"code\":\"pyautogui.press('backspace', presses=2)\\npyautogui.write(message='production')\\npyautogui.press('enter')\",\"image\":\"7f451018-57cb-4c7b-9442-70236c402712.png\"},{\"action\":\"Click on the \\\"类别\\\" field in the data pane to select it, preparing to hide this column as required by the task.\",\"code\":\"pyautogui.click(x=0.4432, y=0.4861)\",\"image\":\"e3e6881c-e5fb-4554-a3b1-e003fec466b3.png\"},{\"action\":\"Click on the dropdown arrow next to the \\\"production\\\" table name to access table operations menu.\",\"code\":\"pyautogui.click(x=0.1406, y=0.3157)\",\"image\":\"c5cccf90-0709-4bf2-9eee-e03b79a90bb0.png\"},{\"action\":\"Click on the \\\"类别\\\" field in the production table to select it.\",\"code\":\"pyautogui.click(x=0.0969, y=0.3528)\",\"image\":\"3e094b42-119c-42cd-8de1-e6a7054befd7.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241202150210_3f90ab6a-6f2b-48b3-9df9-6475bb41cd21","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"95f6840b86b3e34d8897d7be721f51a96688fe7428ec0e9ad5473fbbe618ab50"}
{"messages":[{"role":"user","content":"Find new releases by browsing the “New & Popular” section"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Netflix browser preview shown in the Random Task section to make it the active window.\",\"code\":\"pyautogui.click(x=0.573, y=0.903)\",\"image\":\"1600efb9-a72c-4eaf-a11e-196ba3d201c8.png\"},{\"action\":\"Click on the \\\"New & Popular\\\" navigation option in the Netflix menu bar to access that section of content.\",\"code\":\"pyautogui.click(x=0.357, y=0.218)\",\"image\":\"503f0e01-f17c-45b0-a8ab-17e056ccff0d.png\"},{\"action\":\"Click the X (close) button on the battery saver notification pop-up to dismiss it and continue browsing Netflix content unobstructed.\",\"code\":\"pyautogui.click(x=0.976, y=0.839)\",\"image\":\"7473da91-ee2d-41bf-b15d-7e2db7fbc9af.png\"},{\"action\":\"Move the cursor to the middle-right area of the screen where \\\"THE LINCOLN LAWYER\\\" preview is displayed and scroll down to see more content in the \\\"New & Popular\\\" section.\",\"code\":\"pyautogui.moveTo(x=0.671, y=0.483)\\npyautogui.scroll(-10)\",\"image\":\"0c5d46b1-d8ec-40c6-a971-44b1a2412dcd.png\"},{\"action\":\"Complete the task as we have successfully found new releases in the \\\"New & Popular\\\" section.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"67b371c2-274a-478b-b1e8-ccfd1b485628.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241103171756_fdeb1b5b-1387-42f7-b2c1-fde698c4e672","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"11164631fab3617bc0a1f50ec211f620fe3fe393840effbb96bb61697fb32986"}
{"messages":[{"role":"user","content":"Open the Zoom Workplace app on computer, select a meeting date and time in the synced calendar, select each participant's name or email address, click Send to add the Zoom meeting to the calendar and send an invite to attendees."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Windows Start button in the bottom left corner of the taskbar to open the Start menu.\",\"code\":\"pyautogui.click(x=0.257, y=0.983)\",\"image\":\"5b7d0430-21f0-4eed-8007-5d1aa2890eba.png\"},{\"action\":\"Type \\\"zoom\\\" in the search field at the top of the Start menu to find the Zoom Workplace application.\",\"code\":\"pyautogui.write(message='zoom')\",\"image\":\"7666e10b-510d-4c66-8e75-cd06d0b3a19e.png\"},{\"action\":\"Click on the Zoom Workplace app listed under \\\"Best match\\\" to launch the application.\",\"code\":\"pyautogui.click(x=0.306, y=0.243)\",\"image\":\"e0b4c5cd-89f7-4317-b487-f47d66f3c1fa.png\"},{\"action\":\"Click on the empty time slot at 12 PM on Wednesday, October 16 in the calendar to begin creating a new meeting.\",\"code\":\"pyautogui.click(x=0.496, y=0.619)\",\"image\":\"c8ac9673-d6ea-45ea-8395-a23a4a890899.png\"},{\"action\":\"Click on the \\\"Name or email address\\\" input field under the Attendees section to begin adding meeting participants.\",\"code\":\"pyautogui.click(x=0.314, y=0.478)\",\"image\":\"409cf05e-4a9c-467e-8a36-9e1b0ce24b8e.png\"},{\"action\":\"Type \\\"j\\\" in the Name or email address field to begin adding a participant whose name or email starts with that letter.\",\"code\":\"pyautogui.write(message='j')\",\"image\":\"69112e5d-a452-4427-9f5f-d7ba0c653acb.png\"},{\"action\":\"Click on the contact suggestion \\\"Janie\\\" to add her as an attendee to the meeting.\",\"code\":\"pyautogui.click(x=0.287, y=0.527)\",\"image\":\"aaff49d6-432f-48fa-90c2-c4c2a0d077b2.png\"},{\"action\":\"Click the \\\"Send\\\" button to create the Zoom meeting and send the invitation to attendees.\",\"code\":\"pyautogui.click(x=0.529, y=0.871)\",\"image\":\"468e0b18-7943-4f5f-a9fc-24b4c6cd09af.png\"},{\"action\":\"Terminate the task with a success status since all goals have been accomplished: opening Zoom Workplace, selecting a meeting date and time, adding a participant, and sending the meeting invitation.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"93c4f482-ac58-43a9-8a74-32bcbb89e941.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241017070146_957286fa-8f7a-4e77-91cb-16a6108d0163","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"2a77b5ebe9459ed40312afcbdf375a3f7984bd66e80d57600fbb711c2a5b9145"}
{"messages":[{"role":"user","content":"Set Timer on Whiteboard"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Double-click on the minutes field (showing \\\"05\\\") in the timer display to select it for editing.\",\"code\":\"pyautogui.doubleClick(x=0.0406, y=0.275)\",\"image\":\"ca8b9c0d-888c-4c2d-bd26-587729da0111.png\"},{\"action\":\"Click on the minutes field (\\\"05\\\") in the timer widget to confirm selection for editing or to begin entering a new value.\",\"code\":\"pyautogui.click(x=0.0406, y=0.275)\",\"image\":\"2e887479-c68d-4460-ad7d-72c46c7ba8cc.png\"},{\"action\":\"Type \\\"18\\\" to set the timer for 18 minutes.\",\"code\":\"pyautogui.write(message='18')\",\"image\":\"a60c8d00-81ed-4f82-b2c2-818c57b9598c.png\"},{\"action\":\"Click the blue play button on the timer widget to start the 18-minute countdown.\",\"code\":\"pyautogui.click(x=0.1083, y=0.2657)\",\"image\":\"71b5d068-8afd-4bb6-b261-5c6d2c2e75b4.png\"},{\"action\":\"Terminate the task with a success status as the goal of setting a timer on the whiteboard has been successfully completed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"b34ec02b-0fd7-4aff-80a3-6763005e950e.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241104171355_d2e635ff-23e4-4858-9fcb-ed94b0cc1d5f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"625c7fdd266e3e1627689360813e76124f6b6bc8f6b2859bf65f2d6bcfa14d91"}
{"messages":[{"role":"user","content":"In Photoshop, using the Dust and Scratches filter make the borders of the clouds a bit more chaotic."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Filter\\\" menu in the top menu bar to access the filter options.\",\"code\":\"pyautogui.click(x=0.159, y=0.006)\",\"image\":\"3be7887b-7f6f-4cc1-85a9-f156d4800a2e.png\"},{\"action\":\"Click on \\\"Layer 1\\\" in the Layers panel to make it the active layer before applying filters.\",\"code\":\"pyautogui.click(x=0.932, y=0.783)\",\"image\":\"807320b6-a46c-4de8-9399-8fab9f25e034.png\"},{\"action\":\"Click on \\\"Dust & Scratches...\\\" in the Noise submenu to open the filter's dialog box.\",\"code\":\"pyautogui.click(x=0.299, y=0.317)\",\"image\":\"b674f7dd-6c4b-4306-8a40-f97c1a6f8c0a.png\"},{\"action\":\"Drag across the Radius value field from its current position to slightly increase the radius parameter for the Dust & Scratches filter.\",\"code\":\"pyautogui.moveTo(x=0.346, y=0.895)\\npyautogui.dragTo(x=0.355, y=0.895, button='left')\",\"image\":\"7a77f813-dade-4371-b6a2-cc677abdd7b8.png\"},{\"action\":\"Make a small horizontal drag on the Radius value field to slightly increase the value, moving from the current position a tiny bit to the right.\",\"code\":\"pyautogui.moveTo(x=0.355, y=0.895)\\npyautogui.dragTo(x=0.359, y=0.895, button='left')\",\"image\":\"0bb09f8e-ea9f-486d-8365-fdedb07ca294.png\"},{\"action\":\"Drag the Threshold slider from its current position at 0 slightly to the right to increase its value, which will help preserve some cloud detail while still creating the chaotic border effect.\",\"code\":\"pyautogui.moveTo(x=0.345, y=0.944)\\npyautogui.dragTo(x=0.378, y=0.942, button='left')\",\"image\":\"255c67d9-0915-492b-af0b-2af40cd03834.png\"},{\"action\":\"Drag the Threshold slider slightly to the right to increase its value from 6 to a slightly higher value to preserve more cloud detail while maintaining the chaotic border effect.\",\"code\":\"pyautogui.moveTo(x=0.378, y=0.942)\\npyautogui.dragTo(x=0.403, y=0.937, button='left')\",\"image\":\"d404ef2c-0fda-4fa7-8535-58e50327910e.png\"},{\"action\":\"Drag the Radius slider slightly to the right to make a small increase in the Radius value, which will refine the chaotic border effect on the clouds.\",\"code\":\"pyautogui.moveTo(x=0.366, y=0.897)\\npyautogui.dragTo(x=0.373, y=0.898, button='left')\",\"image\":\"4244b71a-b72f-40cc-b2f1-93b7385ad1de.png\"},{\"action\":\"Drag the Radius slider slightly to the left to decrease its value from 25 pixels to a slightly lower setting, which will help maintain detail while keeping the chaotic effect on the cloud borders.\",\"code\":\"pyautogui.moveTo(x=0.373, y=0.898)\\npyautogui.dragTo(x=0.369, y=0.898, button='left')\",\"image\":\"42566bdc-894d-4ede-b1b4-c1ed6aca9473.png\"},{\"action\":\"Make a small drag motion on the Radius slider to slightly decrease its value, moving it a tiny bit to the left.\",\"code\":\"pyautogui.moveTo(x=0.369, y=0.898)\\npyautogui.dragTo(x=0.366, y=0.899, button='left')\",\"image\":\"df152d43-8d6d-41cd-95ac-ee9b1e932941.png\"},{\"action\":\"Double-click on the \\\"Preview\\\" checkbox to toggle the preview on and off, allowing comparison between the original and filtered versions of the image.\",\"code\":\"pyautogui.doubleClick(x=0.459, y=0.713)\",\"image\":\"c1b0aa1c-6f88-4266-9bae-7c1b394a167b.png\"},{\"action\":\"Click the \\\"OK\\\" button in the Dust & Scratches filter dialog box to apply the filter with the current settings (Radius: 27, Threshold: 107).\",\"code\":\"pyautogui.doubleClick(x=0.459, y=0.713)\",\"image\":\"7d64648d-c0a1-4084-b737-5c5018cb3bcc.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241006235732_5e4cad14-0bb9-4d39-9471-9ef9eb70a160","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"447e7e11086b8319015b57b78169e72b01899ca510ab119112f202a10ad6b70e"}
{"messages":[{"role":"user","content":"search recommendation of monitor to gaming size 50 inch or bigger."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Help choose my monitor\\\" link in the Displays dropdown menu.\",\"code\":\"pyautogui.click(x=0.481, y=0.3307)\",\"image\":\"382a74fd-c7c2-4785-b819-edf36a5c9c53.png\"},{\"action\":\"Click on the \\\"Gaming\\\" button to specify that we're looking for a gaming monitor.\",\"code\":\"pyautogui.click(x=0.4634, y=0.3411)\",\"image\":\"b5cf1574-96a8-456f-83ae-e4a71316f74c.png\"},{\"action\":\"Click the \\\"Selanjutnya\\\" (Next) button at the bottom of the screen to continue to the next step in the monitor selection process.\",\"code\":\"pyautogui.click(x=0.4993, y=0.9167)\",\"image\":\"1f70a0b8-9ddf-43d3-be5c-34ea2605d1fc.png\"},{\"action\":\"Click on the \\\"200Hz or above\\\" option to select the highest refresh rate for the gaming monitor.\",\"code\":\"pyautogui.click(x=0.5776, y=0.6484)\",\"image\":\"3f7a9752-f172-40ca-a593-847e6a2192c7.png\"},{\"action\":\"Click the \\\"Temukan monitor Anda\\\" (Find your monitor) button at the bottom of the screen to see the recommended gaming monitors that match our criteria.\",\"code\":\"pyautogui.click(x=0.4868, y=0.8906)\",\"image\":\"aa57c405-bc5e-4936-a474-3f1efdc9954a.png\"},{\"action\":\"Click on the \\\"Lebih detail\\\" (More details) button to view additional information about the 57\\\" Odyssey Neo G9 G95NC monitor.\",\"code\":\"pyautogui.click(x=0.6391, y=0.7122)\",\"image\":\"5e8168e9-ef19-4ac4-9290-a93144224aa1.png\"},{\"action\":\"Press the down arrow key 109 times to scroll down the product page and view more detailed information about the 57\\\" Odyssey Neo G9 G95NC monitor.\",\"code\":\"pyautogui.press('down', presses=109)\",\"image\":\"b28c8304-4f7b-4781-90cc-79fe81fd4545.png\"},{\"action\":\"Terminate the task as it has been successfully completed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"7376e8a3-7dc7-46a6-a423-f2b60190340d.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241102235202_d596cf52-10b6-4492-9195-c3b1c6db9ae4","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"0d4984c8305fe6d52f3afaa8177985e270b934a690f8f50461c565d8a35d6a56"}
{"messages":[{"role":"user","content":"Request to access the calendar of yana.p5630@gmail.com to my calendar in Google Calendar, accept the request from yana.p5630@gmail.com's Gmail, share Yana's calendar with Yup C, show and hide yana.p5630@gmail.com's calendar in my calendar."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click the \\\"+\\\" button next to \\\"Other calendars\\\" in the left sidebar to begin adding a new calendar.\",\"code\":\"pyautogui.click(x=0.1469, y=0.7919)\",\"image\":\"e104b85b-30b9-47ef-afaf-4714685e9bd7.png\"},{\"action\":\"Click on the \\\"Subscribe to calendar\\\" option in the dropdown menu.\",\"code\":\"pyautogui.click(x=0.2035, y=0.5669)\",\"image\":\"8ba6f24e-3355-45be-8c0d-f446c4e3ef16.png\"},{\"action\":\"Click on the \\\"yana.p5630@gmail.com\\\" entry in the dropdown menu to select this calendar for subscription.\",\"code\":\"pyautogui.click(x=0.4375, y=0.3644)\",\"image\":\"85c1685b-d99a-4a3a-bac5-1d238f26074e.png\"},{\"action\":\"Click the \\\"Request access\\\" button to send a calendar access request to yana.p5630@gmail.com.\",\"code\":\"pyautogui.click(x=0.6469, y=0.6288)\",\"image\":\"9d39af5a-c3c1-4c01-a1a8-9c88e9ab0b53.png\"},{\"action\":\"Click the Chrome icon on the taskbar to open or switch to the Chrome browser.\",\"code\":\"pyautogui.click(x=0.616, y=0.9694)\",\"image\":\"2dcdfd49-3596-4bab-8194-f64716a0684b.png\"},{\"action\":\"Click the plus (+) button in the Chrome browser tab bar to open a new tab.\",\"code\":\"pyautogui.click(x=0.232, y=0.0312)\",\"image\":\"721b8bcb-179f-4dee-b70e-4490997cdd98.png\"},{\"action\":\"Click on the email with the subject \\\"Yup C would like to view your calendar\\\" to open it and view the calendar sharing request.\",\"code\":\"pyautogui.click(x=0.6004, y=0.755)\",\"image\":\"fc0d78e4-ece4-4584-a01e-ed3dd4609b14.png\"},{\"action\":\"Click on the calendar sharing link in the email that starts with \\\"https://calendar.google.com/calendar/render?share=\\\" to accept the calendar request.\",\"code\":\"pyautogui.click(x=0.3805, y=0.5281)\",\"image\":\"1841101d-dcab-49b4-876d-65154acb7e85.png\"},{\"action\":\"Click the \\\"Send\\\" button to confirm sharing the calendar with mboss1909@gmail.com.\",\"code\":\"pyautogui.click(x=0.6473, y=0.6819)\",\"image\":\"6f75a79a-f3c4-4ee2-ba7b-c31f9ddaa944.png\"},{\"action\":\"Click the close button on the 'Sharing settings saved' notification toast to dismiss it.\",\"code\":\"pyautogui.click(x=0.5723, y=0.9819)\",\"image\":\"2d8e277c-81e3-414c-9395-5d4cbea0e2de.png\"},{\"action\":\"Click on the back button (clock icon with arrow) in the top-left corner of the Google Calendar settings page to return to the main calendar view.\",\"code\":\"pyautogui.click(x=0.0277, y=0.1787)\",\"image\":\"7da2c9b3-0ec2-47fa-add4-e193cea6d3b4.png\"},{\"action\":\"Click the refresh button in the browser to reload the Google Calendar page.\",\"code\":\"pyautogui.click(x=0.0527, y=0.0631)\",\"image\":\"4baddd51-181e-446b-bb2a-a3de9f44df72.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241104170903_c18d987e-d2de-4c13-91db-820ace9e1813","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"3ba959b9fde675b3d6126e173abbe8753ddac9472d265f5ce09637b49ebe2e3f"}
{"messages":[{"role":"user","content":"In Chrome Settings, Click on Payment methods, uncheck Allow sites to check if you have payment methods saved and Show which rewards and benefits are available for your cards at checkout. Click the question mark to view help information, Click on Addresses and more, click to add an address, fill in home for name and organization, 1st street for address, Beijing for district, city and province, 100081 for zip code, and click Save."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Payment methods\\\" option in the Autofill and passwords section.\",\"code\":\"pyautogui.click(x=0.3619, y=0.3197)\",\"image\":\"8e384b8e-1470-4a87-a4f8-9dae2940b241.png\"},{\"action\":\"Click on the toggle switch for \\\"Allow sites to check if you have payment methods saved\\\" to disable this setting.\",\"code\":\"pyautogui.click(x=0.5056, y=0.5276)\",\"image\":\"fa09ebe0-7fa4-454f-8c13-daf261500e23.png\"},{\"action\":\"Click on the question mark icon next to \\\"Show which rewards and benefits are available for your cards at checkout\\\" to view help information about this feature.\",\"code\":\"pyautogui.click(x=0.4748, y=0.4666)\",\"image\":\"230fb8c8-aa2c-497e-828d-fc5b332f20fa.png\"},{\"action\":\"Click on the toggle switch for \\\"Show which rewards and benefits are available for your cards at checkout\\\" to disable this setting.\",\"code\":\"pyautogui.click(x=0.6833, y=0.2224)\",\"image\":\"56b88137-bc28-485f-ad5a-74ff775d1b66.png\"},{\"action\":\"Click on the \\\"Settings - Payment methods\\\" tab in the browser's tab bar.\",\"code\":\"pyautogui.click(x=0.4185, y=0.0606)\",\"image\":\"aa6a549f-ce55-4eef-94ec-ce55253c4da1.png\"},{\"action\":\"Click on the \\\"Addresses and more\\\" option in the Autofill and passwords settings section.\",\"code\":\"pyautogui.click(x=0.3814, y=0.3735)\",\"image\":\"86182297-d179-4b53-9105-f255bcb7534a.png\"},{\"action\":\"Click the \\\"Add\\\" button next to the Addresses heading to create a new address entry.\",\"code\":\"pyautogui.click(x=0.6675, y=0.3599)\",\"image\":\"a1e17f8d-e0a9-4379-a8f1-1cfb28ec798d.png\"},{\"action\":\"Type \\\"home\\\" in the Name field, then press Enter key to move to the next field (Organization).\",\"code\":\"pyautogui.write(message='home')\\npyautogui.press('enter')\",\"image\":\"1e51821d-0923-492d-a4a9-5bd668064cd4.png\"},{\"action\":\"Click on the Organization field text input box to ensure it's active and ready for text entry.\",\"code\":\"pyautogui.click(x=0.4284, y=0.4853)\",\"image\":\"ba7fbabe-8aa7-4186-ad27-8988158c0584.png\"},{\"action\":\"Type \\\"home\\\" in the Organization field and press Enter to move to the next field.\",\"code\":\"pyautogui.write(message='home')\\npyautogui.press('enter')\",\"image\":\"7bbfb861-4360-4ba2-824e-f01e10367c85.png\"},{\"action\":\"Click on the Street address input field to focus it before typing the address.\",\"code\":\"pyautogui.click(x=0.4086, y=0.5576)\",\"image\":\"5d3d2ad9-7562-4b24-98fb-b90481ed1fef.png\"},{\"action\":\"Type \\\"1st\\\" in the Street address field, press Enter, press Space, and then type \\\"street\\\" to complete the street address entry.\",\"code\":\"pyautogui.write(message='1st')\\npyautogui.press('enter')\\npyautogui.press('space')\\npyautogui.write(message='street')\",\"image\":\"c92466e4-fb9f-4c58-aa92-ee7a135f4cf4.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241218094708_2608a2c1-73be-42e4-83fd-de202dd2264b","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"ebc9944efdd0f783d101623dc0f5536a85a4dc059c65e80edc51be0c93a2612b"}
{"messages":[{"role":"user","content":"In Android Studio, install the Android SDK Command-line Tools."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Settings...\\\" menu option in the Android Studio dropdown menu.\",\"code\":\"pyautogui.click(x=0.0527, y=0.0821)\",\"image\":\"2fd71f7e-f654-46b8-b66d-9ec1158e64d9.png\"},{\"action\":\"Click on the \\\"Build, Execution, Deployment\\\" option in the Settings menu to expand its sub-options.\",\"code\":\"pyautogui.click(x=0.2452, y=0.3554)\",\"image\":\"aef8b36e-9315-4445-9c2c-da440c9c3fb2.png\"},{\"action\":\"Click on the \\\"Android SDK\\\" option in the Languages & Frameworks section to access the Android SDK settings panel.\",\"code\":\"pyautogui.click(x=0.3698, y=0.2633)\",\"image\":\"e81a122d-7fcb-4a5e-beb2-205ca12f6209.png\"},{\"action\":\"Click on the \\\"SDK Tools\\\" tab in the Android SDK settings interface to view and access the tools installation options.\",\"code\":\"pyautogui.click(x=0.3987, y=0.2814)\",\"image\":\"066f19f5-d832-4d6a-ab62-058c399863cb.png\"},{\"action\":\"Click the checkbox next to \\\"Android SDK Command-line Tools (latest)\\\" to select it for installation.\",\"code\":\"pyautogui.click(x=0.3692, y=0.4279)\",\"image\":\"5de25b2e-b0a2-4278-ba7e-14c67d459983.png\"},{\"action\":\"Click the \\\"Apply\\\" button at the bottom of the SDK Manager dialog to start installing the selected Android SDK Command-line Tools.\",\"code\":\"pyautogui.click(x=0.7182, y=0.7829)\",\"image\":\"7203eeff-7466-435d-8b64-0b173d5004c4.png\"},{\"action\":\"Click the \\\"OK\\\" button in the confirmation dialog to proceed with installing the Android SDK Command-line Tools.\",\"code\":\"pyautogui.click(x=0.5335, y=0.5075)\",\"image\":\"7820a108-8c1a-454a-b50b-af7463bc179a.png\"},{\"action\":\"Click the \\\"Finish\\\" button in the SDK Component Installer dialog to complete the installation process.\",\"code\":\"pyautogui.click(x=0.7067, y=0.759)\",\"image\":\"3ddba147-038e-4092-9e79-352ac048aaaa.png\"},{\"action\":\"Click the \\\"OK\\\" button at the bottom right of the Settings dialog to save the changes and close the dialog.\",\"code\":\"pyautogui.click(x=0.7704, y=0.7854)\",\"image\":\"fe028e3c-9e23-4d9a-bba3-016b45b7b6cb.png\"},{\"action\":\"Terminate the process with success status as the Android SDK Command-line Tools have been successfully installed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"016c2379-dba4-4115-a791-8f7d01242129.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241025011452_33257642-d88a-4711-a682-d45be343a935","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"7c22e2ecd288038bc2c10066dab111770bde86ad10955068d67c342981634f3b"}
{"messages":[{"role":"user","content":"Search for HDMI to Type C converter product in Jingdong.com, select a product and view the reviews and Q&A, add it to shop cart and proceed to place the order"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome browser icon in the dock to launch it.\",\"code\":\"pyautogui.click(x=0.302, y=0.971)\",\"image\":\"252beab4-e139-4ec1-ba14-0b1fca1eb35b.png\"},{\"action\":\"Scroll down at the center of the product page to view more content and potentially find related products or the search function to look for an HDMI to Type C converter.\",\"code\":\"pyautogui.moveTo(x=0.466, y=0.601)\\npyautogui.scroll(10)\",\"image\":\"91765b82-eabc-48b0-885c-574b16360cb1.png\"},{\"action\":\"Select the text \\\"mini dp转hdmi\\\" in the search bar by clicking and dragging across it from right to left.\",\"code\":\"pyautogui.moveTo(x=0.480, y=0.315)\\npyautogui.dragTo(x=0.371, y=0.310, button='left')\",\"image\":\"3e7da69f-562d-4ecb-b702-48efaa2d3765.png\"},{\"action\":\"Type \\\"HDMI ZHUAN \\\" to begin replacing the selected search text with \\\"HDMI 转\\\" (HDMI to) as the first part of the new search term for HDMI to Type C converter.\",\"code\":\"pyautogui.write(message='HDMI ZHUAN ')\",\"image\":\"06bddec5-918e-4825-99bc-04582db7e455.png\"},{\"action\":\"Click on the \\\"hdmi转type-c\\\" option in the search suggestion dropdown menu to search for HDMI to Type C converters.\",\"code\":\"pyautogui.click(x=0.397, y=0.469)\",\"image\":\"aaea3eb2-310d-43d4-ae78-c692e363a01f.png\"},{\"action\":\"Scroll down in the middle of the filter options section to view the product listings below.\",\"code\":\"pyautogui.moveTo(x=0.354, y=0.696)\\npyautogui.scroll(-10)\",\"image\":\"20f70fe9-a964-4b1d-a111-a68b2cbaa483.png\"},{\"action\":\"Click on the \\\"UGREEN\\\" brand name in the product listing for the ¥99.00 Type-C to HDMI cable converter.\",\"code\":\"pyautogui.click(x=0.288, y=0.768)\",\"image\":\"07476b7f-a5d4-4a54-b7fb-da10f0b21156.png\"},{\"action\":\"Scroll down in the middle of the product page to view more product details, customer reviews, and Q&A sections.\",\"code\":\"pyautogui.moveTo(x=0.560, y=0.668)\\npyautogui.scroll(-10)\",\"image\":\"7924ffc0-4d5f-47a0-b3d4-4636780d0901.png\"},{\"action\":\"Click on the \\\"商品评价(10万+)\\\" (Product Reviews - 100,000+) tab to view customer reviews of the UGREEN Type-C to HDMI converter.\",\"code\":\"pyautogui.click(x=0.532, y=0.231)\",\"image\":\"f1707333-7a46-4757-be41-a6588f3c872c.png\"},{\"action\":\"Scroll down in the product reviews section to view more customer feedback about the UGREEN Type-C to HDMI converter.\",\"code\":\"pyautogui.moveTo(x=0.511, y=0.760)\\npyautogui.scroll(-10)\",\"image\":\"b11ab815-db90-4717-9d56-2a3ba4fd1d19.png\"},{\"action\":\"Click on the \\\"商品问答\\\" (Product Q&A) tab to view customer questions and answers about the product.\",\"code\":\"pyautogui.click(x=0.644, y=0.199)\",\"image\":\"295eaa7f-2b24-4955-a09c-74d8ae853176.png\"},{\"action\":\"Scroll down in the product Q&A section to view customer questions and answers about the UGREEN converter.\",\"code\":\"pyautogui.moveTo(x=0.533, y=0.574)\\npyautogui.scroll(10)\",\"image\":\"8edfd48d-57c8-41dc-91a9-cfcf661baa7b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241006233027_e5d788e3-af75-4160-b301-0dfae39bb8c5","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"cf0a6f06623310e189d3f8e504203eea094bd24033db77a94ada863f34d4241a"}
{"messages":[{"role":"user","content":"Filter the nonstop flights from New York to Los Angeles on jetBlue, sort the results by shortest flight first, see the details and seat map of the USD 389 flight, compare the different fare packages, select the fare Blue, add refundability service and proceed to traveler details"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Sort & Filter\\\" button to access flight filtering and sorting options.\",\"code\":\"pyautogui.click(x=0.194, y=0.654)\",\"image\":\"d99b1889-84fa-44f5-9a9a-c758c4ea752a.png\"},{\"action\":\"Click on the \\\"Nonstop\\\" checkbox to filter for flights without stops.\",\"code\":\"pyautogui.click(x=0.177, y=0.642)\",\"image\":\"4f89c160-510c-42c5-ac39-f02b01d4b030.png\"},{\"action\":\"Scroll down in the filter modal to see more filter options and locate the apply button.\",\"code\":\"pyautogui.moveTo(x=0.676, y=0.508)\\npyautogui.scroll(-10)\",\"image\":\"91dc10d5-beea-488b-b4ac-48193ff78a9c.png\"},{\"action\":\"Click the \\\"Apply\\\" button to confirm the selected filter options.\",\"code\":\"pyautogui.click(x=0.795, y=0.844)\",\"image\":\"4b21984a-dc58-4282-afc0-49128628a1df.png\"},{\"action\":\"Scroll down on the flight results page to view more content and find additional sorting options.\",\"code\":\"pyautogui.moveTo(x=0.847, y=0.437)\\npyautogui.scroll(-10)\",\"image\":\"061c57cd-65c6-4833-a8ed-f7fcf51a3a3f.png\"},{\"action\":\"Click on the \\\"Default\\\" dropdown menu under the Sort section to access sorting options.\",\"code\":\"pyautogui.click(x=0.415, y=0.344)\",\"image\":\"8e44b5be-987a-4394-b70f-1895768902de.png\"},{\"action\":\"Click on the \\\"Shortest flight\\\" option in the sorting dropdown menu to sort flights by duration.\",\"code\":\"pyautogui.click(x=0.286, y=0.658)\",\"image\":\"7b3f64f1-f71c-464d-a4c4-7bdd9160bb3c.png\"},{\"action\":\"Click the \\\"Apply\\\" button to confirm the filter and sort selections.\",\"code\":\"pyautogui.click(x=0.792, y=0.842)\",\"image\":\"85e313a2-2829-4216-9aa8-f961f9f8d32e.png\"},{\"action\":\"Scroll down to view more details of the $389 flight by moving the mouse to the middle of the flight results section and scrolling down.\",\"code\":\"pyautogui.moveTo(x=0.538, y=0.417)\\npyautogui.scroll(-10)\",\"image\":\"994a31ac-7ebd-4099-9984-83ce940d5e52.png\"},{\"action\":\"Click on the \\\"See fares\\\" link next to the $389 price for the 6:00am-9:20am flight to view the available fare options.\",\"code\":\"pyautogui.click(x=0.597, y=0.781)\",\"image\":\"69bb30bc-29ec-46a0-a1d9-f4fee4fa4dd8.png\"},{\"action\":\"Click on the \\\"Details\\\" tab to view additional flight information for the 6:00am-9:20am JetBlue flight.\",\"code\":\"pyautogui.click(x=0.477, y=0.408)\",\"image\":\"98afa690-62b3-4cc8-8650-364d5d1cc115.png\"},{\"action\":\"Click on the \\\"Seat Map\\\" tab to view the seating layout of the flight.\",\"code\":\"pyautogui.click(x=0.654, y=0.369)\",\"image\":\"b2395e1c-a6d9-431d-81e1-81de098a9608.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114161638_0068bf1b-cf10-4fdc-b027-1177a937e6e3","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"4f637dcee95776a63e14ef47d338b5e460f34800f1106336d59ea79afc8b193e"}
{"messages":[{"role":"user","content":"In Google Sites, open the Christmas Party template, change the theme to Simple, adjust the Font style to heavy, select the color as blue-purple, write the party name as FUN, the location as China, and the time as 8:18PM."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome icon in the dock to launch the web browser.\",\"code\":\"pyautogui.click(x=0.310, y=0.957)\",\"image\":\"3f663c1d-6ca3-4a13-9217-a35297e58a26.png\"},{\"action\":\"Scroll down to view more templates in the gallery, particularly to better see the Christmas Party template in the Small business section.\",\"code\":\"pyautogui.moveTo(x=0.421, y=0.648)\\npyautogui.scroll(-10)\",\"image\":\"4b18ceb0-9453-436b-a2c0-1c290c4243ff.png\"},{\"action\":\"Click on the Christmas Party template in the Small business section to select it.\",\"code\":\"pyautogui.click(x=0.350, y=0.636)\",\"image\":\"99c43e4b-7875-4807-8e40-6432c5bd4c8f.png\"},{\"action\":\"Scroll down in the template editor to view more of the Christmas Party template content.\",\"code\":\"pyautogui.moveTo(x=0.369, y=0.639)\\npyautogui.scroll(10)\",\"image\":\"1723e4a2-01f1-4d21-95c9-93d78f3811fb.png\"},{\"action\":\"Click on the Themes tab in the right sidebar to access theme customization options.\",\"code\":\"pyautogui.click(x=0.974, y=0.194)\",\"image\":\"e3eab68d-863e-4a35-9222-b912dab307e5.png\"},{\"action\":\"Click on the \\\"Simple\\\" theme card in the right sidebar to select it as the theme for the Christmas Party template.\",\"code\":\"pyautogui.click(x=0.912, y=0.468)\",\"image\":\"45932d37-8f4f-4dd3-825b-24eb4499b74e.png\"},{\"action\":\"Click on the purple color circle in the theme color options row to select it as the color scheme for the Christmas Party template.\",\"code\":\"pyautogui.click(x=0.968, y=0.565)\",\"image\":\"b92f6ac5-6fa0-42a3-abf7-925f58130183.png\"},{\"action\":\"Click on the \\\"Heavy\\\" font style option in the dropdown menu to set the font style to Heavy for the Christmas Party template.\",\"code\":\"pyautogui.click(x=0.902, y=0.645)\",\"image\":\"3a03163b-e5fe-4591-b830-ebd684074b6e.png\"},{\"action\":\"Drag the color selector handle from its current position at the red end of the color spectrum toward the blue end of the spectrum to adjust the template's color.\",\"code\":\"pyautogui.moveTo(x=0.868, y=0.626)\\npyautogui.dragTo(x=0.942, y=0.630, button='left')\",\"image\":\"2dca83bc-2774-4037-8549-c1fd2b2a504d.png\"},{\"action\":\"Fine-tune the color selection by dragging the color selector handle slightly within the blue-purple region of the color spectrum to adjust the exact shade.\",\"code\":\"pyautogui.moveTo(x=0.975, y=0.562)\\npyautogui.dragTo(x=0.960, y=0.577, button='left')\",\"image\":\"be0a0cd8-229a-44bb-94c6-d7f5fcfb49aa.png\"},{\"action\":\"Click on the blue checkmark icon in the color options row to confirm the selection of the blue-purple color theme.\",\"code\":\"pyautogui.click(x=0.831, y=0.529)\",\"image\":\"7d42357c-f8d3-4ddc-bb03-89f25bac6d03.png\"},{\"action\":\"Scroll down in the template editor to view more of the Christmas Party template content, particularly to better access the text fields that need editing.\",\"code\":\"pyautogui.moveTo(x=0.681, y=0.609)\\npyautogui.scroll(-10)\",\"image\":\"4790d2ae-37d7-406a-9e72-b1cf4d48c741.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241116223822_2121c076-882c-4c94-b71b-573b9176a891","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"317cc828e847ffa34c882a202df3530867a26883dcdf263a826629471cc926a4"}
{"messages":[{"role":"user","content":"Insert task 'Laundry' in note 'Tasks' in Evernote, set the task as due at 11:30 am today, add a reminder 2 minutes before the task, check the task as completed"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the blue circular formatting button in the toolbar to activate the task creation mode or to access task-related formatting options.\",\"code\":\"pyautogui.click(x=0.5082, y=0.2394)\",\"image\":\"819aeaa4-84b2-447d-a8d6-bf9501c59751.png\"},{\"action\":\"Click on the \\\"Task\\\" option in the dropdown menu to begin creating a new task in the note.\",\"code\":\"pyautogui.click(x=0.5336, y=0.215)\",\"image\":\"aa63ae53-2b39-4ba9-9b4c-d57348182acc.png\"},{\"action\":\"Type \\\"Laundry\\\" to name the task.\",\"code\":\"pyautogui.write(message='Laundry')\",\"image\":\"04f49181-ac21-4db9-b4bf-a548fbf4254e.png\"},{\"action\":\"Click on the calendar icon below the \\\"Laundry\\\" task to open the date and time picker for setting the due date.\",\"code\":\"pyautogui.click(x=0.7043, y=0.5006)\",\"image\":\"364f96e2-24bc-40f4-af0a-d02dffc9c537.png\"},{\"action\":\"Click on the \\\"Add a time\\\" link to open the time selection interface for setting the task due time to 11:30 am.\",\"code\":\"pyautogui.click(x=0.6055, y=0.7044)\",\"image\":\"1dfe66f3-8f0f-411d-b739-f6ceb0240ca8.png\"},{\"action\":\"Type \\\"11\\\" in the hour field of the time picker to change the time from 12:00 AM to 11:00 AM.\",\"code\":\"pyautogui.write(message='11')\",\"image\":\"04afe603-ae2a-4d28-a507-2fb002f2e8a1.png\"},{\"action\":\"Drag from the beginning to the end of the minute field to select the \\\"00\\\" text so it can be replaced with \\\"30\\\".\",\"code\":\"pyautogui.moveTo(x=0.5469, y=0.4419)\\npyautogui.dragTo(x=0.5727, y=0.4412, button='left')\",\"image\":\"6c467793-6474-4d5b-88e7-b1a5a0649af0.png\"},{\"action\":\"Click on the minutes field \\\"00\\\" in the time picker to position the cursor for typing.\",\"code\":\"pyautogui.click(x=0.5523, y=0.4437)\",\"image\":\"4eab1b5b-be19-47b0-be8f-89dfcd629c74.png\"},{\"action\":\"Type \\\"30\\\" to replace the selected \\\"00\\\" in the minutes field, setting the task time to 11:30 AM.\",\"code\":\"pyautogui.write(message='30')\",\"image\":\"e7b33708-d001-41c8-98d4-fbf9c9746a43.png\"},{\"action\":\"Click on the \\\"AM\\\" button to select morning time for the task.\",\"code\":\"pyautogui.click(x=0.5855, y=0.4387)\",\"image\":\"55e1fd11-e4ef-4161-80df-8a1964d1970e.png\"},{\"action\":\"Click the \\\"Done\\\" button to confirm and save the date and time settings for the task.\",\"code\":\"pyautogui.click(x=0.6613, y=0.7925)\",\"image\":\"26cda70d-d76f-43ba-8fd0-4efc682db427.png\"},{\"action\":\"Click on the bell icon next to the due date to open the reminder settings for the Laundry task.\",\"code\":\"pyautogui.click(x=0.7043, y=0.4925)\",\"image\":\"836779ef-17cd-48e1-aa62-ed791efb2bb5.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241122163724_534a4cf4-d4d2-40d1-b332-80965e5a615f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"86e5d300b6f769a230a0861d64aab2e25e745b149c74a6b2b0ed737e31f2a73b"}
{"messages":[{"role":"user","content":"Create an automation 'when the due date is marked as complete in a card by anyone, for each card linked in the attachments, check item \"{triggercardlink}\" in checklist \"Sub-Tasks\"' in Trello, test out the automation"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the lightning bolt icon in the top toolbar to access the automation features.\",\"code\":\"pyautogui.click(x=0.7922, y=0.2313)\",\"image\":\"bdd43c5a-f726-48bd-b742-d9a4227e513d.png\"},{\"action\":\"Click on the \\\"Rules\\\" option in the Automation panel to begin creating a rule for automating actions based on due dates.\",\"code\":\"pyautogui.click(x=0.8234, y=0.4988)\",\"image\":\"86a8142b-909f-40d2-90fc-4cdbff2b6bab.png\"},{\"action\":\"Click on the \\\"Create automation\\\" button to begin setting up a new automation rule.\",\"code\":\"pyautogui.click(x=0.7297, y=0.3456)\",\"image\":\"33efb95e-ee7b-4bda-8493-53ff3d22c093.png\"},{\"action\":\"Click on the \\\"+ Add Trigger\\\" button to begin selecting a trigger for the automation rule.\",\"code\":\"pyautogui.click(x=0.7109, y=0.6438)\",\"image\":\"153c9a9b-efe8-48db-bdca-ab56b4fde725.png\"},{\"action\":\"Click on the \\\"Dates\\\" trigger button with the clock icon to select due date-related triggers for the automation rule.\",\"code\":\"pyautogui.click(x=0.5824, y=0.6787)\",\"image\":\"7436ff82-6073-4b8b-ac14-7e1617554f22.png\"},{\"action\":\"Scroll down in the trigger selection panel to view more trigger options that might better match the requirement for \\\"when the due date is marked as complete in a card by anyone\\\".\",\"code\":\"pyautogui.moveTo(x=0.7422, y=0.7106)\\npyautogui.scroll(-3)\",\"image\":\"3a587b34-cd66-4a84-9e8b-07f2a5ada844.png\"},{\"action\":\"Click on the \\\"by anyone\\\" option in the dropdown menu to change the trigger condition from \\\"by me\\\" to \\\"by anyone\\\".\",\"code\":\"pyautogui.click(x=0.7316, y=0.6969)\",\"image\":\"13edae3f-436e-46db-9001-4718dec9ed27.png\"},{\"action\":\"Click on the blue plus button next to the \\\"when the due date is marked as complete in a card by anyone\\\" trigger condition to add this trigger to the automation rule.\",\"code\":\"pyautogui.click(x=0.9191, y=0.595)\",\"image\":\"8da28038-1987-41b8-a259-a994799e371f.png\"},{\"action\":\"Click on the Cascade button in the action selection panel to set up an action that will apply to cards linked in attachments.\",\"code\":\"pyautogui.click(x=0.4293, y=0.5806)\",\"image\":\"f3ea8bfc-fb5d-435a-986c-faad3083c776.png\"},{\"action\":\"Scroll down in the Cascade action configuration panel to view more options for configuring the automation rule.\",\"code\":\"pyautogui.moveTo(x=0.7316, y=0.6837)\\npyautogui.scroll(-3)\",\"image\":\"7ca516cc-10fc-430b-a367-839cee80d47a.png\"},{\"action\":\"Click the blue plus button next to \\\"for each card linked in the attachments\\\" option to add this condition to the automation rule.\",\"code\":\"pyautogui.click(x=0.9152, y=0.7163)\",\"image\":\"dea84c63-9a82-4a62-bbc7-e19ec5a1604f.png\"},{\"action\":\"Scroll down in the action configuration panel to view more automation action options.\",\"code\":\"pyautogui.moveTo(x=0.7219, y=0.675)\\npyautogui.scroll(-4)\",\"image\":\"e50beea8-8c06-4f01-a46e-684010d0a291.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241106191506_4e8b12cd-198c-4c2b-8656-11705c8f5dd5","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"e6b1ff13af7f423869829dfb47c29f7cdeddb27eb83981df04226a222609bef1"}
{"messages":[{"role":"user","content":"Preview the `fields` table in `RENEWABLE_ELECTRICITY_NETHERLANDS`, then drop the entire database."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Databases\\\" option in the left navigation sidebar to access the database listings.\",\"code\":\"pyautogui.click(x=0.056, y=0.331)\",\"image\":\"c43bc352-a428-4721-bf50-3beb29a516a8.png\"},{\"action\":\"Scroll down the database list to find RENEWABLE_ELECTRICITY_NETHERLANDS database.\",\"code\":\"pyautogui.moveTo(x=0.231, y=0.365)\\npyautogui.scroll(-10)\",\"image\":\"0bacde99-0f90-49cc-b250-1491984829cd.png\"},{\"action\":\"Click the \\\"Show more\\\" button at the bottom of the database list in the left sidebar to display additional database entries.\",\"code\":\"pyautogui.click(x=0.179, y=0.971)\",\"image\":\"7dd6f363-77c6-41ee-afd1-51bcd5f69d15.png\"},{\"action\":\"Scroll down in the database list in the left sidebar to reveal more database entries and continue searching for RENEWABLE_ELECTRICITY_NETHERLANDS.\",\"code\":\"pyautogui.moveTo(x=0.227, y=0.782)\\npyautogui.scroll(-10)\",\"image\":\"7db803a9-6e7f-4ebe-9234-d95597302b87.png\"},{\"action\":\"Click on the RENEWABLE_ELECTRICITY_NETHERLANDS database in the left sidebar to select it and view its contents.\",\"code\":\"pyautogui.click(x=0.223, y=0.805)\",\"image\":\"f690a655-704d-4c3b-87e2-842e60ec397d.png\"},{\"action\":\"Click on the INFORMATION_SCHEMA option under the RENEWABLE_ELECTRICITY_NETHERLANDS database in the left sidebar to access the database's metadata views.\",\"code\":\"pyautogui.click(x=0.158, y=0.835)\",\"image\":\"4c0941d2-d0bf-4561-86cd-fafd4df875a2.png\"},{\"action\":\"Scroll down in the left navigation panel to view more items under the INFORMATION_SCHEMA section.\",\"code\":\"pyautogui.moveTo(x=0.307, y=0.848)\\npyautogui.scroll(-10)\",\"image\":\"cbe50f77-207e-4112-b848-95b494b93896.png\"},{\"action\":\"Click on the PUBLIC schema option under the INFORMATION_SCHEMA section in the left sidebar to expand it and view its contents.\",\"code\":\"pyautogui.click(x=0.162, y=0.732)\",\"image\":\"e0f4d309-3101-47d9-84e3-af20353ec32a.png\"},{\"action\":\"Click on the \\\"VIEWS\\\" option under INFORMATION_SCHEMA in the left navigation panel.\",\"code\":\"pyautogui.click(x=0.171, y=0.726)\",\"image\":\"3b9e45d2-64ee-44f7-bc66-7cb64c962216.png\"},{\"action\":\"Scroll down in the VIEWS section of the left navigation panel to reveal more database objects.\",\"code\":\"pyautogui.moveTo(x=0.230, y=0.798)\\npyautogui.scroll(-10)\",\"image\":\"281fb6e8-2c45-44e0-b9e3-9bc3b28640ca.png\"},{\"action\":\"Click on the FIELDS item in the left navigation panel to view its contents.\",\"code\":\"pyautogui.click(x=0.229, y=0.798)\",\"image\":\"ac47d2b3-9535-4619-98fd-a5b65698ee0b.png\"},{\"action\":\"Scroll down in the FIELDS table view to see more field metadata entries.\",\"code\":\"pyautogui.moveTo(x=0.471, y=0.388)\\npyautogui.scroll(10)\",\"image\":\"6aeea404-1d4c-46f2-baaf-23b86d7f2107.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241125224857_4c590f5d-cfed-4122-a26f-72629d10c745","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"bf0a8677e2af1ed09e3ecbbcfadf23dfe41b0934c7783e1ee36499d2fe45ee42"}
{"messages":[{"role":"user","content":"On Airbnb’s Account page, access Personal info and add a Preferred name. Input ww in Preferred name (optional) and save. Then, add an Emergency contact, set the name to xiao, the relationship to sister, the language to English, and the email to 1234567890@gmail.com. Choose the Country code as China (+86) and set the phone number to 12345678901, then save."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Personal info\\\" section to access personal details settings where preferred name and emergency contact information can be added.\",\"code\":\"pyautogui.click(x=0.2474, y=0.4407)\",\"image\":\"6366fa17-69f5-4d41-8bf2-1312117555fc.png\"},{\"action\":\"Click the \\\"Add\\\" button next to the \\\"Preferred name\\\" section to open the form for adding a preferred name.\",\"code\":\"pyautogui.click(x=0.5333, y=0.4352)\",\"image\":\"5e2a55a3-1037-426c-805b-dcce0fdf3111.png\"},{\"action\":\"Click on the input field labeled \\\"Preferred name (optional)\\\" to focus it and prepare to type the preferred name \\\"ww\\\".\",\"code\":\"pyautogui.click(x=0.2745, y=0.512)\",\"image\":\"74b72237-6f3f-4e84-a67a-a6c3866c40b0.png\"},{\"action\":\"Type \\\"ww\\\" in the Preferred name field and press Enter to save the entry.\",\"code\":\"pyautogui.write(message='ww')\\npyautogui.press('enter')\",\"image\":\"522c2f93-62e0-4f5f-aba9-c7267fdb9f2e.png\"},{\"action\":\"Click the \\\"Save\\\" button below the Preferred name input field to save the entered preferred name \\\"ww\\\".\",\"code\":\"pyautogui.click(x=0.2224, y=0.5963)\",\"image\":\"453d8051-2c34-4662-a383-d76a810d5b5e.png\"},{\"action\":\"Click the \\\"Add\\\" button next to the Emergency contact section to begin adding emergency contact information.\",\"code\":\"pyautogui.click(x=0.5297, y=0.575)\",\"image\":\"dfbe885c-e4b6-4821-b98a-37acf2a8fb65.png\"},{\"action\":\"Click on the \\\"Name\\\" input field to begin entering the emergency contact name.\",\"code\":\"pyautogui.click(x=0.2667, y=0.4833)\",\"image\":\"2a20d92d-00a0-4e5a-bb3d-f7ba76874746.png\"},{\"action\":\"Type \\\"xiao\\\" in the Name field and press Enter to move to the next field.\",\"code\":\"pyautogui.write(message='xiao')\\npyautogui.press('enter')\",\"image\":\"19004f04-97bc-4c48-abc2-05f548d6b93b.png\"},{\"action\":\"Click on the Relationship input field to prepare to enter the emergency contact's relationship.\",\"code\":\"pyautogui.click(x=0.3042, y=0.5731)\",\"image\":\"0e49dfac-7539-4c4c-b798-6931960c5893.png\"},{\"action\":\"Type \\\"sister\\\" in the Relationship field and press Enter to move to the next field.\",\"code\":\"pyautogui.write(message='sister')\\npyautogui.press('enter')\",\"image\":\"c043a57a-2c85-4c92-8038-0a0d65a9073d.png\"},{\"action\":\"Click on the Preferred language dropdown field to open the language selection menu.\",\"code\":\"pyautogui.click(x=0.2948, y=0.638)\",\"image\":\"aab9ff28-7b8a-4b3f-9472-7d91aeb811b7.png\"},{\"action\":\"Click on the \\\"English\\\" option in the language dropdown menu to select it as the preferred language for the emergency contact.\",\"code\":\"pyautogui.click(x=0.2208, y=0.075)\",\"image\":\"8aa6b8e6-fad1-4fe6-9fc6-510bfc116eae.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241213221002_7b6f8180-2a9e-4c40-b367-a088f8e15097","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"4512ef821116d172cced5b5cd7fd99bc142e4de159960b6da56f3ef40e602c56"}
{"messages":[{"role":"user","content":"Compare baggage-inclusive flights to Dubai for April 2025 using Skyscanner website"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"From\\\" field that currently displays \\\"Miami (Any)\\\" to begin editing the departure location for the flight search.\",\"code\":\"pyautogui.click(x=0.1359, y=0.5861)\",\"image\":\"41fe0ae7-51de-46dc-a5c9-5aa491b22a33.png\"},{\"action\":\"Click on the \\\"To\\\" field displaying \\\"Lima (Any)\\\" to begin changing the destination to Dubai.\",\"code\":\"pyautogui.click(x=0.2823, y=0.5481)\",\"image\":\"aed49d6d-9416-41d3-b6a2-c53f2eba9389.png\"},{\"action\":\"Type \\\"dubai\\\" in the destination field and press Enter to select Dubai as the destination.\",\"code\":\"pyautogui.write(message='dubai')\\npyautogui.press('enter')\",\"image\":\"9977a90d-7d5f-4f63-b8de-245bc458baca.png\"},{\"action\":\"Click on the departure date field showing \\\"22/03/2025\\\" to begin changing the departure date from March to April 2025.\",\"code\":\"pyautogui.click(x=0.4557, y=0.5509)\",\"image\":\"6cc62534-bf96-44be-8051-845e9241e61a.png\"},{\"action\":\"Click on the April 10th date in the calendar to select it as the departure date.\",\"code\":\"pyautogui.click(x=0.7005, y=0.6148)\",\"image\":\"6bd44d57-bbdb-46e7-84b6-9286bc695fc2.png\"},{\"action\":\"Click on April 10th in the calendar to confirm the selection and close the date picker.\",\"code\":\"pyautogui.click(x=0.7005, y=0.7593)\",\"image\":\"b68e4f86-45cd-488e-b0de-f943e7107cfe.png\"},{\"action\":\"Click on the blue \\\"Search\\\" button to submit the flight search criteria and view available flights from London to Dubai on April 10th, 2025.\",\"code\":\"pyautogui.click(x=0.9135, y=0.3731)\",\"image\":\"2bba4e3d-a6e8-49fb-b226-b5b1ca5a9442.png\"},{\"action\":\"Click on the \\\"1 stop SAW\\\" text in the Pegasus Airlines flight details to expand more information about this flight connection.\",\"code\":\"pyautogui.click(x=0.4177, y=0.4519)\",\"image\":\"2345dac8-c38f-4c95-b243-5a4da1d336a7.png\"},{\"action\":\"Click on the \\\"Back to results\\\" button in the top left corner of the navigation bar to return to the main flight search results page.\",\"code\":\"pyautogui.click(x=0.0573, y=0.163)\",\"image\":\"0f905d89-a171-4d00-a93b-df873a5fef23.png\"},{\"action\":\"Click on the departure time \\\"10:05\\\" of the second Emirates flight to view more detailed information about this flight option.\",\"code\":\"pyautogui.click(x=0.3771, y=0.6537)\",\"image\":\"b171c0d9-6344-44a2-bc80-e7ba3004523f.png\"},{\"action\":\"Click on the departure time \\\"10:30\\\" of the Lufthansa flight to view more detailed information about this flight option.\",\"code\":\"pyautogui.click(x=0.3615, y=0.7194)\",\"image\":\"ca4326c2-2c8b-4419-9164-bb5a7c5f4717.png\"},{\"action\":\"Click on the \\\"Back to results\\\" button with the left arrow icon in the top left corner of the page to return to the main flight search results.\",\"code\":\"pyautogui.click(x=0.0245, y=0.1546)\",\"image\":\"c92f54fd-6b08-474c-b5b2-95b20d7d603e.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241203230419_850204cb-fdd5-4122-b407-2ca463263d4c","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"1c99f9a01cbf8eaf54e11434a2b731e1dbe15a3d74a27b2ef8c6852d55b94889"}
{"messages":[{"role":"user","content":"znshen: test sending image in the email"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Windows Start button in the taskbar to open the Start menu.\",\"code\":\"pyautogui.click(x=0.432, y=0.98)\",\"image\":\"bd97335a-4a2b-423d-a63a-c9dec805d53e.png\"},{\"action\":\"Click on the Google Chrome icon in the Windows taskbar to bring the browser into focus.\",\"code\":\"pyautogui.click(x=0.4227, y=0.8931)\",\"image\":\"0a335f2b-30b0-4209-a0a2-3b07607f3853.png\"},{\"action\":\"Click the plus (+) button in the upper right corner of the browser window to open a new tab.\",\"code\":\"pyautogui.click(x=0.8672, y=0.0119)\",\"image\":\"5698d79a-c881-4293-9f44-0d9d33967541.png\"},{\"action\":\"Type \\\"gmail\\\" in the Google search bar and press Enter to search for Gmail.\",\"code\":\"pyautogui.write(message='gmail')\\npyautogui.press('enter')\",\"image\":\"e0c7ef22-d180-4865-9c17-ae7f1538eedd.png\"},{\"action\":\"Click on the compose button (the blue button with a pencil icon) in the bottom left corner of the Gmail interface to start creating a new email.\",\"code\":\"pyautogui.click(x=0.0488, y=0.1356)\",\"image\":\"c6bc7f6c-6335-49a8-86ba-5d277ce82986.png\"},{\"action\":\"Type \\\"1641225799@qq.com\\\" in the \\\"To\\\" field of the email composition window.\",\"code\":\"pyautogui.write(message='1641225799@qq.com')\",\"image\":\"eb608a7e-ada7-49db-b5b9-200f4ac36f00.png\"},{\"action\":\"Click on the subject field (labeled \\\"主题\\\" in Chinese) in the email composition window to begin entering a subject for the email.\",\"code\":\"pyautogui.click(x=0.7586, y=0.7106)\",\"image\":\"96c2bcfb-d510-4a17-b965-9ba5a51b309c.png\"},{\"action\":\"Type \\\"test sending image in email\\\" in the subject field.\",\"code\":\"pyautogui.write(message='test sending image in email')\",\"image\":\"ffc75041-37d1-4dd5-bbca-24a6e56d333f.png\"},{\"action\":\"Click on the insert files icon (paperclip symbol) in the email compose toolbar to open the file upload dialog and attach an image to the email.\",\"code\":\"pyautogui.click(x=0.8207, y=0.9431)\",\"image\":\"5d878078-3d23-4d09-b9ae-20d3f4b2f262.png\"},{\"action\":\"Click the blue button in the file attachment dialog that allows uploading files from the computer.\",\"code\":\"pyautogui.click(x=0.4828, y=0.5513)\",\"image\":\"f233fb8f-7172-4934-8580-e52f22106543.png\"},{\"action\":\"Double-click on the \\\"Kowloon_Panorama_by_Ryan_Cheng_2010.jpg\\\" file to select it as the attachment for the email.\",\"code\":\"pyautogui.doubleClick(x=0.2746, y=0.4325)\",\"image\":\"406660d0-ddea-4717-8aea-4eacf35d819d.png\"},{\"action\":\"Click the blue \\\"发送\\\" (Send) button at the bottom of the email compose window to send the email with the attached image.\",\"code\":\"pyautogui.click(x=0.7016, y=0.9244)\",\"image\":\"70dded11-b85a-4e2f-8c35-6dfa5624a82b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240809110627_a49207a0-25f9-44a1-b3e3-31e26175559a","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"f4cb854044416b35d3e492f8a3a1acb4731c65f21321d7dbdec6531afa8a9e86"}
{"messages":[{"role":"user","content":"Go to WebLinx paper website, look through its huggingface dataset, explorer and fine-tuned model"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the URL bar at the end of \\\"google.co.tr\\\" to begin entering a new URL or search query.\",\"code\":\"pyautogui.click(x=0.542, y=0.057)\",\"image\":\"53f35ea0-2bf1-4e77-a615-3454315a85a3.png\"},{\"action\":\"Type \\\"weblinx\\\" in the URL bar and press Enter to search for WebLinx-related resources.\",\"code\":\"pyautogui.write(message='weblinx')\\npyautogui.press('enter')\",\"image\":\"1870fcf5-5994-4e82-90f7-ec992a71e587.png\"},{\"action\":\"Click on the \\\"WebLINX\\\" link in the first search result from McGill NLP to navigate to the official WebLINX project website.\",\"code\":\"pyautogui.click(x=0.162, y=0.371)\",\"image\":\"c103d65f-24f1-4772-b112-51713509a54e.png\"},{\"action\":\"Scroll down the webpage to view more content below the researchers' names section.\",\"code\":\"pyautogui.moveTo(x=0.289, y=0.740)\\npyautogui.scroll(-10)\",\"image\":\"3c72ba79-261e-4f34-9708-3b1c1026635d.png\"},{\"action\":\"Right-click on the hyperlink \\\"You can find our dataset on Huggingface Datasets\\\" to open the context menu with options for how to open this link.\",\"code\":\"pyautogui.rightClick(x=0.332, y=0.200)\",\"image\":\"929a94e9-01bd-4411-a15b-430a2dd072d5.png\"},{\"action\":\"Click on \\\"Open Link in New Tab\\\" option in the context menu to open the Huggingface dataset page in a new browser tab\",\"code\":\"pyautogui.click(x=0.344, y=0.217)\",\"image\":\"6fe116a8-c55a-42ad-8ee7-0941654f8c86.png\"},{\"action\":\"Scroll down the webpage to view more information about the WebLINX Explorer and potential details about the fine-tuned model.\",\"code\":\"pyautogui.moveTo(x=0.423, y=0.423)\\npyautogui.scroll(-10)\",\"image\":\"10f10062-abad-414a-84ae-930c49f02df5.png\"},{\"action\":\"Right-click on the \\\"Check out the WebLINX Explorer on Huggingface Spaces\\\" hyperlink to open a context menu with options for opening this link.\",\"code\":\"pyautogui.rightClick(x=0.372, y=0.449)\",\"image\":\"b2b871fa-c92d-4b82-b945-caf8a31a4710.png\"},{\"action\":\"Click on \\\"Open Link in New Tab\\\" in the context menu to open the WebLINX Explorer on Huggingface Spaces in a new browser tab.\",\"code\":\"pyautogui.click(x=0.391, y=0.460)\",\"image\":\"bd004c8c-c595-4406-8649-396c51cacb62.png\"},{\"action\":\"Scroll down on the current webpage to view more content about the WebLINX project, particularly to look for information about the fine-tuned model.\",\"code\":\"pyautogui.moveTo(x=0.417, y=0.537)\\npyautogui.scroll(-10)\",\"image\":\"d8f9407e-3248-4141-98f7-17d81875a15d.png\"},{\"action\":\"Click on the \\\"McGill-NLP/WebLINX - Datasets at Hugging Face\\\" tab in the browser tab bar to switch to the Huggingface dataset page.\",\"code\":\"pyautogui.click(x=0.675, y=0.111)\",\"image\":\"856b0351-5e33-4c1d-864d-8633a546e889.png\"},{\"action\":\"Scroll down on the Hugging Face Datasets page to view more examples of the WebLINX dataset.\",\"code\":\"pyautogui.moveTo(x=0.559, y=0.532)\\npyautogui.scroll(-10)\",\"image\":\"7b141cbf-e688-42ee-a773-4e00d8dae896.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241023191622_0b70e1fd-185d-404b-a3ee-90f7ae0b46fd","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"2c47251f2ff7bb6f504a1b02cebbe5885922a3026b1c3aca0dd73bf41baee0e2"}
{"messages":[{"role":"user","content":"Read an article relevant to environment which was published in October of 2022 by using advanced search on website arxiv.org"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Type \\\"arxiv\\\" in the Google search box and press Enter to search for the arxiv website.\",\"code\":\"pyautogui.write(message='arxiv')\\npyautogui.press('enter')\",\"image\":\"b72dea86-bfd1-43af-95e7-44157867177d.png\"},{\"action\":\"Click on the \\\"arXiv.org e-Print archive\\\" link in the Google search results to navigate to the arxiv website.\",\"code\":\"pyautogui.click(x=0.243, y=0.476)\",\"image\":\"7e0ce259-5745-49b5-b761-13ebb8de10c4.png\"},{\"action\":\"Click on the \\\"Advanced Search\\\" link below the search bar to access more detailed search options.\",\"code\":\"pyautogui.click(x=0.810, y=0.349)\",\"image\":\"d6d60fa9-ab94-4fda-a825-d150c554765f.png\"},{\"action\":\"Click on the \\\"environment\\\" option in the dropdown suggestion list below the search term field to populate the search field with this term.\",\"code\":\"pyautogui.click(x=0.177, y=0.612)\",\"image\":\"20d9c07e-c0f8-4bed-b01d-77111d369369.png\"},{\"action\":\"Move the mouse to the lower part of the Subject section and scroll down to reveal more search options, likely including date range filters needed to specify October 2022 publications.\",\"code\":\"pyautogui.moveTo(x=0.410, y=0.648)\\npyautogui.scroll(-10)\",\"image\":\"d0b5c262-d5a9-4020-94fb-96c66e4963cd.png\"},{\"action\":\"Click on the text field next to the \\\"Specific year\\\" option to enter or confirm the year 2022 for the date filter.\",\"code\":\"pyautogui.click(x=0.189, y=0.680)\",\"image\":\"30780e07-d118-4bed-8bab-50addc130c1e.png\"},{\"action\":\"Click in the \\\"From\\\" date field in the Date range section to begin entering the starting date for the October 2022 search filter.\",\"code\":\"pyautogui.click(x=0.094, y=0.807)\",\"image\":\"22f5ee68-71ce-4fb0-99eb-86c4b1fc88d1.png\"},{\"action\":\"Click on the \\\"2022-10-31\\\" date in the calendar popup to confirm the end date selection for the date range.\",\"code\":\"pyautogui.click(x=0.241, y=0.864)\",\"image\":\"cf393a15-810d-42d4-ada1-117353e3ff4d.png\"},{\"action\":\"Scroll down the page to reveal more content, particularly to find the search button that will execute the query with our configured parameters.\",\"code\":\"pyautogui.moveTo(x=0.244, y=0.797)\\npyautogui.scroll(-10)\",\"image\":\"15f3ba18-e41a-406a-ac6e-e4d4d1c9087c.png\"},{\"action\":\"Click the blue \\\"Search\\\" button to execute the search for environment-related articles published in October 2022.\",\"code\":\"pyautogui.click(x=0.112, y=0.871)\",\"image\":\"1e125771-99c3-496f-88a2-07b45c82ebd3.png\"},{\"action\":\"Scroll down the search results page to view more environment-related articles from October 2022.\",\"code\":\"pyautogui.moveTo(x=0.461, y=0.564)\\npyautogui.scroll(-10)\",\"image\":\"8e215c29-da9b-4280-adb2-805053933829.png\"},{\"action\":\"Click on the PDF link for the article \\\"an intelligent security centered resource-efficient resource management model for cloud computing environments\\\" (arXiv:2210.16602).\",\"code\":\"pyautogui.click(x=0.283, y=0.511)\",\"image\":\"76f4f3e9-a89e-4e8a-af5c-93c2602c20c4.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241028111050_ecccdef0-2e2c-4738-8540-6fd61153766c","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"548263a6994b6f78e865d394549d886de351ccc4af469aaf2f81409190710583"}
{"messages":[{"role":"user","content":"Use ChatGPT to generate a script about the impact of gaming on teens. Get the generated script, open the blue icon Lumen5, select the Billow template in organic to automatically generate the video, publish the video when you're done and comment \"a good idea\"."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the message input field labeled \\\"Message ChatGPT...\\\" to begin typing a request for a script about the impact of gaming on teens.\",\"code\":\"pyautogui.click(x=0.4594, y=0.5269)\",\"image\":\"0e7054b6-d2b4-4c22-bd85-6a228f39b95a.png\"},{\"action\":\"Write \\\"generate a script for a short video about \\\"the impact of gaming on teenagers\\\"\\\" in the message input field.\",\"code\":\"pyautogui.write(message='generate a script  for a short video about \\\"the impact of gaming on teenagers\\\"')\",\"image\":\"c38984e1-fc74-462e-8a99-aaba5cd9cd57.png\"},{\"action\":\"Click the send button (black circular button with white upward arrow) to submit the message to ChatGPT.\",\"code\":\"pyautogui.click(x=0.8187, y=0.5287)\",\"image\":\"7d5ccad7-b7c8-4748-b79b-5eafd10d2c60.png\"},{\"action\":\"Click on the script content in the \\\"Scene 1: The Benefits of Gaming\\\" section, specifically near the text describing a teenager playing a strategy game.\",\"code\":\"pyautogui.click(x=0.5823, y=0.8028)\",\"image\":\"739695a8-6b7d-4ed4-bb39-21ca7c11b10a.png\"},{\"action\":\"Click on the copy button (document icon) below the \\\"End of script\\\" text to copy the entire script to the clipboard.\",\"code\":\"pyautogui.click(x=0.3896, y=0.6963)\",\"image\":\"7830c9ac-d9d9-441b-9ad1-ceeb2bee6b61.png\"},{\"action\":\"Click on the second browser tab labeled \\\"新标签页\\\" (which appears to be a \\\"New Tab\\\" page in Chinese) at the top of the browser window.\",\"code\":\"pyautogui.click(x=0.2516, y=0.0324)\",\"image\":\"956d6e8d-5ebe-450a-94bc-20c733003c1d.png\"},{\"action\":\"Click on the Google search bar that contains the placeholder text \\\"在 Google 中搜索，或输入网址\\\" (Search in Google or enter a URL) to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.4057, y=0.4676)\",\"image\":\"438ef861-c164-4e93-a1ec-b0caf8629fac.png\"},{\"action\":\"Type \\\"lumen5\\\" in the Google search bar and press Enter to search for the Lumen5 website.\",\"code\":\"pyautogui.write(message='lumen5')\\npyautogui.press('enter')\",\"image\":\"42b11e5f-3fb7-4ace-9dff-fdff60dc5320.png\"},{\"action\":\"Scroll down on the Google search results page to see more results for Lumen5.\",\"code\":\"pyautogui.moveTo(x=0.3521, y=0.512)\\npyautogui.scroll(-4)\",\"image\":\"a4f8c91a-c6e4-417b-8dfa-634f635fe559.png\"},{\"action\":\"Click on the Lumen5 website link that says \\\"Lumen5 - Video Maker | Create Videos Online in Minutes\\\" to access the official Lumen5 platform.\",\"code\":\"pyautogui.click(x=0.225, y=0.6287)\",\"image\":\"16db41ed-0508-4a81-b45b-f5b32cf05b6d.png\"},{\"action\":\"Click on the \\\"New video\\\" purple tile to start creating a new video.\",\"code\":\"pyautogui.click(x=0.2687, y=0.5426)\",\"image\":\"c12870a0-e812-4c71-af44-8ba36fa5a0cd.png\"},{\"action\":\"Click on the \\\"Organic\\\" option in the left sidebar menu to view organic style templates.\",\"code\":\"pyautogui.click(x=0.0745, y=0.4426)\",\"image\":\"d566ca01-8f3a-4a42-b325-150cc172eb97.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241018122553_e79fbbfe-3d09-4810-bd92-dca23a1d971e","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"324a81b63ee41ab93cd3d6f86c364e0955d0a8fd74dd1f84582d6ec3487319e3"}
{"messages":[{"role":"user","content":"Enter Blogger.com from Google search, sign in with Google account mboss1909c@gmail.com, create a blog titled Stargazer, set the address of the blog as 'stargazer1909.blogspot.com', set the display name as 'Stargazer', view the blog"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google search bar to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.4465, y=0.3962)\",\"image\":\"6ee77b55-72ba-4756-a81b-1cbb7634bee3.png\"},{\"action\":\"Type \\\"blogger\\\" in the Google search bar and press Enter to search for Blogger.\",\"code\":\"pyautogui.write(message='blogger')\\npyautogui.press('enter')\",\"image\":\"600c717b-f9ce-488f-9986-ff85e2709ef3.png\"},{\"action\":\"Click on the title \\\"Blogger.com - Create a unique and beautiful blog easily\\\" in the search results to navigate to the Blogger website.\",\"code\":\"pyautogui.click(x=0.2703, y=0.4119)\",\"image\":\"7bd77155-a12a-45ad-89c2-58c9314f7a70.png\"},{\"action\":\"Click on the \\\"SIGN IN\\\" button located in the top-right corner of the Blogger homepage to begin the sign-in process with the Google account.\",\"code\":\"pyautogui.click(x=0.9363, y=0.1794)\",\"image\":\"52cb24b0-7cc5-428e-8934-831041ba27b4.png\"},{\"action\":\"Click on the \\\"Yup C mboss1909c@gmail.com\\\" account to select it and continue with the sign-in process.\",\"code\":\"pyautogui.click(x=0.6227, y=0.4156)\",\"image\":\"287ef189-5ac6-49ad-8bfb-2829a1a06090.png\"},{\"action\":\"Click on the Title input field to begin entering the blog name.\",\"code\":\"pyautogui.click(x=0.5082, y=0.5156)\",\"image\":\"43106124-72e7-4d02-8816-91c0d0fc4785.png\"},{\"action\":\"Type \\\"Stargazer\\\" in the Title input field.\",\"code\":\"pyautogui.write(message='Stargazer')\",\"image\":\"8b958386-823d-4139-9e7c-5a1086783394.png\"},{\"action\":\"Click the \\\"NEXT\\\" button to proceed to the next step in the blog creation process.\",\"code\":\"pyautogui.click(x=0.6258, y=0.7688)\",\"image\":\"cff26efe-728c-492b-8a8b-fe7ca3fc4852.png\"},{\"action\":\"Click on the Address input field to begin typing the blog URL.\",\"code\":\"pyautogui.click(x=0.4469, y=0.5194)\",\"image\":\"1428ba97-2251-468d-872c-9dd0ef749b68.png\"},{\"action\":\"Type \\\"stargazer1909\\\" in the Address input field to create the blog URL \\\"stargazer1909.blogspot.com\\\".\",\"code\":\"pyautogui.write(message='stargazer1909')\",\"image\":\"bdefedcd-b5c1-40a5-b16f-db32db4c14a1.png\"},{\"action\":\"Click the \\\"NEXT\\\" button to proceed to the next step in the blog creation process.\",\"code\":\"pyautogui.click(x=0.6121, y=0.7688)\",\"image\":\"ba96912d-63da-4de1-9c9c-a03cc234e030.png\"},{\"action\":\"Click on the \\\"Display name\\\" text field to begin entering the blog's display name.\",\"code\":\"pyautogui.click(x=0.4004, y=0.5319)\",\"image\":\"dea0e2c7-5f51-4c8e-8f52-7dcc5284b5e7.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241113165330_76064d78-2bbc-49f9-b5ae-a750487f8470","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"d7703526c57e1eec3c0b36c960fd4c8d937ee1a124d2202846eab676f833982c"}
{"messages":[{"role":"user","content":"In Google Play, add it to wishlist and read the free sample. Read pages 2 to 5. Then change the font to Times New Roman, size to 88%, dark theme, and two-page layout. Click on the display options button, add a bookmark. Then view contents and click on Semana 7 - Día 1."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Add to wishlist\\\" button to save this book for later.\",\"code\":\"pyautogui.click(x=0.3832, y=0.4971)\",\"image\":\"73c3cb02-2fc2-4a77-b213-2d0f529c5667.png\"},{\"action\":\"Click on the \\\"Free sample\\\" button to open the e-book's free sample.\",\"code\":\"pyautogui.click(x=0.3783, y=0.5943)\",\"image\":\"f8ac08e4-0356-495b-880e-83dd6c20a98c.png\"},{\"action\":\"Double-click on the next page arrow button to navigate from page 2 to page 3 of the book.\",\"code\":\"pyautogui.doubleClick(x=0.7877, y=0.894)\",\"image\":\"772caaab-d313-44c8-883b-3123aeee32ec.png\"},{\"action\":\"Click on the right arrow navigation button at the bottom of the page to move to the next page (page 5).\",\"code\":\"pyautogui.click(x=0.7877, y=0.894)\",\"image\":\"ae19bb5e-42e1-471c-a849-60907eb3e84c.png\"},{\"action\":\"Click on the display options button (the \\\"A\\\" icon) in the top toolbar to access reading settings and preferences.\",\"code\":\"pyautogui.click(x=0.7713, y=0.1505)\",\"image\":\"c0d42c5e-cb35-4298-b780-4038fb6f0294.png\"},{\"action\":\"Click on the font dropdown menu showing \\\"Original\\\" to see available font options.\",\"code\":\"pyautogui.click(x=0.8444, y=0.4157)\",\"image\":\"fb35a66f-3bf1-407a-b5f4-4f796d11d792.png\"},{\"action\":\"Click on \\\"Times New Roman\\\" in the font dropdown menu to change the book's font.\",\"code\":\"pyautogui.click(x=0.8184, y=0.6388)\",\"image\":\"dbad5f3e-e548-4527-8f4e-2dab34d4c196.png\"},{\"action\":\"Click on the font size adjustment control with the \\\"T\\\" symbol to adjust the font size from 100% to 88%.\",\"code\":\"pyautogui.click(x=0.8136, y=0.5387)\",\"image\":\"95dda507-604d-40dd-8eb1-5901ff321769.png\"},{\"action\":\"Click on the Dark theme toggle switch at the top of the Display options panel to enable dark mode for reading.\",\"code\":\"pyautogui.click(x=0.9542, y=0.2927)\",\"image\":\"6f616c7d-03c5-4b86-bacb-47b9fbaf4eef.png\"},{\"action\":\"Scroll down in the Display options panel to reveal more settings, particularly the Page layout options that should be below the current visible settings.\",\"code\":\"pyautogui.moveTo(x=0.8593, y=0.622)\\npyautogui.scroll(-20)\",\"image\":\"0e74b95c-4885-4545-b49e-b68fb82bd968.png\"},{\"action\":\"Click on the two-page layout button (the middle option) in the Page layout section of the Display options panel.\",\"code\":\"pyautogui.click(x=0.878, y=0.8211)\",\"image\":\"3fe96f28-7336-4edd-8b26-eae4b66881ba.png\"},{\"action\":\"Click on the two-page layout button in the Display options panel to change the page layout.\",\"code\":\"pyautogui.click(x=0.8347, y=0.1549)\",\"image\":\"98a8ba0c-dfcc-4c92-8add-e41011c3dc1d.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241122144154_cd6a5538-2773-44d5-9b37-7afd92984892","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c725dfc947d890e0cfac0bc79712df235804a0726b1ed039a33e2ed6fb622c97"}
{"messages":[{"role":"user","content":"In YouTube, upload a video, set the title as 'billiards', set the description as 'cool', set the video as private video, then save it and play it."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome icon in the taskbar at the bottom of the screen.\",\"code\":\"pyautogui.click(x=0.2172, y=0.9824)\",\"image\":\"a880620d-95e8-4a62-a87b-efd0d7defe8d.png\"},{\"action\":\"Click on the camera icon with plus symbol (create/upload button) in the top-right section of the YouTube interface to access the upload options.\",\"code\":\"pyautogui.click(x=0.8901, y=0.1352)\",\"image\":\"66ce3b8f-8101-4f9c-b2c4-8a2d168511c1.png\"},{\"action\":\"Click on the \\\"Upload video\\\" option in the dropdown menu that appears after clicking the create/upload button.\",\"code\":\"pyautogui.click(x=0.9448, y=0.1972)\",\"image\":\"91099bab-2671-4038-be06-b44390ad53a1.png\"},{\"action\":\"Click the \\\"Select files\\\" button in the upload dialog to browse and select a video file from the computer.\",\"code\":\"pyautogui.click(x=0.4995, y=0.6361)\",\"image\":\"1fc52a0f-05e7-4efd-9e16-910de2fc1e2c.png\"},{\"action\":\"Click on the \\\"Select files\\\" button to open the file browser dialog.\",\"code\":\"pyautogui.click(x=0.5005, y=0.6519)\",\"image\":\"2830928f-85a1-4876-93e0-64cbe213f50f.png\"},{\"action\":\"Double-click on the video_20240109_500151.mp4 file to select it for upload to YouTube.\",\"code\":\"pyautogui.doubleClick(x=0.2026, y=0.2407)\",\"image\":\"b5562d54-4599-4c95-86fe-d86e26096a6d.png\"},{\"action\":\"Type \\\"billiards\\\" to replace the currently selected title text.\",\"code\":\"pyautogui.write(message='billiards')\",\"image\":\"74ca6059-c089-4c10-93c1-c24d26b6cf86.png\"},{\"action\":\"Click in the description field where the placeholder text \\\"Tell viewers about your video (type @ to mention a channel)\\\" is displayed, to prepare for typing the description.\",\"code\":\"pyautogui.click(x=0.4328, y=0.6037)\",\"image\":\"5a57aa5c-f615-4fb2-a5db-fb0019737aec.png\"},{\"action\":\"Type \\\"cool\\\" in the description field.\",\"code\":\"pyautogui.write(message='cool')\",\"image\":\"9d4fef5b-0654-4295-bf1f-bd4620da838d.png\"},{\"action\":\"Scroll down in the description field area of the video upload form to view more options below the currently visible content.\",\"code\":\"pyautogui.moveTo(x=0.4318, y=0.6046)\\npyautogui.scroll(-3)\",\"image\":\"f3d61b35-d5b2-4276-974f-72256d32eb2b.png\"},{\"action\":\"Click the \\\"Next\\\" button located in the bottom-right corner of the YouTube Studio upload dialog to proceed to the next step of the video upload process.\",\"code\":\"pyautogui.click(x=0.7734, y=0.8565)\",\"image\":\"7c639235-9985-4970-b5a0-d7a0628b37e2.png\"},{\"action\":\"Click on the \\\"Yes, it's made for kids\\\" radio button to select this option for the video classification question.\",\"code\":\"pyautogui.click(x=0.3005, y=0.6611)\",\"image\":\"b63d236e-1d65-4237-9197-07dbbf4e3eb0.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241026185105_41c2aa3b-6780-4ca3-9e6b-901432fd8fab","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"dd02be519221cd0e24f5ce919ff7fc8c5340d12df14b2a23aabaddbbdec552db"}
{"messages":[{"role":"user","content":"Download the assigned book(Nonlinear programming) from Z-library and open it with Zhiyuntranslator"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click in the URL bar where it says \\\"Search or enter web address\\\" to begin typing the Z-library web address.\",\"code\":\"pyautogui.click(x=0.2016, y=0.0491)\",\"image\":\"cfd22a6e-8c3e-48c4-a3f0-22bae2a81b4d.png\"},{\"action\":\"Type \\\"z-library\\\" in the URL bar and press Enter to navigate to the Z-library website.\",\"code\":\"pyautogui.write(message='z-library')\\npyautogui.press('enter')\",\"image\":\"6d43c1b7-8702-4fb9-9141-8b70b7dbaf5e.png\"},{\"action\":\"Click on the first search result link titled \\\"Z-Library – 世界上最大的电子图书馆。自由访问知识和文化。\\\" to navigate to the Z-Library website.\",\"code\":\"pyautogui.click(x=0.274, y=0.3796)\",\"image\":\"15007fe0-bfa4-4c1a-9337-c50040ef7193.png\"},{\"action\":\"Click in the search box to prepare for typing \\\"Nonlinear programming\\\" in the main search field of the Z-Library website.\",\"code\":\"pyautogui.click(x=0.4219, y=0.7167)\",\"image\":\"e5d4c661-0452-49dc-af89-5c993bca54b6.png\"},{\"action\":\"Type \\\"nonlinear programming\\\" in the search field and press Enter to search for the book.\",\"code\":\"pyautogui.write(message='nonlinear programming')\\npyautogui.press('enter')\",\"image\":\"b5d0b845-2d3d-47bf-8bae-aa4fc1048946.png\"},{\"action\":\"Click on the book title \\\"Nonlinear Programming\\\" by Dimitri P. Bertsekas to view its details page and access download options.\",\"code\":\"pyautogui.click(x=0.2167, y=0.55)\",\"image\":\"9f3ac067-8e5b-4108-8777-0b50faf48bfd.png\"},{\"action\":\"Click on the download button labeled \\\"(下载PDF, 24.03 MB)\\\" to download the PDF version of the Nonlinear Programming book.\",\"code\":\"pyautogui.click(x=0.2578, y=0.7287)\",\"image\":\"f366abc6-337a-458c-a6c1-a11a21288748.png\"},{\"action\":\"Click on the \\\"Save as\\\" option in the download dialog to choose where to save the PDF file of \\\"Nonlinear Programming\\\" by Dimitri P. Bertsekas.\",\"code\":\"pyautogui.click(x=0.8073, y=0.1731)\",\"image\":\"30771fc3-adef-4424-b870-2e2ec24e7e3f.png\"},{\"action\":\"Click on the \\\"新加卷 (F:)\\\" drive in the navigation panel of the Save As dialog to select it as the save location for the PDF file.\",\"code\":\"pyautogui.click(x=0.0563, y=0.4306)\",\"image\":\"3e204131-4d6b-45a7-86de-19eb1a3d3f99.png\"},{\"action\":\"Double-click on the \\\"人工智能\\\" folder to open it and navigate into that directory for saving the PDF file.\",\"code\":\"pyautogui.doubleClick(x=0.1609, y=0.3722)\",\"image\":\"f1e7fff3-99d2-4fcf-abce-d4e0d4f7be15.png\"},{\"action\":\"Double-click on the \\\"数学\\\" (Mathematics) folder to navigate into it before saving the PDF file.\",\"code\":\"pyautogui.doubleClick(x=0.1625, y=0.313)\",\"image\":\"73eaf4ac-3051-403f-b7cd-e90429cad857.png\"},{\"action\":\"Click the \\\"Save\\\" button to save the PDF file in the current mathematics folder.\",\"code\":\"pyautogui.click(x=0.313, y=0.5472)\",\"image\":\"9dafbf99-3101-48c4-9dbf-e67fcbe4cccb.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241011133141_87e4ddc6-cc0b-462b-80ef-4b5d1d7afc83","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"a30eb5ab91a36e69ac69e4681de6882dabbdfe849d518bb45804169f27b527ca"}
{"messages":[{"role":"user","content":"Enable the waiting room feature for participants joining a Zoom meeting"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Meetings\\\" option in the left sidebar navigation menu to access meeting settings where the waiting room feature can be configured.\",\"code\":\"pyautogui.click(x=0.2349, y=0.6861)\",\"image\":\"76c747e9-27be-4cfe-ae3b-6cba98463772.png\"},{\"action\":\"Click on the \\\"Settings\\\" option in the left sidebar navigation menu to access the Zoom settings page where meeting configurations can be managed.\",\"code\":\"pyautogui.click(x=0.0609, y=0.6593)\",\"image\":\"d87428c8-f54f-40f6-9210-228d2f0c935c.png\"},{\"action\":\"Terminate the task with success status as the waiting room feature is already enabled.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"df957b1c-c8bb-49d3-8f64-26fc14c56e91.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241111172042_38adbcfd-7916-4abe-920a-970935556438","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"edf42cebba6e86d494b3e78f2955b67ef6a6be0ac1a0d5f5fd28be6e7fbd90d4"}
{"messages":[{"role":"user","content":"Create a new text channel on which only admin role users can write commands to the MEE6 bot."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Edytuj kanał\\\" (Edit channel) button next to the \\\"#ogólny\\\" channel in the left sidebar to access channel settings.\",\"code\":\"pyautogui.click(x=0.1854, y=0.3019)\",\"image\":\"735614e0-ec61-49b1-9e04-03bbe79f26b0.png\"},{\"action\":\"Write \\\"mee6-admin-info\\\" in the channel name field.\",\"code\":\"pyautogui.write(message='mee6-admin-info')\",\"image\":\"4b98cad3-9850-422d-b1ed-3c03c0f19eac.png\"},{\"action\":\"Click the \\\"Utworzenie kanału\\\" (Create channel) button to create the new channel with the specified settings.\",\"code\":\"pyautogui.click(x=0.5906, y=0.7889)\",\"image\":\"a38173aa-868a-4220-b8fc-d27bdada96d0.png\"},{\"action\":\"Click on the \\\"Edytuj kanał\\\" (Edit channel) button that appears next to the \\\"mee6-admin-info\\\" channel in the left sidebar.\",\"code\":\"pyautogui.click(x=0.1844, y=0.3704)\",\"image\":\"f8731ee3-bc11-466a-acac-4a5cdb3f1a89.png\"},{\"action\":\"Click on the \\\"Uprawnienia\\\" (Permissions) option in the left sidebar of the channel settings dialog.\",\"code\":\"pyautogui.click(x=0.2323, y=0.2241)\",\"image\":\"f3d8f24a-1efd-4072-9e6d-c5a66d08fd6a.png\"},{\"action\":\"Click the green checkmark button next to the \\\"Wyświetlanie kanału\\\" (View channel) permission to confirm that everyone should be able to see the channel.\",\"code\":\"pyautogui.click(x=0.7427, y=0.625)\",\"image\":\"87113fce-1d0b-4df8-b2d3-c58f7dc7cd4a.png\"},{\"action\":\"Scroll down in the permissions list by positioning the mouse in the permissions area and scrolling downward to reveal additional permission options.\",\"code\":\"pyautogui.moveTo(x=0.7078, y=0.737)\\npyautogui.scroll(-6)\",\"image\":\"a11e9c54-13ab-4d1d-a080-1a49f87d3f53.png\"},{\"action\":\"Click on the red X button next to the \\\"Wysyłanie wiadomości\\\" (Send Messages) permission to deny this permission for the @everyone role.\",\"code\":\"pyautogui.click(x=0.7047, y=0.725)\",\"image\":\"89629c27-e316-49e3-865d-77451ee7ad5b.png\"},{\"action\":\"Click the \\\"Zapisz zmiany\\\" (Save changes) button at the bottom of the screen to save the permission changes made to the channel.\",\"code\":\"pyautogui.click(x=0.7323, y=0.9481)\",\"image\":\"e42de398-6b0c-45d8-a41e-cb3fe27dccd0.png\"},{\"action\":\"Click on the plus (+) button next to the roles/members section to add a new role to the channel permissions.\",\"code\":\"pyautogui.click(x=0.4609, y=0.3481)\",\"image\":\"0dc62e5e-3edf-4898-b130-9ecebacfc354.png\"},{\"action\":\"Click on the MEE6#4876 user in the members list to add it to the channel permissions.\",\"code\":\"pyautogui.click(x=0.3667, y=0.663)\",\"image\":\"859fc2c0-db27-426f-81f4-fb0bcb56bec5.png\"},{\"action\":\"Click the green checkmark button next to the \\\"Wyświetlanie kanału\\\" (View channel) permission to grant this permission to the MEE6#4876 bot.\",\"code\":\"pyautogui.click(x=0.7448, y=0.4028)\",\"image\":\"1e784741-6b21-449e-8434-2f8083e0b268.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241029064706_0d6c8e76-f647-4797-a848-311529dae207","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"79c2bc03a088bd0ebbd0ee0ed68897295dbc3028d80e351650b87f6ca2e1bc18"}
{"messages":[{"role":"user","content":"Open Twitter, locate Elon Musk's most recent post, and share it via email"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the plus (+) icon next to the current browser tab to open a new tab.\",\"code\":\"pyautogui.click(x=0.286, y=0.064)\",\"image\":\"a94e5ea8-fc01-48d6-906f-df3fbe4ec706.png\"},{\"action\":\"Type \\\"twitter\\\" in the Google search bar and press Enter to search for Twitter.\",\"code\":\"pyautogui.write(message='twitter')\\npyautogui.press('enter')\",\"image\":\"4d07824b-486d-4bf8-80e3-e90c25f48e8f.png\"},{\"action\":\"Click on the Twitter/X link that appears near the top of the search results, showing \\\"X, 尽早开始使用 / X\\\".\",\"code\":\"pyautogui.click(x=0.188, y=0.340)\",\"image\":\"714bf2ab-855d-4fac-8054-52360743ebea.png\"},{\"action\":\"Click on Elon Musk's profile picture to view his profile or expand his post about checking voting registration status.\",\"code\":\"pyautogui.click(x=0.280, y=0.483)\",\"image\":\"40a0d86c-1b70-43cd-88b1-59e3d69c7174.png\"},{\"action\":\"Click on the three-dot menu button (more options) next to Elon Musk's pinned tweet about the Trump rally in Pennsylvania to access sharing options.\",\"code\":\"pyautogui.click(x=0.596, y=0.807)\",\"image\":\"f624f3a0-a51c-4be6-aabe-b74b90a1f074.png\"},{\"action\":\"Click on the share button (upward arrow icon) in the dropdown menu to access additional sharing options.\",\"code\":\"pyautogui.click(x=0.632, y=0.800)\",\"image\":\"5524fccf-d7a4-4748-b5be-2f669634a9c6.png\"},{\"action\":\"Click on the \\\"Share post via...\\\" option in the sharing menu to reveal additional sharing methods.\",\"code\":\"pyautogui.click(x=0.600, y=0.845)\",\"image\":\"f41738fe-af58-409f-bff6-d5cfb439cbf6.png\"},{\"action\":\"Click on the \\\"Mail\\\" option in the system's sharing menu to share Elon Musk's post via email.\",\"code\":\"pyautogui.click(x=0.585, y=0.680)\",\"image\":\"0ddb4612-161f-489b-8eca-d08e9664c51b.png\"},{\"action\":\"Click in the empty area of the email body below the Twitter URL to position the cursor for typing additional message content.\",\"code\":\"pyautogui.click(x=0.307, y=0.396)\",\"image\":\"9a8b83e4-7b7c-4f69-9582-04bce51b01d7.png\"},{\"action\":\"Click the \\\"Send Anyway\\\" button in the warning dialog box to confirm sending the email without a subject.\",\"code\":\"pyautogui.click(x=0.430, y=0.522)\",\"image\":\"4017c6e6-3cb0-4b41-8b9d-1c8f02c61555.png\"},{\"action\":\"Terminate the process with success status as the task has been completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"f8b31757-e085-4e89-88d9-1fea7d681ac4.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241006214925_4979795f-9bf7-4c2e-9d6b-4a232f1ad522","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"627651e63d7b246048b2a8fdd95ab2537311ea2078bd98061633a3d009fbaf00"}
{"messages":[{"role":"user","content":"Use the photopea extension in Figma to adjust the view position by dragging, rotating, and zooming in/out."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Hand tool in the left toolbar to activate it for panning the canvas.\",\"code\":\"pyautogui.moveTo(x=0.138, y=0.723)\\npyautogui.dragTo(x=0.163, y=0.719, button='left')\",\"image\":\"1e9ea8f4-3a80-4155-9f5c-960afca37f92.png\"},{\"action\":\"Click and drag from the middle of the canvas downward slightly to pan the view position.\",\"code\":\"pyautogui.moveTo(x=0.485, y=0.548)\\npyautogui.dragTo(x=0.470, y=0.599, button='left')\",\"image\":\"19fff3b1-5dc8-41be-85b1-78cf37f6b7be.png\"},{\"action\":\"Click on the Hand tool in the left toolbar and drag slightly downward and to the right to continue adjusting the view position of the canvas.\",\"code\":\"pyautogui.moveTo(x=0.141, y=0.726)\\npyautogui.dragTo(x=0.160, y=0.750, button='left')\",\"image\":\"c99c897b-f8c4-4ad2-a441-6182c4368a25.png\"},{\"action\":\"Click on the empty canvas area below the \\\"HelloWorld\\\" text to prepare for further view adjustments.\",\"code\":\"pyautogui.click(x=0.404, y=0.700)\",\"image\":\"878a7336-b370-49f5-87bf-502ac4301870.png\"},{\"action\":\"Using the Hand tool that's already active, click in the empty canvas area below the \\\"HelloWorld\\\" text and drag slightly downward and to the right to continue adjusting the canvas view position.\",\"code\":\"pyautogui.moveTo(x=0.135, y=0.720)\\npyautogui.dragTo(x=0.143, y=0.741, button='left')\",\"image\":\"040f75b5-f244-4e3c-9f09-e34eb1ad0f6c.png\"},{\"action\":\"Click on the Rotate View tool in the left toolbar to activate it for rotating the canvas.\",\"code\":\"pyautogui.click(x=0.137, y=0.744)\",\"image\":\"13118d2a-c876-4395-a2f2-8f2fba8559e6.png\"},{\"action\":\"Double-click on the \\\"W\\\" in \\\"HelloWorld\\\" to set it as the pivot point for rotation.\",\"code\":\"pyautogui.doubleClick(x=0.291, y=0.476)\",\"image\":\"09ecc159-c5f4-4058-9cbd-924b4a37e5b6.png\"},{\"action\":\"Click on the \\\"Pixel to Pixel\\\" option in the view toolbar near the top of the screen to change the view mode to a 1:1 pixel representation.\",\"code\":\"pyautogui.click(x=0.208, y=0.193)\",\"image\":\"847dbe71-5881-4e1c-87d6-6d1bc1619b5d.png\"},{\"action\":\"Click on the \\\"Fit To Area\\\" option in the view toolbar near the top of the screen to adjust the zoom so the content fits within the visible area.\",\"code\":\"pyautogui.click(x=0.258, y=0.194)\",\"image\":\"091cced4-5335-4ca3-9f70-33ce7708948d.png\"},{\"action\":\"Terminate the task with success status as all required view adjustment capabilities have been demonstrated.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"bcb06d85-9cad-410e-b9ed-13c948311734.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241116113240_f26a9fe3-fadf-4346-881e-2ffbc285a7aa","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"46a3fe3b18b6d1fa9957def28f947840a2eb67705d24c5ee7a7595c090d5bb24"}
{"messages":[{"role":"user","content":"In Instagram, search for clouds, click on the first account you find, view his post, send him a message \"nice to meet you\", view details, click move to general, view hiden requests, return to the account homepage, view the first post and like it."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome browser icon in the dock to launch the web browser.\",\"code\":\"pyautogui.click(x=0.3085, y=0.9592)\",\"image\":\"525cd867-1e4a-4e14-ab4f-8a501596270a.png\"},{\"action\":\"Click on the search input field in the Instagram Search panel to activate it for typing.\",\"code\":\"pyautogui.click(x=0.1858, y=0.2207)\",\"image\":\"7e2b72a9-5ef2-4bdc-87bc-b38b2e55faf1.png\"},{\"action\":\"Type \\\"clouds\\\" in the search field and press Enter to execute the search.\",\"code\":\"pyautogui.write(message='clouds')\\npyautogui.press('enter')\",\"image\":\"2c445627-900e-4d8c-a167-ececd05d65f6.png\"},{\"action\":\"Click on the \\\"cloudstreet.sg\\\" account in the search results list to navigate to this cloud-related profile.\",\"code\":\"pyautogui.click(x=0.1152, y=0.2766)\",\"image\":\"91b233cd-5cac-4f54-90ba-afbc54cb14e5.png\"},{\"action\":\"Click on the X (close) button in the top-right corner of the Instagram Stories view to exit the story and return to the cloudstreet.sg profile page.\",\"code\":\"pyautogui.click(x=0.9779, y=0.1454)\",\"image\":\"9ccf392b-b273-4ad7-9643-5ef8944f7e40.png\"},{\"action\":\"Click on the \\\"Message\\\" button on the cloudstreet.sg Instagram profile to open the messaging interface.\",\"code\":\"pyautogui.click(x=0.6613, y=0.161)\",\"image\":\"9b4eafd8-2ba5-4e37-ba5e-39559a2e31b9.png\"},{\"action\":\"Click on the message input field at the bottom of the conversation panel to begin typing a message to Cloudstreet.\",\"code\":\"pyautogui.click(x=0.5367, y=0.9698)\",\"image\":\"8e7bcd42-3042-415f-a7e8-340afa0c4372.png\"},{\"action\":\"Click the send button (paper airplane icon) at the bottom right of the message input field to send the \\\"nice to meet you\\\" message to Cloudstreet.\",\"code\":\"pyautogui.click(x=0.9614, y=0.9632)\",\"image\":\"331d74cb-8248-41ed-98f0-c5764422b92a.png\"},{\"action\":\"Click on the information/details button (i icon) in the top-right corner of the conversation header to view details about the Cloudstreet account and conversation.\",\"code\":\"pyautogui.click(x=0.9876, y=0.1427)\",\"image\":\"028ef613-9aab-4877-b4ba-010ec7147f38.png\"},{\"action\":\"Click on the \\\"Move\\\" button next to \\\"Move to general\\\" in the details panel to move the conversation to the general tab.\",\"code\":\"pyautogui.click(x=0.9601, y=0.2116)\",\"image\":\"df86efc0-4570-448a-9721-ed792cbe472f.png\"},{\"action\":\"Click on the \\\"Requests\\\" tab in the Instagram message navigation bar to view hidden message requests.\",\"code\":\"pyautogui.click(x=0.2353, y=0.1983)\",\"image\":\"b574520f-d008-4327-a0d4-ae9a07f29f08.png\"},{\"action\":\"Click on the \\\"Hidden requests\\\" option in the message requests navigation menu to view hidden message requests.\",\"code\":\"pyautogui.click(x=0.2008, y=0.2235)\",\"image\":\"11e03fc6-43ba-461e-8ac2-f70c2799f24b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241122194315_a82b23de-6d17-484d-9fb8-2c4640f2d2e7","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"acbeb9d6ba35898db1837c261f72be7182886eab00776299d95441ed9ada702e"}
{"messages":[{"role":"user","content":"find availability colors of galaxy tab A9 Smart Book Cover."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Type \\\"sa\\\" in the Google search box, then press the down arrow key once to navigate through the suggestions, and press enter to select that suggestion.\",\"code\":\"pyautogui.write(message='sa')\\npyautogui.press('down')\\npyautogui.press('enter')\",\"image\":\"f6e9697c-da93-4c5c-8963-f6e3d331af22.png\"},{\"action\":\"Click on the \\\"Samsung Indonesia Official\\\" text link in the search results to navigate to Samsung's official Indonesian website.\",\"code\":\"pyautogui.click(x=0.3953, y=0.4753)\",\"image\":\"3b74ff75-c3cf-4d4c-b621-60a3c2a3bbc5.png\"},{\"action\":\"Click on the \\\"Accessories\\\" item in the top navigation menu to navigate to the accessories section of the Samsung Indonesia website.\",\"code\":\"pyautogui.click(x=0.5051, y=0.1667)\",\"image\":\"991bd7c8-3131-4138-88ef-3b024a5c48e3.png\"},{\"action\":\"Click on the search field labeled \\\"Cari aksesori\\\" (Search for accessories) to begin typing a search query.\",\"code\":\"pyautogui.click(x=0.4261, y=0.3919)\",\"image\":\"6fa07ed6-be1c-4f1a-a74f-d52fa641c4e5.png\"},{\"action\":\"Type \\\"tablet case\\\" in the search field, press Enter to search, and then press the down arrow key 16 times to navigate through the search results.\",\"code\":\"pyautogui.write(message='tablet case')\\npyautogui.press('enter')\\npyautogui.press('down', presses=16)\",\"image\":\"063af2d4-fe7e-45f4-a912-f2b56c86f12c.png\"},{\"action\":\"Click on the \\\"Lihat Lebih Detail\\\" (View More Details) link for the Galaxy Tab A9 Smart Book Cover to access the detailed product page.\",\"code\":\"pyautogui.click(x=0.5915, y=0.5846)\",\"image\":\"5c202677-a29b-4472-bcaf-654e359c5baa.png\"},{\"action\":\"Click on the black color option in the color selection section to view the Galaxy Tab A9 Smart Book Cover in black.\",\"code\":\"pyautogui.click(x=0.6025, y=0.8099)\",\"image\":\"85712404-a136-4a81-9c99-73bcd08ece51.png\"},{\"action\":\"Click on the white color option (the rightmost color circle) in the color selection area to view the Galaxy Tab A9 Smart Book Cover in white.\",\"code\":\"pyautogui.click(x=0.6332, y=0.8633)\",\"image\":\"c0473f1a-d4fe-4ff1-82fb-15e9578d3397.png\"},{\"action\":\"Click on the light blue color circle in the \\\"Pilih Warna Anda\\\" (Choose Your Color) section to select the light blue version of the Galaxy Tab A9 Smart Book Cover.\",\"code\":\"pyautogui.click(x=0.713, y=0.7904)\",\"image\":\"0986ee58-4493-4319-9727-f63aac40bbdf.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241102235413_8fc8ebe3-033c-4f27-8225-cad748861cb2","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"a396ac3b05fb2453c96917c6fe7f337e24e8939c4508b2b158dd51fd5b260277"}
{"messages":[{"role":"user","content":"Search for the Sun Yat sen Memorial Hall on Google Maps, save the location and view the route from IFC MALL to the Sun Yat sen Memorial Hall, as well as view the surrounding real-life photos"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Maps search result to access the Google Maps application.\",\"code\":\"pyautogui.click(x=0.142, y=0.320)\",\"image\":\"0b05bb25-bc4d-4b42-b8c1-50cfdbb4209b.png\"},{\"action\":\"Click on the Sun Yat-sen Memorial Hall location marker visible on the map in the water area south of Hong Kong Island.\",\"code\":\"pyautogui.click(x=0.649, y=0.576)\",\"image\":\"bce29cb9-972a-400c-8551-4ce4b6052954.png\"},{\"action\":\"Scroll down in the Sun Yat-sen Memorial Hall information panel to view more content including photos and additional options.\",\"code\":\"pyautogui.moveTo(x=0.656, y=0.624)\\npyautogui.scroll(10)\",\"image\":\"dfe5ffb1-0839-4324-8466-e914610e9300.png\"},{\"action\":\"Drag from the upper portion of the map downward to explore more of the surrounding area and potentially locate IFC Mall or find direction controls needed to set up the route from IFC Mall to Sun Yat-sen Memorial Hall.\",\"code\":\"pyautogui.moveTo(x=0.599, y=0.176)\\npyautogui.dragTo(x=0.559, y=0.899, button='left')\",\"image\":\"daf92b8a-da46-440d-9b2e-a53c0f017385.png\"},{\"action\":\"Click on the \\\"保存\\\" (Save) button in the information panel to save the Sun Yat-sen Memorial Hall location.\",\"code\":\"pyautogui.click(x=0.105, y=0.522)\",\"image\":\"3919dea3-d044-48c8-8da0-cfdeb93ba6f5.png\"},{\"action\":\"Click on \\\"收藏的地点\\\" (Saved places) option in the save menu to save the Sun Yat-sen Memorial Hall to my saved places.\",\"code\":\"pyautogui.click(x=0.124, y=0.555)\",\"image\":\"d53f5ee8-ed3e-4e6c-ad4c-dd2d85bb27a5.png\"},{\"action\":\"Click on the blue directions button (labeled \\\"路线\\\") in the action toolbar to begin planning a route to the Sun Yat-sen Memorial Hall.\",\"code\":\"pyautogui.click(x=0.068, y=0.427)\",\"image\":\"1736d91a-13eb-4b31-b8f3-5da9b2cbbf9c.png\"},{\"action\":\"Click on the International Finance Center (国际金融中心) information card on the right side of the screen to select it as the potential starting point for our route.\",\"code\":\"pyautogui.click(x=0.865, y=0.244)\",\"image\":\"23906be0-de02-4cdf-9dae-b83b8df70aeb.png\"},{\"action\":\"Click on the first route option \\\"途经紅棉路和漢皇巨道\\\" (via Cotton Tree Drive and Harcount Road) that shows a 7-minute travel time.\",\"code\":\"pyautogui.click(x=0.118, y=0.471)\",\"image\":\"c739e436-661e-4cc5-b28f-eade487ea3ac.png\"},{\"action\":\"Click on the Pegman (yellow Street View character) icon in the bottom right corner of the map to activate Street View mode and view real-life photos of the surrounding area.\",\"code\":\"pyautogui.click(x=0.983, y=0.930)\",\"image\":\"93980810-f75d-46df-b2bf-5dc85903f2d9.png\"},{\"action\":\"Click on the Sun Yat-sen Memorial Hall marker on the map to enter Street View and see real-life photos of the location.\",\"code\":\"pyautogui.click(x=0.533, y=0.431)\",\"image\":\"e6221f93-070e-48e9-bd5b-11b554126daf.png\"},{\"action\":\"Click on the Street View icon in the information panel's action bar (the 360-degree panorama view button) to view real-life photos around the Sun Yat-sen Memorial Hall.\",\"code\":\"pyautogui.click(x=0.515, y=0.431)\",\"image\":\"229c90de-5089-4e0f-92c9-80a0a8c39116.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241019222041_97546615-800f-4baa-a6db-fe733fbe9a89","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"d1fe9ce78b1d8a2ad4f722efac94feca833316fb9a414052581f19e941898839"}
{"messages":[{"role":"user","content":"Search for \"Limited Edition Sneakers\" and Review Product Details in Shoes Section using Shopee website"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search bar that currently contains \\\"Unilever Diskon s/d 70%\\\" to focus on it and prepare to enter a new search term.\",\"code\":\"pyautogui.click(x=0.320, y=0.217)\",\"image\":\"f486d1d4-c620-4b7a-86d0-2beb7f39dbf6.png\"},{\"action\":\"Type \\\"limited edition sneaker shoes\\\" in the search bar and press Enter to search for limited edition sneaker products.\",\"code\":\"pyautogui.write(message='limited edition sneaker shoes')\\npyautogui.press('enter')\",\"image\":\"f98c5583-3ccc-48d0-8514-cc6cc5fd7e9c.png\"},{\"action\":\"Click on the \\\"Harga: Tinggi ke Rendah\\\" (Price: High to Low) option in the dropdown menu to sort the search results by price in descending order.\",\"code\":\"pyautogui.click(x=0.561, y=0.909)\",\"image\":\"7fc067d9-f44a-4793-b708-9f75275edd0e.png\"},{\"action\":\"Click on the \\\"100% Ori\\\" checkbox in the left sidebar filter options to filter for authentic original products.\",\"code\":\"pyautogui.click(x=0.059, y=0.870)\",\"image\":\"904f68ba-f7e6-4254-8d64-10ed69421413.png\"},{\"action\":\"Scroll down to view more search results for limited edition sneakers.\",\"code\":\"pyautogui.moveTo(x=0.482, y=0.776)\\npyautogui.scroll(-10)\",\"image\":\"c32db9a8-7d3d-4458-8990-134e456f68c1.png\"},{\"action\":\"Click on the Columbia Men's Konos Trs Outdry Mid sneaker to view its detailed product information.\",\"code\":\"pyautogui.click(x=0.272, y=0.280)\",\"image\":\"10b7e2d2-827c-42f2-95ae-4f048824ae57.png\"},{\"action\":\"Scroll down on the product page to view more details about the Columbia Men's Konos Trs Outdry Mid sneaker.\",\"code\":\"pyautogui.moveTo(x=0.548, y=0.714)\\npyautogui.scroll(-10)\",\"image\":\"fbb19ed4-3c65-4eb4-9348-54d3ebb624d9.png\"},{\"action\":\"Click the \\\"Klaim\\\" button in the red label to claim the 5% discount voucher for the Columbia Men's Konos Trs Outdry Mid sneakers.\",\"code\":\"pyautogui.click(x=0.680, y=0.616)\",\"image\":\"0951c845-d7c2-4bff-9147-ba4092f26249.png\"},{\"action\":\"Scroll down at the current mouse position to view more product information below.\",\"code\":\"pyautogui.moveTo(x=0.329, y=0.811)\\npyautogui.scroll(10)\",\"image\":\"2b91e620-c5d8-45ce-96e4-c52c49ed11b1.png\"},{\"action\":\"Click the back button at the top left corner of the browser to return to the previous page of search results.\",\"code\":\"pyautogui.click(x=0.013, y=0.083)\",\"image\":\"d0991d81-b67b-4b94-8115-45931e363449.png\"},{\"action\":\"Scroll down at the location of the Limited Edition Adidas Originals Bali product to view more search results.\",\"code\":\"pyautogui.moveTo(x=0.529, y=0.725)\\npyautogui.scroll(10)\",\"image\":\"8abc6905-d7a6-4b80-8774-066f986bfcce.png\"},{\"action\":\"Click on the \\\"Terlaris\\\" (Best-Selling) sorting tab to re-sort the search results by popularity.\",\"code\":\"pyautogui.click(x=0.447, y=0.400)\",\"image\":\"016a1810-a80e-4ef3-b2ed-a517f488c086.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241127075455_543bc57e-1702-439e-a26c-8cc90e47bdd3","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"7d3cfa3c54e853b2dc264a023c85d583c0d0ad286222fc2d8ea5e6a73379945f"}
{"messages":[{"role":"user","content":"Rename saved form, change its text styling (change font, font weight), set background color to blue, and header photo from photo browser."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the menu button (three dots) next to the \\\"Formularz bez nazwy\\\" form to access form options.\",\"code\":\"pyautogui.click(x=0.2922, y=0.7204)\",\"image\":\"27582a3d-35ef-42bd-8978-e8a3094c877a.png\"},{\"action\":\"Click on \\\"Zmień nazwę\\\" (Change name) in the dropdown menu to rename the form.\",\"code\":\"pyautogui.click(x=0.275, y=0.7574)\",\"image\":\"4decafa0-54b8-42aa-86de-39770d674778.png\"},{\"action\":\"Type \\\"new name\\\" into the form name field and press Enter to confirm the change.\",\"code\":\"pyautogui.write(message='new name')\\npyautogui.press('enter')\",\"image\":\"b14f3f94-add2-4521-9251-97fb435eb6a7.png\"},{\"action\":\"Click on the \\\"new name\\\" form thumbnail in the \\\"Ostatnie formularze\\\" section to open it for editing.\",\"code\":\"pyautogui.click(x=0.2635, y=0.5972)\",\"image\":\"d6b32f5e-b25e-4ba0-9b2b-a6d787bab189.png\"},{\"action\":\"Click the X button in the top-right corner of the \\\"Style your text\\\" popup dialog to close it.\",\"code\":\"pyautogui.click(x=0.9083, y=0.2417)\",\"image\":\"be8da637-743e-4c6d-b4b8-f980443bccea.png\"},{\"action\":\"Click on the theme customization icon (paintbrush button) in the upper right area of the form editor to access form styling options.\",\"code\":\"pyautogui.click(x=0.7974, y=0.1704)\",\"image\":\"53bce912-9c49-410e-a60d-7e2a45c9d4ab.png\"},{\"action\":\"Click the \\\"OKAY, GOT IT!\\\" button in the \\\"Style your text\\\" dialog box to dismiss it and gain access to the styling options.\",\"code\":\"pyautogui.click(x=0.8771, y=0.362)\",\"image\":\"daf0ddbc-0195-4b13-97e7-a7ff4c60fe88.png\"},{\"action\":\"Click on the \\\"Roboto\\\" dropdown menu in the \\\"Nagłówek\\\" (Header) section of the styling panel to view and select a different font.\",\"code\":\"pyautogui.click(x=0.9016, y=0.3917)\",\"image\":\"2c44f1dd-c69f-4d23-8fbf-fa43f5988999.png\"},{\"action\":\"Click on the \\\"Impact\\\" font option in the font dropdown menu to change the header font from Roboto to Impact.\",\"code\":\"pyautogui.click(x=0.8979, y=0.6944)\",\"image\":\"ee6a8803-d023-4be6-8ee3-e0ff7301b477.png\"},{\"action\":\"Click on the \\\"Roboto\\\" dropdown menu in the \\\"Pytania\\\" (Questions) section to open the font selection options.\",\"code\":\"pyautogui.click(x=0.8927, y=0.4602)\",\"image\":\"2e9f51dc-c9fc-4960-863f-c384ec454be6.png\"},{\"action\":\"Click on the \\\"Lexend\\\" font option in the question font dropdown menu to change the font from Roboto to Lexend.\",\"code\":\"pyautogui.click(x=0.8906, y=0.6991)\",\"image\":\"be912758-4fda-4d27-a29d-2627d21c6ae8.png\"},{\"action\":\"Click on the light blue color option in the color palette to set the form's background color to blue.\",\"code\":\"pyautogui.click(x=0.9344, y=0.7815)\",\"image\":\"59c484cc-b836-4aae-91f9-b6711f824ac1.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241102022426_b33db0a4-fd78-4330-977b-b0887b30fbf5","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"54f8fc3c262d6444bc9255e785cc3ded78da4f6f91e2cf5b29334d0c4da7bc2c"}
{"messages":[{"role":"user","content":"Configure your Chrome browser to prevent pop-ups while researching healthy recipes on a food blog at the local café today."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the three-dot menu button in the top-right corner of the Chrome browser window to access browser settings.\",\"code\":\"pyautogui.click(x=0.9828, y=0.0762)\",\"image\":\"237d87ec-6367-4986-9d4f-06169ff6d6cc.png\"},{\"action\":\"Click on the search bar in the Chrome Web Store to begin searching for extensions.\",\"code\":\"pyautogui.click(x=0.4734, y=0.1794)\",\"image\":\"afb858eb-a47f-4d3a-9de7-2bc5344059d4.png\"},{\"action\":\"Type \\\"popup blocker\\\" in the search bar to search for popup blocking extensions in the Chrome Web Store.\",\"code\":\"pyautogui.write(message='popup blocker')\",\"image\":\"d1e9dc90-a4c0-4aef-b297-49c9f62a96bd.png\"},{\"action\":\"Click on the \\\"Popup Blocker (strict)\\\" extension in the search results to view its details page.\",\"code\":\"pyautogui.click(x=0.5059, y=0.2531)\",\"image\":\"2eae0aad-9684-43f0-b906-7fb9a4c16e36.png\"},{\"action\":\"Click the \\\"Add to Chrome\\\" button to install the Popup Blocker (strict) extension.\",\"code\":\"pyautogui.click(x=0.8199, y=0.3356)\",\"image\":\"ba84a3d1-c171-4819-b2e9-6de04fc4a4ab.png\"},{\"action\":\"Click the \\\"Add extension\\\" button in the confirmation dialog to complete the installation of the Popup Blocker (strict) extension.\",\"code\":\"pyautogui.click(x=0.5191, y=0.3)\",\"image\":\"c78000e4-71ee-4b54-ae4d-5f97cec4c9cd.png\"},{\"action\":\"Click on the Popup Blocker (strict) extension icon in the Chrome toolbar to check its status and settings.\",\"code\":\"pyautogui.click(x=0.9203, y=0.0756)\",\"image\":\"52798717-424e-43d6-b439-5bb0ea48dc38.png\"},{\"action\":\"Click on the pin icon next to the Popup Blocker (strict) extension in the Extensions dropdown menu to toggle its pinned status in the Chrome toolbar.\",\"code\":\"pyautogui.click(x=0.8582, y=0.3312)\",\"image\":\"59b4a0d2-e941-4140-a4a0-fff9bdd21dfc.png\"},{\"action\":\"Terminate the task with success status as the goal of configuring Chrome to prevent pop-ups has been successfully accomplished.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"0b69eaa7-a476-41dd-922b-9f3a0938eddb.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241224151123_564e4048-cb9b-40bf-8a03-126936c69e3d","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"cf4bfc445eb8fb7f644f727adca9272e131983ed083794ae09c8abd21d05830b"}
{"messages":[{"role":"user","content":"Open the Outlook web version in Google, select rules in mail under setting and add rule. The name of the rule is auto. The first condition is I am on the To line, the second condition is that the information subject includes wage, and the action is to move the mail to Inbox. The second action is Pin to top, select Run rule now, and save"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"All Bookmarks\\\" text in the upper right section of the browser to check available bookmarks.\",\"code\":\"pyautogui.click(x=0.9455, y=0.1033)\",\"image\":\"93a43434-a881-41b8-990b-1f2cd7ad1ff3.png\"},{\"action\":\"Click on the \\\"Mail - Ryan YanJie Outlook\\\" bookmark in the bookmarks dropdown menu to open Outlook.\",\"code\":\"pyautogui.click(x=0.8667, y=0.2433)\",\"image\":\"8bcfe1be-8dba-4d2a-b1a2-a62edeca0d47.png\"},{\"action\":\"Click on the settings/gear icon in the upper-right corner of the Outlook interface to access settings.\",\"code\":\"pyautogui.click(x=0.9236, y=0.1144)\",\"image\":\"4181dbaa-03fe-4bde-aec7-5cc4f163ad65.png\"},{\"action\":\"Click on \\\"Rules\\\" in the left navigation menu of the Settings panel.\",\"code\":\"pyautogui.click(x=0.2448, y=0.3128)\",\"image\":\"e84ddd2f-446b-47a6-bab7-ceede04ee39b.png\"},{\"action\":\"Click on the \\\"+ Add new rule\\\" button in the Rules interface to begin creating a new email rule.\",\"code\":\"pyautogui.click(x=0.8531, y=0.2339)\",\"image\":\"e4552df7-5488-4432-be0a-1d1e1dda032e.png\"},{\"action\":\"Click on the \\\"Name your rule\\\" text field to begin entering the rule name.\",\"code\":\"pyautogui.click(x=0.5757, y=0.2633)\",\"image\":\"9be2e0ca-7407-4cac-b438-6c5f96150186.png\"},{\"action\":\"Type \\\"auto\\\" in the \\\"Name your rule\\\" text field.\",\"code\":\"pyautogui.write(message='auto')\",\"image\":\"db52adf4-25ac-4920-9195-7fb997dc45a9.png\"},{\"action\":\"Click on the \\\"Select a condition\\\" dropdown to view the available condition options for the rule.\",\"code\":\"pyautogui.click(x=0.5521, y=0.3656)\",\"image\":\"b195cb2c-87b3-408c-8276-70051c0bcf9b.png\"},{\"action\":\"Click on the \\\"I'm on the To line\\\" option in the dropdown menu to select it as the first condition for the email rule.\",\"code\":\"pyautogui.click(x=0.6312, y=0.3339)\",\"image\":\"ffc5abe5-3d0f-4f11-95b6-7515634f6dcc.png\"},{\"action\":\"Click on the \\\"Add another condition\\\" link below the first condition to add the second condition for the email rule.\",\"code\":\"pyautogui.click(x=0.4667, y=0.4044)\",\"image\":\"2c668285-deae-4154-a8e0-f14b14bca368.png\"},{\"action\":\"Click on the \\\"Subject includes\\\" option in the dropdown menu to select it as the second condition for the email rule.\",\"code\":\"pyautogui.click(x=0.5514, y=0.4106)\",\"image\":\"989bc289-c0f5-4e4c-88bf-38806db09202.png\"},{\"action\":\"Click on the text field labeled \\\"And...\\\" to enter the specific text for the \\\"Subject includes\\\" condition.\",\"code\":\"pyautogui.click(x=0.5517, y=0.4156)\",\"image\":\"fa96b8f7-ebfd-4031-93ec-fc74210948ee.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241216185836_62fa74a9-818f-4425-8125-c7216e06dcad","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"d4382c060d759e3d40ecb20acf4a006ad79170507f6cd2627a3a499cb12f6d05"}
{"messages":[{"role":"user","content":"Mark the last message of the first message session in Prolific to be unread, go to the second message session and enter the study that participant is in, copy the id of the pending approval participant in that study, search for the history chat with the participant of that id to check the historical messages with him in the current study"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Messages\\\" button in the top navigation bar that has a notification badge showing \\\"1\\\".\",\"code\":\"pyautogui.click(x=0.1241, y=0.1001)\",\"image\":\"c7c77af1-8b47-44c4-8e3a-da8d33503043.png\"},{\"action\":\"Click on the options menu icon next to the first message conversation with participant ID \\\"099556546c32377e7a5d4e71e\\\" to access message options.\",\"code\":\"pyautogui.click(x=0.1329, y=0.2207)\",\"image\":\"0f8d3b60-b56f-4cf1-b914-95b56e835ce4.png\"},{\"action\":\"Click on the \\\"Mark as unread\\\" option in the dropdown menu that appears next to the first message conversation.\",\"code\":\"pyautogui.click(x=0.6218, y=0.8817)\",\"image\":\"b1567229-235f-4123-9617-d7aa9af66bb5.png\"},{\"action\":\"Click on the second message conversation in the list, which is the conversation with participant ID \\\"099fe7f9f93bfe5d4f78bcc5\\\" that has the preview text \\\"No problem sir\\\".\",\"code\":\"pyautogui.click(x=0.1021, y=0.2833)\",\"image\":\"74528e60-84bf-49df-9f5b-d38d9f98dad3.png\"},{\"action\":\"Click on the \\\"[Zosen] - [Formal Study]\\\" link in the status update to enter the study that the participant has completed.\",\"code\":\"pyautogui.click(x=0.6532, y=0.8717)\",\"image\":\"fb6577de-794c-4269-8050-cba9e40764e9.png\"},{\"action\":\"Select the participant's Prolific ID by clicking and dragging across the participant ID text in the table row.\",\"code\":\"pyautogui.moveTo(x=0.0882, y=0.3278)\\npyautogui.dragTo(x=0.141, y=0.3229, button='left')\",\"image\":\"ea07e054-2333-4d6a-8cbf-ff735735ac13.png\"},{\"action\":\"Press Command+C (⌘+C) to copy the selected Prolific participant ID to the clipboard.\",\"code\":\"pyautogui.hotkey(['cmd', 'c'])\",\"image\":\"e48135a3-8d37-4184-ac51-7630cf876e7e.png\"},{\"action\":\"Click on the \\\"Messages\\\" button in the blue navigation bar at the bottom right of the screen.\",\"code\":\"pyautogui.click(x=0.9264, y=0.9808)\",\"image\":\"f13fe173-6238-4366-a6e3-316849f689aa.png\"},{\"action\":\"Press Command+F (⌘+F) to open the browser's search function for finding text on the page.\",\"code\":\"pyautogui.hotkey(['cmd', 'f'])\",\"image\":\"cd65db5a-4fa7-4615-a481-6cd8e50ab76f.png\"},{\"action\":\"Paste the copied participant ID into the search box by pressing Command+V\",\"code\":\"pyautogui.hotkey(['cmd', 'v'])\",\"image\":\"31b56a71-4c8d-49c1-a6d8-42d6bc05e805.png\"},{\"action\":\"Click on the highlighted message entry for participant ID \\\"099fe7f9f93bfe5d4f78bcc5\\\" in the Messages panel to view the conversation history with this participant.\",\"code\":\"pyautogui.click(x=0.9566, y=0.2846)\",\"image\":\"fd3f3509-807b-462a-8809-ba2e64cf3e10.png\"},{\"action\":\"Terminate the task with success status as all required steps have been completed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"428f748c-e71b-4ae0-8ac6-b27ff625405b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241028201515_e29e5e63-39c0-4a98-98c5-13f7b0334fcc","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"d02fa94cdcaad87c113a9ca4b7c96a3eb8626b73339b8963793694d7f4bbb273"}
{"messages":[{"role":"user","content":"Help me calculate the bonus for each person as well as the total bonuses based on two strategies: 1. larger than threshold 7 million, and 2. more than the total average of totals, respectively. Otherwise, fill in the cell with text \"no bonus\". The commission rate is 5%."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the cell in column F containing the value $766,146, which shows Anthony Neil K.'s May sales figure.\",\"code\":\"pyautogui.click(x=0.5581, y=0.5737)\",\"image\":\"e8a34223-db6a-43a6-8de2-6b68f6d8d223.png\"},{\"action\":\"Click on the first cell under \\\"Bonus based on threshold\\\" (cell O3) to begin entering the formula for calculating bonuses based on the 7 million threshold.\",\"code\":\"pyautogui.click(x=0.6402, y=0.4711)\",\"image\":\"7a4b3e4c-5dae-4274-a077-326ac369acee.png\"},{\"action\":\"Type \\\"=if(\\\" in cell O3 to begin creating the conditional formula for the threshold-based bonus calculation.\",\"code\":\"pyautogui.write(message='=if(')\",\"image\":\"3c5da216-158a-4aca-ad89-0f4f76a9b095.png\"},{\"action\":\"Click on cell N3 to reference it in the formula instead of using the SUM function.\",\"code\":\"pyautogui.click(x=0.5623, y=0.4702)\",\"image\":\"195e6889-e589-49f6-b2f7-dbf8578ba2ea.png\"},{\"action\":\"Type \\\">7000000,\\\" to continue building the IF formula that will check if the sales total exceeds the 7 million threshold.\",\"code\":\"pyautogui.write(message='>7000000,')\",\"image\":\"e27a1d6e-a3a7-47f0-bff4-ef294785fe2d.png\"},{\"action\":\"Click on cell N3 that contains the value $8,392,490 to reference it in the formula for calculating the 5% commission if the threshold is met.\",\"code\":\"pyautogui.click(x=0.5532, y=0.4736)\",\"image\":\"ad38de23-a5aa-4f3e-85fe-9a906cd95d9f.png\"},{\"action\":\"Type \\\"*0.05,\\\"no bonus\\\")\\\" and press Enter to complete the IF formula.\",\"code\":\"pyautogui.write(message='*0.05,\\\"no bonus\\\")')\\npyautogui.press('enter')\",\"image\":\"93aaa1a7-702a-46fd-8a6d-e9cdd529e7a6.png\"},{\"action\":\"Click the green checkmark button in the AutoFill suggestion popup to accept the automatic fill suggestion for the bonus calculation formula.\",\"code\":\"pyautogui.click(x=0.7044, y=0.6656)\",\"image\":\"649b0a9b-cdfc-4c81-9236-9552eaa8e287.png\"},{\"action\":\"Click on cell O28 which contains the total bonus amount \\\"$9,042,991.71\\\" for the threshold-based bonus strategy.\",\"code\":\"pyautogui.click(x=0.6026, y=0.7597)\",\"image\":\"7cbcafe2-456c-4633-b2e9-18801cccfc77.png\"},{\"action\":\"Click on cell P3, which is the first data cell in column P, to begin implementing the second bonus strategy (calculating bonuses based on whether sales exceed the average of all totals).\",\"code\":\"pyautogui.click(x=0.5959, y=0.4121)\",\"image\":\"a7e3e14d-f194-4997-a9d7-919bb6f6f98e.png\"},{\"action\":\"Type \\\"=if(\\\" in cell P3 to begin creating the conditional formula for the average-based bonus calculation.\",\"code\":\"pyautogui.write(message='=if(')\",\"image\":\"efb6f07b-d6a1-41db-bdbb-33c78c333223.png\"},{\"action\":\"Click on cell N3 that contains the total sales value $8,392,490 to reference it in the formula for calculating the average-based bonus.\",\"code\":\"pyautogui.click(x=0.4909, y=0.4756)\",\"image\":\"e8264f57-265b-4c49-93ea-4f1e949c7586.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241018170605_64643077-5703-4dfc-a013-7eb352dfaf36","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8d2990e531c88a7b07677ab9e350a7f92b2c3471e65c6c179479fb77b9c614f3"}
{"messages":[{"role":"user","content":"Visit Tripadvisor on Google, type Hong Kong and find hotels located there. Click Hyatt Centric Victoria Harbour Hong Kong then \"the Royal Garden\". Scroll down to see the room features of it and capture the screen. Open Wechat and share the pictures with my friend 余庆. Send “not bad!” to him."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google search box in the center of the page to ensure it's active and ready for text input.\",\"code\":\"pyautogui.click(x=0.4292, y=0.4741)\",\"image\":\"0c837ab4-e9ba-47b5-88e7-1a0653f87156.png\"},{\"action\":\"Type \\\"trip\\\" in the search field, then press the right arrow key to complete the suggestion, and finally press Enter to execute the search.\",\"code\":\"pyautogui.write(message='trip')\\npyautogui.press('right')\\npyautogui.press('enter')\",\"image\":\"f93aeb10-a123-4444-a1b3-1f564752cf72.png\"},{\"action\":\"Click on the \\\"Tripadvisor®\\\" text in the first search result title to navigate to the official Tripadvisor website.\",\"code\":\"pyautogui.click(x=0.3047, y=0.3833)\",\"image\":\"f8d4568d-3152-461c-844c-f03cfc02ca75.png\"},{\"action\":\"Click on the search input field where the placeholder text \\\"Places to go, things to do, hotels...\\\" is displayed to begin entering our search query.\",\"code\":\"pyautogui.click(x=0.3333, y=0.4546)\",\"image\":\"43a505c8-fab2-4daa-9570-2d2e3090ee48.png\"},{\"action\":\"Type \\\"hong \\\" in the search field to begin searching for Hong Kong destinations.\",\"code\":\"pyautogui.write(message='hong ')\",\"image\":\"6be2cfda-4d9f-44c7-b0bf-f0d309e575d2.png\"},{\"action\":\"Click on the \\\"Hong Kong, China, Asia\\\" suggestion in the dropdown menu to select Hong Kong as the destination.\",\"code\":\"pyautogui.click(x=0.35, y=0.5231)\",\"image\":\"4db89368-ecc1-4123-977b-73a679bb705f.png\"},{\"action\":\"Click on the \\\"Hotels\\\" tab in the navigation menu to access the hotel listings for Hong Kong.\",\"code\":\"pyautogui.click(x=0.2365, y=0.1843)\",\"image\":\"2cc6a872-41c6-4010-ab64-011dd4cedad1.png\"},{\"action\":\"Click on the Check-out date field that displays \\\"Sat, 07 Dec\\\" to bring up the date selection calendar.\",\"code\":\"pyautogui.click(x=0.5609, y=0.4009)\",\"image\":\"d0741eb0-14cf-412d-b29a-d0fc040fff98.png\"},{\"action\":\"Click on the December 8 date in the calendar to select it as the check-out date.\",\"code\":\"pyautogui.click(x=0.4938, y=0.6852)\",\"image\":\"728072c6-d5da-489f-9bc1-411df2ea1497.png\"},{\"action\":\"Scroll down on the hotel search results page to see more of the hotel listings, particularly to fully reveal the Hyatt Centric Victoria Harbour Hong Kong listing.\",\"code\":\"pyautogui.moveTo(x=0.5818, y=0.4583)\\npyautogui.scroll(-4)\",\"image\":\"7ffff703-e1a9-425d-ad6d-810a7c8f1282.png\"},{\"action\":\"Click on the \\\"1. Hyatt Centric Victoria Harbour Hong Kong\\\" hotel name to view its detailed information page.\",\"code\":\"pyautogui.click(x=0.5833, y=0.4407)\",\"image\":\"7a52bc89-3bb6-4614-a790-9d982475342c.png\"},{\"action\":\"Click on the \\\"View hotel\\\" button for The Royal Garden hotel, which shows a price of S$243.\",\"code\":\"pyautogui.click(x=0.4552, y=0.5352)\",\"image\":\"c52dce1a-5dbc-4eb0-92c4-0f4eeb321095.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241110210935_24ff7fba-2eaa-41f8-b139-ca6ad59e7296","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"123d484389386baae73221382477e0f68e3efc3b613086c184f6d2f6b4809221"}
{"messages":[{"role":"user","content":"Open Telegram via Chrome browser and create a channel named TRAVELLERS CHANNEL and add members, ask members to check out a place we are to visit over the weekend, open Google Maps and copy a link of Tudor Water Sports and share it with the members, conduct a poll asking members if the place is suitable."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"telegram - Google Search\\\" suggestion in the dropdown menu to navigate to Google search results for Telegram.\",\"code\":\"pyautogui.click(x=0.1391, y=0.0583)\",\"image\":\"22b85e4c-ee41-4cde-ac39-5dcb987f3643.png\"},{\"action\":\"Click on the URL \\\"https://web.telegram.org\\\" displayed under the official Telegram listing at the top of the search results.\",\"code\":\"pyautogui.click(x=0.1641, y=0.3463)\",\"image\":\"66d053d1-3f97-45bc-8770-0e606d23e8b8.png\"},{\"action\":\"Click on the \\\"New Channel\\\" option in the popup menu to begin creating a new Telegram channel.\",\"code\":\"pyautogui.click(x=0.2208, y=0.9046)\",\"image\":\"e8b9b4ad-c923-4981-94cf-013549e0eeb8.png\"},{\"action\":\"Scroll down the contact list to see more potential members to add to the channel.\",\"code\":\"pyautogui.moveTo(x=0.199, y=0.7565)\\npyautogui.scroll(-4)\",\"image\":\"3864a05e-90eb-4956-a363-a40b31fba1a9.png\"},{\"action\":\"Click on the checkbox next to Dan Home's name to select him as a member to add to the \\\"TRAVELLERS CHANNEL\\\".\",\"code\":\"pyautogui.click(x=0.1943, y=0.7806)\",\"image\":\"9cd715ae-2731-43ac-a9be-a5d4f6d2fd1a.png\"},{\"action\":\"Click on the checkbox next to Joseph Omondi's name to select him as a member to add to the channel.\",\"code\":\"pyautogui.click(x=0.1615, y=0.9111)\",\"image\":\"0c61fa12-4fa9-4e7c-9d15-28b3d931ebcd.png\"},{\"action\":\"Click on the checkbox next to Travis's name to select him as a member to add to the TRAVELLERS CHANNEL.\",\"code\":\"pyautogui.click(x=0.1698, y=0.4398)\",\"image\":\"37efa6ea-aa8e-4afb-aa53-adacb076ccfa.png\"},{\"action\":\"Click on the checkbox next to Damian's name to select him as a member to add to the TRAVELLERS CHANNEL.\",\"code\":\"pyautogui.click(x=0.1682, y=0.7019)\",\"image\":\"565ed957-a25d-4806-9b34-b26e47a49662.png\"},{\"action\":\"Scroll down in the contacts list to view more potential members to add to the channel.\",\"code\":\"pyautogui.moveTo(x=0.176, y=0.7083)\\npyautogui.scroll(-5)\",\"image\":\"bbc6db6c-9f9d-4d1d-9051-52a7dc31fe52.png\"},{\"action\":\"Click on the checkbox next to Lave's name to select them as a member to add to the TRAVELLERS CHANNEL.\",\"code\":\"pyautogui.click(x=0.1328, y=0.8546)\",\"image\":\"b529ae51-d33c-4e62-9747-8efbff2bad57.png\"},{\"action\":\"Click on the checkbox next to Lemington's name to select them as a member to add to the TRAVELLERS CHANNEL.\",\"code\":\"pyautogui.click(x=0.2193, y=0.8991)\",\"image\":\"a9441592-6183-41e8-92b3-e62532caf73d.png\"},{\"action\":\"Click on the \\\"Channel name\\\" text field to position the cursor for editing the channel name.\",\"code\":\"pyautogui.click(x=0.0823, y=0.3648)\",\"image\":\"816fa3fd-bbd0-4c09-8ace-c7b5777b4b2b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241027162539_4d956635-b423-42c6-9c98-c94c64df6b9f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"e7e73c7d3df6a85762d892165fe8fb53d5a29d891fbf05c0f3b578e9fc7a6dcf"}
{"messages":[{"role":"user","content":"Search trip ideas on Kayak, ask KAYAK on cheap flights from Beijing to a relaxing place, find flights to Macau on Dec 3 and return on Dec 6, view the USD 138 deal and proceed to book"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Plan your trip\\\" option in the left navigation sidebar, which is represented by a suitcase icon.\",\"code\":\"pyautogui.click(x=0.0367, y=0.5044)\",\"image\":\"78db5eb9-837d-4039-96e3-00e5dc433800.png\"},{\"action\":\"Scroll down on the KAYAK Navigator page to view more content that might help with searching for flights from Beijing to Macau.\",\"code\":\"pyautogui.moveTo(x=0.4094, y=0.3494)\\npyautogui.scroll(-10)\",\"image\":\"b0670896-61da-4489-9329-652c939641b4.png\"},{\"action\":\"Click on the \\\"Ask away\\\" button to start interacting with KAYAK's recommendation system.\",\"code\":\"pyautogui.click(x=0.1492, y=0.7556)\",\"image\":\"41363737-6975-49e4-9d96-b2726e66eb7b.png\"},{\"action\":\"Click on \\\"Singapore\\\" in the \\\"Starting destination:\\\" field to change it to Beijing.\",\"code\":\"pyautogui.click(x=0.2852, y=0.6362)\",\"image\":\"e8d7e5ac-1462-4864-aac9-302d43d174e4.png\"},{\"action\":\"Type \\\"beijing\\\" in the Origin text field to set it as the starting location for the flight search.\",\"code\":\"pyautogui.write(message='beijing')\",\"image\":\"2a7ab300-7731-4f26-a384-2acdbbd49f90.png\"},{\"action\":\"Click on the \\\"Beijing, China\\\" suggestion in the location selection dropdown to select it as the departure city.\",\"code\":\"pyautogui.click(x=0.3723, y=0.6325)\",\"image\":\"dfe727b0-3575-402e-b446-092ad2c25798.png\"},{\"action\":\"Click the \\\"Set location\\\" button to confirm Beijing, China as the starting destination for the flight search.\",\"code\":\"pyautogui.click(x=0.3883, y=0.6681)\",\"image\":\"ffa2c7af-743d-4394-ba26-45a3f1959ecd.png\"},{\"action\":\"Click on the text input field labeled \\\"What are you looking for in your next trip?\\\" to begin entering the search query for flights from Beijing to Macau.\",\"code\":\"pyautogui.click(x=0.373, y=0.69)\",\"image\":\"aef8f62e-7350-4a67-9a9b-f8d98a78f470.png\"},{\"action\":\"Type \\\"Cheap flights to a relaxing place with beach\\\" in the search text field.\",\"code\":\"pyautogui.write(message='Cheap flights to a relaxing place with beach')\",\"image\":\"3417cc7b-1d40-435f-9507-f4255ee73c16.png\"},{\"action\":\"Click the \\\"Find flights\\\" button for the Macau destination listing.\",\"code\":\"pyautogui.click(x=0.4328, y=0.7662)\",\"image\":\"5af2015b-2eeb-4c31-ab37-771a251e9c22.png\"},{\"action\":\"Click on the date field showing \\\"Fri 11/22 - Mon 12/2\\\" to open the date selector and change the travel dates.\",\"code\":\"pyautogui.click(x=0.5883, y=0.1775)\",\"image\":\"8daf072a-54c7-4ddc-8aae-8263eab22ab2.png\"},{\"action\":\"Click on December 3 in the calendar to select it as the departure date.\",\"code\":\"pyautogui.click(x=0.5488, y=0.5681)\",\"image\":\"bb522905-6407-4089-8185-0cb35569b9b8.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241118173027_0e6bb338-37c1-4fde-8510-da4aa8d6a145","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8ad5acf984de6da2ec6b7b9b1496f5989a035bfdb7a896bbc4fbe209ac4cfae4"}
{"messages":[{"role":"user","content":"In photoshop, add gradient map to the black-white picture, and set the dark area in purple, the middle in blue and the light area in very light yellow"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"调整\\\" (Adjust) submenu option in the Image menu to see the available adjustment options.\",\"code\":\"pyautogui.click(x=0.1508, y=0.0619)\",\"image\":\"a9cf1672-07d4-48cf-b771-21148a2e4e88.png\"},{\"action\":\"Click on the \\\"渐变映射(G)...\\\" (Gradient Map) option in the adjustment submenu to open the gradient map dialog.\",\"code\":\"pyautogui.click(x=0.275, y=0.3444)\",\"image\":\"a3554467-de71-46fd-a73e-33423cc0695b.png\"},{\"action\":\"Click on the gradient preview bar in the Gradient Map dialog box to open the Gradient Editor, where we can customize the colors for different tonal ranges.\",\"code\":\"pyautogui.click(x=0.4453, y=0.1688)\",\"image\":\"cf4c6c4a-1cc6-432c-b441-5854850f1232.png\"},{\"action\":\"Drag the color stop on the gradient preview bar in the Gradient Editor from its current position to a new position, adjusting where the color transition occurs in the gradient.\",\"code\":\"pyautogui.moveTo(x=0.6512, y=0.4256)\\npyautogui.dragTo(x=0.6949, y=0.3275, button='left')\",\"image\":\"a0ce858e-8737-4cef-8d26-99101ef62716.png\"},{\"action\":\"Double-click on the color stop in the gradient preview bar to open the color picker and modify its color value.\",\"code\":\"pyautogui.doubleClick(x=0.6258, y=0.6706)\",\"image\":\"04dcf04e-cc92-4ea4-92f2-751ad3d7d469.png\"},{\"action\":\"Drag the color selector in the color picker's vertical color slider from its current position downward to select a different color for the gradient stop.\",\"code\":\"pyautogui.moveTo(x=0.427, y=0.8037)\\npyautogui.dragTo(x=0.4371, y=0.6538, button='left')\",\"image\":\"73580da3-d459-4f45-8142-98429a770636.png\"},{\"action\":\"Click in the main color selection area of the color picker to fine-tune the purple color for the dark areas of the gradient map.\",\"code\":\"pyautogui.click(x=0.4105, y=0.705)\",\"image\":\"35ed0e8c-e535-4b2e-a46f-d01f83a252d8.png\"},{\"action\":\"Click the \\\"确定\\\" (OK) button in the color picker dialog to confirm the selected purple color for the gradient stop.\",\"code\":\"pyautogui.click(x=0.5418, y=0.5806)\",\"image\":\"89a57b91-8e7e-439a-aa77-e57a64017514.png\"},{\"action\":\"Double-click on the middle area of the gradient preview bar in the Gradient Editor to create a new color stop for the middle tones.\",\"code\":\"pyautogui.doubleClick(x=0.6691, y=0.66)\",\"image\":\"3e52368a-0808-4834-a049-191707dc9d81.png\"},{\"action\":\"Click on the blue section of the vertical color slider in the color picker to select a blue color range for the middle gradient stop.\",\"code\":\"pyautogui.click(x=0.4406, y=0.68)\",\"image\":\"fa6ba27c-8985-464d-b213-d30a41a79d89.png\"},{\"action\":\"Click in the main color selection area of the color picker to select a specific shade of blue for the middle gradient stop.\",\"code\":\"pyautogui.click(x=0.4285, y=0.6756)\",\"image\":\"68a8e156-beca-4a6f-8015-b88975bc79af.png\"},{\"action\":\"Drag the blue color stop in the gradient preview bar slightly to the left to adjust its position in the tonal range, moving it from its current position toward the darker end of the gradient.\",\"code\":\"pyautogui.moveTo(x=0.4105, y=0.7075)\\npyautogui.dragTo(x=0.4008, y=0.675, button='left')\",\"image\":\"900546a7-3a2a-4487-b028-6442b583fcc8.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241026143327_158df6dc-bf0a-4831-a0ac-9ea16b2fe6d2","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"7a6c578fd239a804b8ba41e6cb33300cdf7b9e2b1c17671ba2e6a2ef4f752b36"}
{"messages":[{"role":"user","content":"Using the Fig3D extension in FigJam, draw a circle and a triangle shape, select the triangle and open the Fig3D extension, adjust the depth and apply."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search bar in the top navigation to begin searching for the Fig3D extension.\",\"code\":\"pyautogui.click(x=0.859, y=0.143)\",\"image\":\"7e5bfd28-063b-4c69-851e-c4234ab13d53.png\"},{\"action\":\"Click on the Fig3D plugin entry in the search results sidebar, which shows colored spheres with a \\\"3d\\\" label and is created by Palash E, John I and 1 other.\",\"code\":\"pyautogui.click(x=0.873, y=0.543)\",\"image\":\"0b599b33-6d3e-4e03-ae9c-df04ea34d491.png\"},{\"action\":\"Click the blue \\\"Open in...\\\" button to launch the Fig3D plugin.\",\"code\":\"pyautogui.click(x=0.206, y=0.346)\",\"image\":\"997ca5fd-c79f-4a1f-9abc-e9b00e5882cf.png\"},{\"action\":\"Click on the \\\"Magic Cut - remove backgrou...\\\" file from the dropdown menu to open the Fig3D plugin in this file.\",\"code\":\"pyautogui.click(x=0.209, y=0.432)\",\"image\":\"6f224a55-c1d3-45e8-b712-1f7c182553e8.png\"},{\"action\":\"Click on the circle drawing tool in the bottom toolbar to begin creating a circle shape.\",\"code\":\"pyautogui.click(x=0.458, y=0.930)\",\"image\":\"7da3f56e-1d1d-47bd-9c3e-9333da13a7d4.png\"},{\"action\":\"Using the currently selected polygon tool, click on the empty area in the middle-right portion of the canvas and drag diagonally down and to the right to begin creating a triangle shape.\",\"code\":\"pyautogui.moveTo(x=0.581, y=0.456)\\npyautogui.dragTo(x=0.682, y=0.660, button='left')\",\"image\":\"b4d0bb47-11d4-4b4a-bbb9-01f9b2ce4b42.png\"},{\"action\":\"Click on the 3D cube icon in the bottom toolbar to open the Fig3D extension for the selected triangle.\",\"code\":\"pyautogui.click(x=0.564, y=0.931)\",\"image\":\"6408943c-fdb4-4b5d-83ef-c1de44cf9508.png\"},{\"action\":\"Click on the \\\"Plugins & Widgets\\\" tab in the dropdown menu to access the Fig3D extension controls.\",\"code\":\"pyautogui.click(x=0.464, y=0.625)\",\"image\":\"96317acf-75e3-4a5c-bb26-fe170287a983.png\"},{\"action\":\"Click on the Fig3D plugin in the plugins list to access the 3D controls for the selected triangle.\",\"code\":\"pyautogui.click(x=0.434, y=0.866)\",\"image\":\"de1cf351-1ad6-4b49-b33a-fde5e6a9a8c9.png\"},{\"action\":\"Drag the Exposure slider in the Fig3D panel slightly to adjust the lighting of the 3D triangle.\",\"code\":\"pyautogui.moveTo(x=0.432, y=0.398)\\npyautogui.dragTo(x=0.436, y=0.397, button='left')\",\"image\":\"0f2ee7fa-3861-4472-907a-91e2050b747b.png\"},{\"action\":\"Drag the Height slider in the Fig3D panel slightly to the right to increase the depth/extrusion of the 3D triangle.\",\"code\":\"pyautogui.moveTo(x=0.418, y=0.434)\\npyautogui.dragTo(x=0.424, y=0.435, button='left')\",\"image\":\"4616f813-fa29-4e73-b0fb-b32556562103.png\"},{\"action\":\"Continue dragging the Height slider in the Fig3D panel to the right to increase the depth/extrusion of the 3D triangle.\",\"code\":\"pyautogui.moveTo(x=0.244, y=0.594)\\npyautogui.dragTo(x=0.272, y=0.621, button='left')\",\"image\":\"b0c8ff36-7ded-4103-bc24-03b00dbf8e3b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241209165822_010fc5f7-fb43-470a-be96-5c4d94109c5c","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"6d8844f315f39130ff52d672eebec2dcb9c524294be8d951e8e615bc48b9b091"}
{"messages":[{"role":"user","content":"In Photoshop, drag the red background into the canvas, convert it to a smart object, rasterize the layer, enter the text \"BREAK\", rotate the text clockwise by 90°, zoom in, drag it to the middle of the canvas, rasterize the text, use the rectangular marquee tool to draw a rectangular selection on the left half of the text, cut the left half of the text, move the left half of the text downwards, and achieve a dislocation effect"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Right-click on Layer 1 in the Layers panel to open the context menu with layer options.\",\"code\":\"pyautogui.rightClick(x=0.878, y=0.759)\",\"image\":\"babd3459-a4ee-44ac-82e4-84cc2aedaff2.png\"},{\"action\":\"Click on the \\\"Convert to Smart Object\\\" option in the context menu to convert Layer 1 to a Smart Object.\",\"code\":\"pyautogui.click(x=0.781, y=0.344)\",\"image\":\"61715d36-8626-4cd4-8759-36b6bedead96.png\"},{\"action\":\"Right-click on Layer 1 in the Layers panel to open the context menu with rasterizing options.\",\"code\":\"pyautogui.rightClick(x=0.887, y=0.765)\",\"image\":\"cc145091-ac6e-44d3-b61c-0d64b436eeb6.png\"},{\"action\":\"Click on \\\"Rasterize Layer\\\" in the context menu to convert the Smart Object back to a regular rasterized layer.\",\"code\":\"pyautogui.click(x=0.769, y=0.426)\",\"image\":\"e038762e-6070-4322-b9f9-82def373b260.png\"},{\"action\":\"Click on the Text Tool (T) in the left toolbar to select it for adding text to the document.\",\"code\":\"pyautogui.click(x=0.012, y=0.541)\",\"image\":\"e2f03a98-af34-44b5-badd-9a06abf45972.png\"},{\"action\":\"Click on the center of the red rectangular background to create a text insertion point for typing.\",\"code\":\"pyautogui.click(x=0.358, y=0.491)\",\"image\":\"301ea585-8718-495f-b9e3-cd63b401515e.png\"},{\"action\":\"Type \\\"BREAK\\\" at the text cursor position to add the text to the document.\",\"code\":\"pyautogui.write(message='BREAK')\",\"image\":\"a62d00dd-45ad-41de-b620-1ddf161b75f8.png\"},{\"action\":\"Click on the Move Tool located at the top of the left toolbar in Photoshop to select it for manipulating the text layer.\",\"code\":\"pyautogui.click(x=0.012, y=0.106)\",\"image\":\"894c9e56-41f5-4f18-bcb6-246a0e15011d.png\"},{\"action\":\"Press Ctrl+T to activate the Free Transform mode, which will allow you to rotate the text.\",\"code\":\"pyautogui.hotkey(['ctrl', 't'])\",\"image\":\"4ac77f09-1ed1-4910-ac49-2ce439732ffd.png\"},{\"action\":\"Drag from outside the transform box near the \\\"BREAK\\\" text in a clockwise direction to rotate the text 90 degrees clockwise.\",\"code\":\"pyautogui.moveTo(x=0.341, y=0.439)\\npyautogui.dragTo(x=0.434, y=0.397, button='left')\",\"image\":\"cb640a27-726d-4708-be06-02539f4553b8.png\"},{\"action\":\"Drag the transform control handle near the top of the rotated \\\"BREAK\\\" text slightly to the left to reposition the text more centrally on the red background.\",\"code\":\"pyautogui.moveTo(x=0.416, y=0.444)\\npyautogui.dragTo(x=0.389, y=0.474, button='left')\",\"image\":\"a1ffeecd-06f0-4d7a-9b43-ad9b3ccfe061.png\"},{\"action\":\"Click the checkmark button in the options bar to confirm and apply the Free Transform operation.\",\"code\":\"pyautogui.click(x=0.374, y=0.425)\",\"image\":\"e246d337-5d1c-473b-b0d2-3c9d24ce545c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241027001800_3a3e5696-5701-4941-aa45-50f20a0af64c","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"3e21118c5cbc268060e8cb73e761df3a86d79afe2c73587fa0751e44df27ae42"}
{"messages":[{"role":"user","content":"Opens LinkedIn via Google shortcut, selects for business, selects group, opens new group, sets name and description, selects profile, specifies industry, selects group location, presses create group button."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the LinkedIn shortcut located on the right side of the Google homepage shortcuts row.\",\"code\":\"pyautogui.click(x=0.8276, y=0.5861)\",\"image\":\"c8f96bd6-fde6-4fbc-8d6a-fa0dbc94c940.png\"},{\"action\":\"Click on the \\\"For Business\\\" option in the top navigation bar of the LinkedIn interface.\",\"code\":\"pyautogui.click(x=0.7901, y=0.263)\",\"image\":\"ffcd3f4f-7721-4cd8-9091-997d737a85a9.png\"},{\"action\":\"Click on the \\\"Groups\\\" option in the \\\"My Apps\\\" dropdown menu to access LinkedIn's group functionality.\",\"code\":\"pyautogui.click(x=0.4609, y=0.5194)\",\"image\":\"64b92f23-4b45-4b63-a123-9b6d7890591a.png\"},{\"action\":\"Click on the \\\"Create group\\\" button to begin creating a new LinkedIn group.\",\"code\":\"pyautogui.click(x=0.5979, y=0.3426)\",\"image\":\"2d8ade61-f0d2-402b-9de3-089d01ecf442.png\"},{\"action\":\"Click in the \\\"Group name\\\" text field that contains \\\"Inspiring Entrepreneurs in DC\\\" to position the cursor for editing the group name.\",\"code\":\"pyautogui.click(x=0.4495, y=0.5602)\",\"image\":\"c1171335-2741-447b-9071-3be6a30e686a.png\"},{\"action\":\"Type \\\"test\\\" in the description field to provide a basic description for the group being created.\",\"code\":\"pyautogui.write(message='test')\",\"image\":\"b22639c4-62b5-43d0-aa96-9f970d99e0af.png\"},{\"action\":\"Click on the pencil icon (edit button) next to the group profile picture placeholder to change the group's profile image.\",\"code\":\"pyautogui.click(x=0.3198, y=0.4954)\",\"image\":\"ff2f0c71-22a5-4a74-8d97-402cfb87d165.png\"},{\"action\":\"Click on the \\\"Upload logo\\\" button in the file upload interface to select a profile picture for the LinkedIn group.\",\"code\":\"pyautogui.click(x=0.3547, y=0.575)\",\"image\":\"d3edf5d8-4ee6-45f0-9925-ac04dc70ebe1.png\"},{\"action\":\"Click on the image with the light blue background in the file explorer to select it as the LinkedIn group logo.\",\"code\":\"pyautogui.click(x=0.2896, y=0.4)\",\"image\":\"ef1ca50f-4843-4172-85bd-e01f95698703.png\"},{\"action\":\"Click the \\\"Open\\\" button to confirm the selection of the anime character image as the LinkedIn group logo.\",\"code\":\"pyautogui.click(x=0.6193, y=0.5056)\",\"image\":\"c8d4c404-a319-47c0-ab23-2a94648f5725.png\"},{\"action\":\"Scroll down in the image editor dialog to view additional options that may be below the currently visible area.\",\"code\":\"pyautogui.moveTo(x=0.701, y=0.5657)\\npyautogui.scroll(-7)\",\"image\":\"ac2d4b18-8552-40da-a7af-7ac454459e91.png\"},{\"action\":\"Drag the anime character image downward within the image editor to position it properly in the crop area.\",\"code\":\"pyautogui.moveTo(x=0.7615, y=0.5037)\\npyautogui.dragTo(x=0.7609, y=0.6593, button='left')\",\"image\":\"d127d7cb-e60d-4a19-84a2-db951e7da270.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241020193154_19050d17-fd2c-4438-a18f-7976604f65ef","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"e3985eba7f5eaaf5d1834d8dd97a53cb115d06995fa6dee92898499bcf37bb0d"}
{"messages":[{"role":"user","content":"Create a new project titled 'purchase domain name', assign it to Xiaochuan, set the priority to High, and mark it as 'Blocked by' website redesign"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"+ New\\\" button that appears below the \\\"Website redesign\\\" project in the Planning section to add a new project.\",\"code\":\"pyautogui.click(x=0.1149, y=0.3969)\",\"image\":\"a77cace2-ff92-4c73-bf0e-5b5f31479e5f.png\"},{\"action\":\"Press backspace to delete the selected \\\"New Project\\\" text, type \\\"Purchase domain name\\\" in its place, and then press Enter to confirm the new project name.\",\"code\":\"pyautogui.press('backspace')\\npyautogui.write(message='Purchase domain name')\\npyautogui.press('enter')\",\"image\":\"863855cf-7391-4020-a280-c590db3195ac.png\"},{\"action\":\"Click on the Owner field (currently showing Nate Martins) for the \\\"Purchase domain\\\" project to change the assigned person.\",\"code\":\"pyautogui.click(x=0.3405, y=0.3867)\",\"image\":\"15f9fa45-7a2d-4930-a014-f3a8b4a393a9.png\"},{\"action\":\"Type \\\"Xiaochuan\\\" in the search field to find and assign this person as the owner of the \\\"Purchase domain name\\\" project.\",\"code\":\"pyautogui.write(message='Xiaochuan')\",\"image\":\"d9ea558a-839b-4ad8-a5d4-11da035ec815.png\"},{\"action\":\"Click on the \\\"Xiaochuan (You)\\\" option in the dropdown menu to select Xiaochuan as the project owner.\",\"code\":\"pyautogui.click(x=0.3637, y=0.4647)\",\"image\":\"dd930663-3d65-4f9c-920c-2ead6ee568ec.png\"},{\"action\":\"Click on the empty Priority cell in the \\\"Purchase domain name\\\" project row to open the priority selection dropdown.\",\"code\":\"pyautogui.click(x=0.5498, y=0.4023)\",\"image\":\"62231d91-adef-4540-9b80-897c81abaadb.png\"},{\"action\":\"Click in the center of the Priority cell for the \\\"Purchase domain name\\\" project to bring up or activate the priority selection dropdown menu.\",\"code\":\"pyautogui.click(x=0.5097, y=0.4023)\",\"image\":\"524a3d16-eff9-4208-b12f-cd78ff053cf4.png\"},{\"action\":\"Click on the \\\"31\\\" date in the October 2024 calendar to select it as the due date for the \\\"Purchase domain name\\\" project.\",\"code\":\"pyautogui.click(x=0.5078, y=0.6733)\",\"image\":\"d420a1d6-5650-4399-8609-0295bc196eb4.png\"},{\"action\":\"Click in the empty space outside of the calendar popup to close it and confirm October 31, 2024 as the due date for the \\\"Purchase domain name\\\" project.\",\"code\":\"pyautogui.click(x=0.6241, y=0.4699)\",\"image\":\"34c98a61-132f-4456-8263-27f0ee7504ec.png\"},{\"action\":\"Click on the empty Priority cell for the \\\"Purchase domain name\\\" project to open the priority selection options.\",\"code\":\"pyautogui.click(x=0.6074, y=0.3958)\",\"image\":\"54054039-fd87-44db-ad94-35d010db9b2e.png\"},{\"action\":\"Click on the \\\"High\\\" option in the priority dropdown menu to set the priority level for the \\\"Purchase domain name\\\" project.\",\"code\":\"pyautogui.click(x=0.6182, y=0.5161)\",\"image\":\"12055a06-8ff1-416e-ac31-b3da2725181c.png\"},{\"action\":\"Click on the empty cell in the Blocked By column for the \\\"Purchase domain name\\\" project to open the selection dropdown.\",\"code\":\"pyautogui.click(x=0.8155, y=0.4247)\",\"image\":\"1e59cc50-1867-499a-8741-58e83c3bb49e.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241013153119_cb624f0c-bc92-45e2-a753-b64e61629218","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"a0577e4543dc86b437d60881ae655d4e498ed42ca79e35a191ffb3bfd8bd6b67"}
{"messages":[{"role":"user","content":"In Google Maps, search for New York NY. Change layers to Terrain. Click on Area 53 - Adventure Park, view Tickets, click on 3 more options. Go to the Reviews section, click on the zip line tag, read the first review. Go to Overview and save it as Want to go. Search for Directions from Barclays Center to the park, and choose the best way."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search field in Google Maps to begin typing the search query for New York NY.\",\"code\":\"pyautogui.click(x=0.2126, y=0.1584)\",\"image\":\"0023cdcb-d089-4aea-8ed4-b98594db01f2.png\"},{\"action\":\"Type \\\"newyork\\\" in the search field and press Enter to search for New York.\",\"code\":\"pyautogui.write(message='newyork')\\npyautogui.press('enter')\",\"image\":\"1babbeb3-6df7-47a9-943b-bb484b943aeb.png\"},{\"action\":\"Click on \\\"New York NY\\\" in the search results dropdown to select it and navigate to New York on the map.\",\"code\":\"pyautogui.click(x=0.1342, y=0.2157)\",\"image\":\"8700e8aa-5984-49be-a77b-a5a88d15b371.png\"},{\"action\":\"Click on the Layers button in the bottom right corner of the map to access map view options.\",\"code\":\"pyautogui.click(x=0.37, y=0.8301)\",\"image\":\"8ac3d7b4-e624-4652-95c4-708429a66590.png\"},{\"action\":\"Click on the Terrain option in the layers panel to change the map view to Terrain mode.\",\"code\":\"pyautogui.click(x=0.4302, y=0.8556)\",\"image\":\"b0e635a3-10de-4e19-b706-202db5d551b8.png\"},{\"action\":\"Click on the Area 53 - Adventure Park label on the map to access its information panel.\",\"code\":\"pyautogui.click(x=0.6327, y=0.4843)\",\"image\":\"18b1e5e5-c0e6-43ef-8e4d-c699e8411084.png\"},{\"action\":\"Click on the \\\"Tickets\\\" tab in the Area53 - Adventure Park information panel to view ticket information.\",\"code\":\"pyautogui.click(x=0.1501, y=0.5072)\",\"image\":\"5c81792a-79ae-42c4-a1b5-d199895ddf1a.png\"},{\"action\":\"Click on the \\\"3 more options\\\" dropdown button below the Area53 - Adventure Park official site ticket option.\",\"code\":\"pyautogui.click(x=0.1618, y=0.4036)\",\"image\":\"380f55cb-17fd-4180-a956-1c144dcffa08.png\"},{\"action\":\"Click on the \\\"Reviews\\\" tab in the Area53 - Adventure Park information panel to navigate to the reviews section.\",\"code\":\"pyautogui.click(x=0.2388, y=0.2399)\",\"image\":\"39952f5b-a1eb-468d-91a6-4c3af0319f4e.png\"},{\"action\":\"Click on the \\\"zip line\\\" tag in the review filters to see reviews mentioning zip lines at Area53 - Adventure Park.\",\"code\":\"pyautogui.click(x=0.1499, y=0.584)\",\"image\":\"00292948-ab97-45ec-9a53-75e8d5d483d8.png\"},{\"action\":\"Click on the \\\"More\\\" button in Monee Mccrimmon's review to expand and read the full review text.\",\"code\":\"pyautogui.click(x=0.2596, y=0.444)\",\"image\":\"6b3445fc-be32-4df7-97f8-9ebe03679cbc.png\"},{\"action\":\"Click on the \\\"Overview\\\" tab in the Area53 - Adventure Park information panel to navigate to the general information section.\",\"code\":\"pyautogui.click(x=0.096, y=0.2097)\",\"image\":\"dff4cecd-134d-4d5a-b18f-1a7e1dd9005d.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241205111442_b76568d0-d405-4a20-96ca-60797bfcc58b","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"0b483ff831a8e93ae1253fd390c5e3ab63e22eeaa61f8164ff96b591fbecb699"}
{"messages":[{"role":"user","content":"Apply Tilt-Shift using Blur Gallery with Tilt-Shift Filter on Image Layer"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Photoshop (Ps) icon in the Windows taskbar to launch or bring Photoshop into focus.\",\"code\":\"pyautogui.click(x=0.177, y=0.970)\",\"image\":\"eb4bbc95-b18e-4e86-918d-453259d5e649.png\"},{\"action\":\"Click on the Giant_Panda_2004-03-2.jpg image thumbnail in the Recent files grid to open it in Photoshop.\",\"code\":\"pyautogui.click(x=0.295, y=0.345)\",\"image\":\"7c7eab2c-ecc8-4240-afa8-ecd2294fbfd3.png\"},{\"action\":\"Click on the \\\"Filter\\\" option in the top menu bar of Photoshop to open the Filter dropdown menu.\",\"code\":\"pyautogui.click(x=0.213, y=0.003)\",\"image\":\"118afa6d-6000-408e-8c87-d5ba3af964de.png\"},{\"action\":\"Click on the \\\"Tilt-Shift...\\\" option in the Blur Gallery submenu to open the Tilt-Shift filter interface.\",\"code\":\"pyautogui.click(x=0.432, y=0.316)\",\"image\":\"3b2bf3b5-5c2c-4c10-ae6a-2558f6c92203.png\"},{\"action\":\"Drag the tilt-shift control point from its current position on the middle horizontal line to slightly adjust the focus area of the tilt-shift effect on the panda image.\",\"code\":\"pyautogui.moveTo(x=0.420, y=0.528)\\npyautogui.dragTo(x=0.398, y=0.541, button='left')\",\"image\":\"14a449ad-aac7-4a00-a646-f4cb237b4ed1.png\"},{\"action\":\"Drag the tilt-shift control point from its current position near the right side of the panda's face to a position further left, refining the position of the focus area of the tilt-shift effect.\",\"code\":\"pyautogui.moveTo(x=0.409, y=0.513)\\npyautogui.dragTo(x=0.297, y=0.508, button='left')\",\"image\":\"61b3c205-ae1f-4e0a-ad58-8380c5686818.png\"},{\"action\":\"Drag the small control point located above the main circular control to a new position slightly lower and to the right to adjust the tilt-shift blur transition area.\",\"code\":\"pyautogui.moveTo(x=0.296, y=0.447)\\npyautogui.dragTo(x=0.334, y=0.488, button='left')\",\"image\":\"6b81b007-06fd-4981-b194-f828809b2a95.png\"},{\"action\":\"Drag the small control point positioned above the main tilt-shift control circle to a slightly different position to refine the blur transition area.\",\"code\":\"pyautogui.moveTo(x=0.326, y=0.468)\\npyautogui.dragTo(x=0.344, y=0.453, button='left')\",\"image\":\"c3ad1c9c-8123-4a3f-998a-4744487e253a.png\"},{\"action\":\"Drag the small tilt-shift control point located to the left of the panda's face slightly down and to the right to adjust the blur transition area.\",\"code\":\"pyautogui.moveTo(x=0.250, y=0.496)\\npyautogui.dragTo(x=0.244, y=0.525, button='left')\",\"image\":\"99fbffea-607d-48e5-a1a0-324e6f7a6759.png\"},{\"action\":\"Drag the small tilt-shift control point on the left side of the panda image slightly to the right and upward to further adjust the blur transition area.\",\"code\":\"pyautogui.moveTo(x=0.162, y=0.552)\\npyautogui.dragTo(x=0.210, y=0.534, button='left')\",\"image\":\"072415db-5056-4ca6-8ee2-212a53b87547.png\"},{\"action\":\"Drag the small tilt-shift control point in the upper right area (near the dark area of the panda) downward and to the left to adjust the blur transition line angle.\",\"code\":\"pyautogui.moveTo(x=0.412, y=0.388)\\npyautogui.dragTo(x=0.371, y=0.438, button='left')\",\"image\":\"93691719-da8b-42ae-9501-fa6306cd865e.png\"},{\"action\":\"Click the \\\"OK\\\" button to apply the Tilt-Shift filter effect to the image.\",\"code\":\"pyautogui.click(x=0.549, y=0.042)\",\"image\":\"3737d5d0-4fb8-46a5-b4ee-0f37073fd54b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241117225730_75ba5e17-ebc1-4075-b4ad-00d52fc237c3","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"58103983f7b258d1589c0e82d3427b2aa2d20c34dd29afe989f9456a18255300"}
{"messages":[{"role":"user","content":"Create a predesigned task flow in my workspace using the general template and try to add a new task in the first section in Microsoft Power BI"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"workspace\\\" option in the left navigation panel to access my workspace area.\",\"code\":\"pyautogui.click(x=0.0172, y=0.6401)\",\"image\":\"0cc6f624-a06e-4754-bd89-31d97bd75a47.png\"},{\"action\":\"Click the \\\"Select a predesigned task flow\\\" button to access the template options for creating a new task flow.\",\"code\":\"pyautogui.click(x=0.4569, y=0.6427)\",\"image\":\"8b7139fa-3c52-484c-98c7-dc78f229f5f9.png\"},{\"action\":\"Click the \\\"Select\\\" button to confirm the choice of the General predesigned task flow template.\",\"code\":\"pyautogui.click(x=0.5228, y=0.9511)\",\"image\":\"bec68a31-39c1-41f0-9a20-34afabbaef42.png\"},{\"action\":\"Click the \\\"Got it\\\" button on the \\\"Show or hide the task flow\\\" modal to dismiss it and continue with creating the task flow.\",\"code\":\"pyautogui.click(x=0.9201, y=0.7192)\",\"image\":\"673d5c2a-76e4-4034-82dd-6894a35a33ea.png\"},{\"action\":\"Click in the central area of the \\\"Get data\\\" section to interact with this section and prepare to add a new item to it.\",\"code\":\"pyautogui.click(x=0.1013, y=0.52)\",\"image\":\"6aa5bd61-0d13-46dc-9095-1f3418d1d435.png\"},{\"action\":\"Click on the \\\"New item\\\" button at the top of the workspace to begin the process of adding a new task to the first section of the task flow.\",\"code\":\"pyautogui.click(x=0.1167, y=0.273)\",\"image\":\"bca53904-9ecb-44b7-ad7d-ad29a42c9c0e.png\"},{\"action\":\"Click on the Data pipeline option in the \\\"Get data\\\" category to add this type of task to the task flow.\",\"code\":\"pyautogui.click(x=0.4868, y=0.4769)\",\"image\":\"e28aa256-d811-489b-94c9-8b67850dbb17.png\"},{\"action\":\"Click the \\\"OK\\\" button to dismiss the error dialog about the Microsoft Fabric trial capacity limitation.\",\"code\":\"pyautogui.click(x=0.5779, y=0.6435)\",\"image\":\"0c90dae8-3d10-4815-a400-5537de790874.png\"},{\"action\":\"Terminate the task with a success status.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"e9cee177-44e8-4108-8cb2-3005548c4c7c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241001053351_c6152913-56e1-4b5e-b8b8-637e1798902d","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"5b67d0b7198c2f9026279a8e5b35e93770986413ad0f8e4a9fafbbce6992780f"}
{"messages":[{"role":"user","content":"Set the Mail List to 100 in 163, set Line spacing Smail, check Show mail size, select Reading pane as Rottom, select Name display as Name in Contacts (recommended), and select Auto reply as 2024-11-14"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Settings\\\" button in the top navigation bar to open the settings menu.\",\"code\":\"pyautogui.click(x=0.2557, y=0.0963)\",\"image\":\"42f5b89d-261e-4342-9fb9-d5058570336f.png\"},{\"action\":\"Click on \\\"General settings\\\" in the settings menu to access the display and interface configuration options.\",\"code\":\"pyautogui.click(x=0.2875, y=0.1417)\",\"image\":\"26e5077e-aa4c-4184-a059-65e7ac2f7b50.png\"},{\"action\":\"Click on the \\\"Mail List\\\" dropdown menu that currently displays \\\"20 mails per page\\\" to expand the available options.\",\"code\":\"pyautogui.click(x=0.2474, y=0.2259)\",\"image\":\"cf85fd45-0bae-41c5-bf0d-687c9fca054d.png\"},{\"action\":\"Click on the \\\"100 mails per page\\\" option in the Mail List dropdown menu to change the mail display quantity setting from 20 to 100.\",\"code\":\"pyautogui.click(x=0.2531, y=0.3796)\",\"image\":\"b97ecb8c-78f4-46e4-bcdd-56bd80732a0d.png\"},{\"action\":\"Click on the Line spacing dropdown that currently shows \\\"Normal\\\" to expand the options list.\",\"code\":\"pyautogui.click(x=0.2437, y=0.2685)\",\"image\":\"604cd4a3-bd23-443e-8271-90e15efc678c.png\"},{\"action\":\"Click on \\\"Line spacing Small\\\" in the expanded dropdown menu to change the line spacing setting from Normal to Small.\",\"code\":\"pyautogui.click(x=0.2422, y=0.3389)\",\"image\":\"2f75f689-e3a6-41b0-aa90-a8328d13a19e.png\"},{\"action\":\"Click on the \\\"Show mail size\\\" checkbox to enable this setting feature.\",\"code\":\"pyautogui.click(x=0.2323, y=0.2889)\",\"image\":\"36a36c9f-d2aa-4a92-81ab-a4748a79fcf0.png\"},{\"action\":\"Click on the \\\"Bottom\\\" radio button in the Reading pane section to change the reading pane layout from Hide to Bottom.\",\"code\":\"pyautogui.click(x=0.3083, y=0.4176)\",\"image\":\"a8393fdc-ee3a-4e50-860f-4d073f4254f6.png\"},{\"action\":\"Click on the \\\"Name you input as a recipient\\\" dropdown to expand the Name display options.\",\"code\":\"pyautogui.click(x=0.3036, y=0.4676)\",\"image\":\"55ce4e25-7eca-4a8a-b7be-637713816f10.png\"},{\"action\":\"Click on \\\"Name in Contacts (recommended)\\\" in the dropdown menu to select it as the Name display setting.\",\"code\":\"pyautogui.click(x=0.301, y=0.5009)\",\"image\":\"5fdb8516-2809-4b98-8633-4c6b63a93afb.png\"},{\"action\":\"Click on the date field \\\"2023-11-13\\\" in the Auto-reply section to edit the auto-reply date.\",\"code\":\"pyautogui.click(x=0.2182, y=0.4528)\",\"image\":\"da11d70c-5194-4f8f-a722-5f75416455d3.png\"},{\"action\":\"Click on the day portion (\\\"13\\\") of the date field in the Auto-reply section to position the cursor for editing the day value from 13 to 14.\",\"code\":\"pyautogui.click(x=0.3021, y=0.487)\",\"image\":\"98a3d8f8-28db-4cc3-ac74-e8acd3266675.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241209184616_52dea6ee-eaba-44ed-ad6c-cb25998e99c1","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"5e1043032e03483665c4cdf40378ae6abc541a26305951e2ac3d0d772d6ccc9c"}
{"messages":[{"role":"user","content":"Access the Python official website through the Run window. Search for \"numpy\" in the top search bar and go to \"SIG for Built-in Matrix Types in Python.\" Open \"numpy.sf.net\" and download \"/NumPy/1.11.2/numpy-1.11.2.tar.gz.\" Then, return to the PyTorch homepage, navigate to \"Downloads,\" select \"Windows,\" and choose \"Windows installer (64-bit)\" under the section \"Note that Python 3.13.1 cannot be used on Windows 7 or earlier\" to download. After downloading, open the file \"python-3.13.0-amd64.exe.\""},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search input field in the top-right portion of the Python.org website, which currently displays the placeholder text \\\"Search\\\".\",\"code\":\"pyautogui.click(x=0.603, y=0.184)\",\"image\":\"5251778f-97eb-47ca-9394-466dbd7f3652.png\"},{\"action\":\"Type \\\"numpy\\\" in the search field and press Enter to submit the search query.\",\"code\":\"pyautogui.write(message='numpy')\\npyautogui.press('enter')\",\"image\":\"6eec7569-13dc-4fe5-b340-040a01f8bef4.png\"},{\"action\":\"Move the mouse to the \\\"Results\\\" area of the search results page and scroll down slightly to view more search results.\",\"code\":\"pyautogui.moveTo(x=0.230, y=0.565)\\npyautogui.scroll(-10)\",\"image\":\"3e03af74-0551-493a-980d-d1fc58895b71.png\"},{\"action\":\"Click on the \\\"SIG for Built-in Matrix Types in Python\\\" link in the search results.\",\"code\":\"pyautogui.click(x=0.253, y=0.300)\",\"image\":\"a764fe6e-6433-4d7c-adbb-8c49f63c11d7.png\"},{\"action\":\"Move the mouse to the middle of the page and scroll down slightly to better view the Numerical Python section and the numpy.sf.net link.\",\"code\":\"pyautogui.moveTo(x=0.409, y=0.479)\\npyautogui.scroll(-10)\",\"image\":\"da3910fb-73ab-44fc-837f-c35aea88a1c1.png\"},{\"action\":\"Click on the \\\"numpy.sf.net\\\" link under the Numerical Python section to navigate to the NumPy website.\",\"code\":\"pyautogui.click(x=0.545, y=0.476)\",\"image\":\"7e1a546e-651e-4003-8ea9-88711657327d.png\"},{\"action\":\"Move the cursor to the middle of the page and scroll down slightly to view more of the Project Activity section and file listings.\",\"code\":\"pyautogui.moveTo(x=0.460, y=0.533)\\npyautogui.scroll(-10)\",\"image\":\"b06f9ea9-aaab-471a-a0d4-4507f437c9af.png\"},{\"action\":\"Click on the \\\"/NumPy/1.11.2/numpy-1.11.2.tar.gz\\\" link in the Project Activity section to download the NumPy 1.11.2 tarball file.\",\"code\":\"pyautogui.click(x=0.221, y=0.523)\",\"image\":\"530509ee-f370-4ae7-a00d-e1bc2f3b7370.png\"},{\"action\":\"Click the back button in the top-left corner of the browser window to return to the previous page.\",\"code\":\"pyautogui.click(x=0.008, y=0.057)\",\"image\":\"c5482b30-f554-491e-b363-e67c6d84fa2e.png\"},{\"action\":\"Click the back button in the top-left corner of the browser window to navigate to the previous page.\",\"code\":\"pyautogui.click(x=0.016, y=0.053)\",\"image\":\"7539da78-dd87-4818-9010-830f1feae5ed.png\"},{\"action\":\"Move the cursor to the empty space below the Python Software Foundation information box and scroll up slightly to view more of the page content.\",\"code\":\"pyautogui.moveTo(x=0.191, y=0.279)\\npyautogui.scroll(10)\",\"image\":\"303b0134-f9a3-4d38-982e-093ba96c0f34.png\"},{\"action\":\"Click on the \\\"Downloads\\\" link in the top navigation menu of the Python.org website.\",\"code\":\"pyautogui.click(x=0.345, y=0.259)\",\"image\":\"d618840b-6ed9-4468-82f8-f5b5a499e3da.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241227185806_4408ddbb-af11-4dc4-be0e-70fa16f63ed8","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"8c79b7939f8bf7b5e28d20fb479de30a0ffb0e1299bb5b529796d915638606e0"}
{"messages":[{"role":"user","content":"Use Docker graphical mode to search for nginx services and write parameters before running. Check in the browser for success."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search bar at the top of the Docker Desktop window that says \\\"Search for images, containers, volumes...\\\"\",\"code\":\"pyautogui.click(x=0.4813, y=0.1815)\",\"image\":\"e10e04af-c12e-454b-a64e-5b9bc90bbdd7.png\"},{\"action\":\"Click on the \\\"Pull\\\" button next to the nginx image tag selector to download the selected nginx image (stable-alpine3.20-perl) to the local Docker environment.\",\"code\":\"pyautogui.click(x=0.6198, y=0.3389)\",\"image\":\"ed8bc745-95e6-4260-aa98-e4dc185eef0b.png\"},{\"action\":\"Click on the official nginx image name in the search results list to navigate to its detailed view.\",\"code\":\"pyautogui.click(x=0.2911, y=0.3343)\",\"image\":\"a91e9c87-6e9f-4259-85da-de31a40e0c32.png\"},{\"action\":\"Scroll down on the nginx image details page to view more information about configuration options and parameters.\",\"code\":\"pyautogui.moveTo(x=0.6359, y=0.5491)\\npyautogui.scroll(-22)\",\"image\":\"2b8f570c-26ef-436b-beec-b4438b099072.png\"},{\"action\":\"Click the \\\"Run\\\" button in the top-right corner of the nginx image details section to open the container configuration dialog.\",\"code\":\"pyautogui.click(x=0.7479, y=0.4315)\",\"image\":\"2c866065-dd42-4d34-88ee-b3ce2c0c1a4a.png\"},{\"action\":\"Click on the dropdown arrow next to \\\"Optional settings\\\" in the container configuration dialog to expand the additional parameter options.\",\"code\":\"pyautogui.click(x=0.4568, y=0.4907)\",\"image\":\"28057d65-4187-49a6-b325-aa9a61d28df4.png\"},{\"action\":\"Click on the Container name input field to begin entering a custom name for the nginx container.\",\"code\":\"pyautogui.click(x=0.4208, y=0.3787)\",\"image\":\"6234a7e5-ac3c-4e15-92fc-64fcd4d0b5e3.png\"},{\"action\":\"Type \\\"abc\\\" in the Container name input field.\",\"code\":\"pyautogui.write(message='abc')\",\"image\":\"2d1284d6-19e4-4fb0-869e-d310186f91d7.png\"},{\"action\":\"Click in the Host port input field to enter a port number for the nginx container.\",\"code\":\"pyautogui.click(x=0.4214, y=0.4981)\",\"image\":\"d7256800-2de9-4fbf-8f64-ebddf4ba09c5.png\"},{\"action\":\"Type \\\"8000\\\" in the Host port input field to map this port on the host machine to port 80 in the nginx container.\",\"code\":\"pyautogui.write(message='8000')\",\"image\":\"55949888-b94e-4511-8feb-99c46b74fcdf.png\"},{\"action\":\"Click on the Host path input field in the Volumes section to begin entering a directory path for volume mapping.\",\"code\":\"pyautogui.click(x=0.4146, y=0.5852)\",\"image\":\"0d87626d-6792-4f7a-9f58-f8b10be90f10.png\"},{\"action\":\"Type \\\"/some/content\\\" in the Host path input field for volume mapping.\",\"code\":\"pyautogui.write(message='/some/content')\",\"image\":\"3de4fbde-6929-49c0-9dfd-fe88bbc269e8.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114143935_65d8c6fd-37b5-4368-b90b-5ecdffa91ff6","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"2940940945ec62715bb7b80253171e328bb5a5f5996a54d6f6b868d1b8cf990a"}
{"messages":[{"role":"user","content":"Open twitch. Enter the Creator Dashboard. Edit Raid Channel. Then view the live data overview and achievements. Finally, check the live streaming code. Modify live broadcast settings."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Right-click on the Google Chrome icon in the Windows taskbar to open the context menu.\",\"code\":\"pyautogui.rightClick(x=0.569, y=0.9829)\",\"image\":\"76a7150a-373e-48d8-8bc5-8c184e449896.png\"},{\"action\":\"Click on the \\\"Microsoft Edge\\\" option in the context menu to launch the browser.\",\"code\":\"pyautogui.click(x=0.5755, y=0.8708)\",\"image\":\"f013fcf3-77fd-441c-bcb5-fb5f46510d56.png\"},{\"action\":\"Click on the user profile icon in the top-right corner of the Twitch page to open the account menu.\",\"code\":\"pyautogui.click(x=0.9794, y=0.1255)\",\"image\":\"dd518611-012f-4fa9-830c-53601d33cb06.png\"},{\"action\":\"Click on the \\\"Creator Dashboard\\\" option in the user dropdown menu to access the streamer's dashboard interface.\",\"code\":\"pyautogui.click(x=0.9362, y=0.2884)\",\"image\":\"3961d8dd-d187-4b41-a60f-aefb9db2474c.png\"},{\"action\":\"Click on the \\\"Stream Manager\\\" option in the left navigation menu of the Twitch Creator Dashboard.\",\"code\":\"pyautogui.click(x=0.0612, y=0.2579)\",\"image\":\"f3d19ad8-dc07-4575-8f1b-1f1d1806335c.png\"},{\"action\":\"Click on the raid/megaphone icon in the Quick Actions bar to access raid channel settings.\",\"code\":\"pyautogui.click(x=0.2703, y=0.8218)\",\"image\":\"c68c4cd3-8745-44b9-a0d7-ff9c24ce536a.png\"},{\"action\":\"Click on the \\\"Vstreamer\\\" tag in the first streamer card to either filter the raid list or select this specific channel for raiding.\",\"code\":\"pyautogui.click(x=0.5583, y=0.3495)\",\"image\":\"c0d91f8e-cf3b-4af9-b443-33a6654e2172.png\"},{\"action\":\"Click the \\\"Start Raid\\\" button to initiate the raid to the selected streamer's channel.\",\"code\":\"pyautogui.click(x=0.7708, y=0.9255)\",\"image\":\"a7c7d48c-fa73-467c-8053-829a96e040e8.png\"},{\"action\":\"Click on the \\\"Overview\\\" option in the Analytics submenu in the left sidebar to access the channel's data overview dashboard.\",\"code\":\"pyautogui.click(x=0.062, y=0.3616)\",\"image\":\"af897530-9cea-44be-a39f-3ee4eb1dded1.png\"},{\"action\":\"Click on the \\\"Achievements\\\" option in the left sidebar navigation menu under the Analytics section.\",\"code\":\"pyautogui.click(x=0.0596, y=0.4338)\",\"image\":\"74fb34af-ce57-4e64-b78c-63bba9572bb7.png\"},{\"action\":\"Click on the \\\"Settings\\\" option in the left sidebar navigation menu to access stream configuration features.\",\"code\":\"pyautogui.click(x=0.0607, y=0.6981)\",\"image\":\"54bc789c-dd15-46a7-9aa8-f4ffa6f49902.png\"},{\"action\":\"Click on the \\\"Stream\\\" option in the Settings submenu in the left navigation panel to access streaming configuration settings.\",\"code\":\"pyautogui.click(x=0.0555, y=0.7338)\",\"image\":\"3e5e715f-2fd3-4cc2-b3c1-d7ace11cd2ca.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241128081522_77ea86b6-cf30-4314-ae8d-e81f4e80965b","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"333230577fcac7b40692b4181e744aca5bd141d06191f630ef824af885370d37"}
{"messages":[{"role":"user","content":"Search 电视 on jd.com, filter products of brands MI, Hisense, TCL, Skyworth, HUAWEI, Sony, Samsung, LG, filter smart TVs, filter products within the price range of CNY 2000 and CNY 3000, add Skyworth S75E to comparison, add Redmi A65 to comparison, add Vidda Hisense R65 to comparison, add TCL 55T6K to comparison, start comparing and highlight the differences, view details of TCL 55T6K and add it to focus."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click in the search box where \\\"牛奶鲜奶\\\" (milk) is currently displayed to position the cursor for editing the search text.\",\"code\":\"pyautogui.click(x=0.3398, y=0.3419)\",\"image\":\"ae00b9ff-84b1-4f78-80a0-5f768844d763.png\"},{\"action\":\"Type \\\"dianshi \\\" in the search box to input \\\"电视\\\" (TV) in Chinese characters.\",\"code\":\"pyautogui.write(message='dianshi ')\",\"image\":\"96e71a5f-e33e-4add-a673-c55dd4f5eb81.png\"},{\"action\":\"Click the red \\\"搜索\\\" (search) button to submit the search query for TVs.\",\"code\":\"pyautogui.click(x=0.7879, y=0.3581)\",\"image\":\"31d8040f-56e9-44f2-966a-b2c20d39ab71.png\"},{\"action\":\"Click the \\\"+ 多选\\\" (multiple selection) button next to the brand filters to enable selecting multiple TV brands simultaneously.\",\"code\":\"pyautogui.click(x=0.8969, y=0.4263)\",\"image\":\"829757ee-11fc-4ca3-8fd0-64c738900749.png\"},{\"action\":\"Click the \\\"多选\\\" (multiple selection) button in the top-right corner of the brand filter section to activate the multi-select functionality for brands.\",\"code\":\"pyautogui.click(x=0.934, y=0.4288)\",\"image\":\"5913988f-eafa-4f4d-8289-a3b7b32bd63d.png\"},{\"action\":\"Click on the Xiaomi (小米) brand checkbox to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.1539, y=0.4856)\",\"image\":\"c3e43206-244c-4d9a-833b-8aa4f73fd363.png\"},{\"action\":\"Click on the Hisense (海信) brand checkbox to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.241, y=0.4844)\",\"image\":\"446f3382-5393-47a0-a816-82d760b55d3e.png\"},{\"action\":\"Click on the TCL brand checkbox in the brand filter section to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.3324, y=0.4844)\",\"image\":\"6da09b4e-85ef-428d-bcf2-f4f7d30ce9fa.png\"},{\"action\":\"Click on the Skyworth (创维) brand checkbox to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.4191, y=0.4794)\",\"image\":\"9c7569d1-cce2-493e-be58-afbdd33201c4.png\"},{\"action\":\"Click on the HUAWEI (华为) brand checkbox to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.5195, y=0.4856)\",\"image\":\"cea3dd3e-9017-4846-be62-b569eff57c7e.png\"},{\"action\":\"Click on the Sony brand checkbox to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.2703, y=0.5531)\",\"image\":\"373b08d7-f0df-4024-a01e-c3b119663bb0.png\"},{\"action\":\"Click on the Samsung (三星) brand checkbox in the brand selection grid to select it as part of the multi-brand filter.\",\"code\":\"pyautogui.click(x=0.1363, y=0.5594)\",\"image\":\"8e5d72ff-29db-4b4b-91c3-944930766817.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241206092557_74be8a70-9c1d-4674-9fe0-56a5907bab6e","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"f5a8f7584f7886d8828056bd0a0ff8ed95236695bd5a9ded86159642690187e4"}
{"messages":[{"role":"user","content":"Share the Notion link with friends (only can view), publish the page to the website. Then leave a comment and save the page to favorites. Finally, export the page as a PDF file and save it to downloads."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Write \\\"11.30 DDL\\\" in the comment field.\",\"code\":\"pyautogui.write(message='11.30 DDL')\",\"image\":\"e6b81a22-5f86-4a74-a05e-62a0e4320f64.png\"},{\"action\":\"Press the backspace key to delete the current auto-generated filename and then type \\\"PS3\\\" as the new filename for the PDF export.\",\"code\":\"pyautogui.press('backspace')\\npyautogui.write(message='PS3')\",\"image\":\"90d86ecd-9bb4-4551-a1eb-6f69ae25e842.png\"},{\"action\":\"Terminate the task with success status.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"96da826f-12fd-4413-81fb-b62dca554e0b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241130200855_db82c789-4e45-4c6e-8901-1825f38c0456","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"47e43283f63f4db54dd49f79ff4e2bda9a477ad559512128c138e211ace7a581"}
{"messages":[{"role":"user","content":"go to google and open google images, search a loop gif, open the fifth result and click to visit his main page, then download the gif"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the restore down/maximize window button in the top-right corner of the Chrome browser.\",\"code\":\"pyautogui.click(x=0.8799, y=0.0444)\",\"image\":\"f65aeba0-00f8-46d1-80d1-cad61481de8a.png\"},{\"action\":\"Click on the \\\"Imágenes\\\" link in the top navigation bar of Google to navigate to Google Images.\",\"code\":\"pyautogui.click(x=0.8569, y=0.1744)\",\"image\":\"425eaf33-3cc0-4179-8da5-bfae09a98040.png\"},{\"action\":\"Click on the Google Images search bar in the middle of the screen to begin entering the search query.\",\"code\":\"pyautogui.click(x=0.3924, y=0.4544)\",\"image\":\"98c6fe03-e80b-482b-86d5-9c8ba26d5618.png\"},{\"action\":\"Type \\\"loop gif\\\" in the search field and press Enter to search for loop GIFs on Google Images.\",\"code\":\"pyautogui.write(message='loop gif')\\npyautogui.press('enter')\",\"image\":\"1c66d876-ddb6-4aff-904f-297d160cc949.png\"},{\"action\":\"Click on the black and white space-themed GIF showing a planet with rings against a starry background, which appears to be from Pinterest.\",\"code\":\"pyautogui.click(x=0.1167, y=0.4956)\",\"image\":\"495b567b-2dad-4d26-bd9b-0f1e75cdc8e2.png\"},{\"action\":\"Click the \\\"Visitar\\\" button to navigate to the Pinterest page where the black and white loop GIF is hosted.\",\"code\":\"pyautogui.click(x=0.7542, y=0.4567)\",\"image\":\"d95ab8aa-3870-4e08-9f8a-6c1a1017ce6a.png\"},{\"action\":\"Right-click on the black and white space GIF showing the ringed planet against the starry background to open the context menu for downloading the image.\",\"code\":\"pyautogui.rightClick(x=0.2083, y=0.4356)\",\"image\":\"937af7a7-30df-4882-9e63-98bb0999efab.png\"},{\"action\":\"Click on the \\\"Copiar imagen\\\" (Copy image) option in the context menu to copy the GIF to the clipboard.\",\"code\":\"pyautogui.click(x=0.3125, y=0.5044)\",\"image\":\"928812c4-edfb-4aa6-97dd-5dc915ccd82e.png\"},{\"action\":\"Click the \\\"Guardar\\\" (Save) button to download and save the GIF file to the Downloads folder.\",\"code\":\"pyautogui.click(x=0.4542, y=0.5467)\",\"image\":\"275381f9-68f4-4b35-ae54-b8f72a6583dd.png\"},{\"action\":\"Terminate the task with a success status, as all required steps have been completed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"d5972053-f074-4baa-b34c-db8af3957a18.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241101021356_d407d566-b79e-4ff2-8682-08b1e962d185","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"edffeab5f69ccca05129bd9e8b10438b47a05b470e024ee8d66a18093cd07cfc"}
{"messages":[{"role":"user","content":"In Zapier, configure chatbot settings. Toggle the chatbot AI_Assistant, change its public name to Bot, change the message placeholder, drag the creativity bar to 0.3, choose to display disclosure message and edit it, change the browser title to Conversation and hide the generated by Zapier icon"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Chatbots\\\" option in the left navigation menu to access the chatbot configuration page.\",\"code\":\"pyautogui.click(x=0.0694, y=0.5528)\",\"image\":\"aabd8d86-e0dd-4ec5-bb08-cfe8daca1235.png\"},{\"action\":\"Click on the \\\"AI_Assistant\\\" chatbot in the list to access its configuration settings.\",\"code\":\"pyautogui.click(x=0.256, y=0.4611)\",\"image\":\"7a9426e7-25aa-484f-9383-569150f1b164.png\"},{\"action\":\"Click on the \\\"Advanced\\\" tab in the navigation menu to access additional configuration settings for the AI_Assistant chatbot.\",\"code\":\"pyautogui.click(x=0.4222, y=0.2208)\",\"image\":\"1884cd94-587f-4b8c-99b7-15106912a9b4.png\"},{\"action\":\"Double-click on the public name text field containing \\\"AI_Assistant\\\" to select all text for editing.\",\"code\":\"pyautogui.doubleClick(x=0.1245, y=0.3931)\",\"image\":\"29b4c7a3-1a76-4d6a-8853-d080c7d69d76.png\"},{\"action\":\"Type \\\"Bot\\\" to replace the selected text in the public name field.\",\"code\":\"pyautogui.write(message='Bot')\",\"image\":\"08bb09c2-cdd7-454e-9a12-2f425e0ee687.png\"},{\"action\":\"Double-click on the \\\"Ask me anything\\\" text in the User message placeholder field to select all text for editing.\",\"code\":\"pyautogui.doubleClick(x=0.1162, y=0.516)\",\"image\":\"b1f6ed9a-c716-4dd6-9f67-caa75b168cad.png\"},{\"action\":\"Click within the User message placeholder text field to position the cursor for typing the new placeholder text.\",\"code\":\"pyautogui.click(x=0.1162, y=0.516)\",\"image\":\"f3f240cc-d809-4a0d-bd45-18617b152eb2.png\"},{\"action\":\"Type \\\"Please state your question\\\" in the User message placeholder field.\",\"code\":\"pyautogui.write(message='Please state your question')\",\"image\":\"7b894dfd-8e81-4971-94bc-d6bdbe356ca1.png\"},{\"action\":\"Drag the creativity slider from its current position at 0.7 toward the left to the 0.3 position (closer to the \\\"Predictable\\\" end of the scale).\",\"code\":\"pyautogui.moveTo(x=0.3019, y=0.6049)\\npyautogui.dragTo(x=0.1384, y=0.6104, button='left')\",\"image\":\"32c600bb-4c57-45b9-8427-b4619568e0e2.png\"},{\"action\":\"Click on the toggle switch next to \\\"Display disclosure message in Greeting\\\" to turn it on.\",\"code\":\"pyautogui.click(x=0.0431, y=0.7069)\",\"image\":\"79bc62fb-7965-45dc-a793-dd67a8ccbd25.png\"},{\"action\":\"Double-click on the \\\"chat\\\" text in the Browser title field to select all the text for editing.\",\"code\":\"pyautogui.doubleClick(x=0.069, y=0.6479)\",\"image\":\"8776088c-ff7f-4261-a7d6-15897e30d393.png\"},{\"action\":\"Type \\\"Conversation\\\" to replace the selected text in the browser title field.\",\"code\":\"pyautogui.write(message='Conversation')\",\"image\":\"9e4131a1-e940-4248-bdca-bb01926fc043.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241202165709_62b53f0a-6ee4-43f4-93cf-8b1b1c7b05a8","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"d9591fe50a5a2eac0fa23d7f6b86fe81a944eca89c1a073add087e2b8efd13f8"}
{"messages":[{"role":"user","content":"Search for Hong Kong holiday in 2024, check two websites for information about the national day holiday on October 1st"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the address bar at the top of the browser to enter a search query.\",\"code\":\"pyautogui.click(x=0.235, y=0.079)\",\"image\":\"0b48ca85-b096-4f67-9ea5-92c6860bf9e4.png\"},{\"action\":\"Type \\\"hong kong holiday\\\" in the address bar, then press the down arrow key twice to select the third search suggestion from the dropdown menu, and press enter to execute the search.\",\"code\":\"pyautogui.write(message='hong kong holiday')\\npyautogui.press('down', presses=2)\\npyautogui.press('enter')\",\"image\":\"33c6d9ea-fb20-4e86-9084-4b8dfc08dc01.png\"},{\"action\":\"Click on the \\\"General holidays for 2024\\\" link from the GovHK website search result to access official information about Hong Kong's 2024 public holidays.\",\"code\":\"pyautogui.click(x=0.257, y=0.730)\",\"image\":\"ba8defb0-3438-4954-aa0f-b12562a73f90.png\"},{\"action\":\"Scroll down the page to view more of the holiday table and find information about the National Day holiday.\",\"code\":\"pyautogui.moveTo(x=0.538, y=0.631)\\npyautogui.scroll(-10)\",\"image\":\"9c9584aa-e870-4096-821a-c9fbd3093f91.png\"},{\"action\":\"Click the back button in the browser navigation bar to return to the previous page.\",\"code\":\"pyautogui.click(x=0.021, y=0.072)\",\"image\":\"5fe57c26-4ee3-448f-b94a-a2680851e961.png\"},{\"action\":\"Scroll down the search results page to view more website options for Hong Kong holiday information.\",\"code\":\"pyautogui.moveTo(x=0.386, y=0.633)\\npyautogui.scroll(-10)\",\"image\":\"16b50273-dfc0-42cf-85fc-4a1aeac5c40a.png\"},{\"action\":\"Click on the \\\"Holidays and Observances in Hong Kong in 2024\\\" search result from the Time and Date website to open it as our second source for Hong Kong holiday information.\",\"code\":\"pyautogui.click(x=0.361, y=0.715)\",\"image\":\"7d3a01dc-a923-4350-a72b-6f7e93489442.png\"},{\"action\":\"Scroll down the webpage to view more holiday entries in the table, specifically to find information about National Day in October.\",\"code\":\"pyautogui.moveTo(x=0.518, y=0.678)\\npyautogui.scroll(-10)\",\"image\":\"66c9fddc-952d-43f3-b711-7726096f8cfd.png\"},{\"action\":\"Terminate the task with a success status since we have completed the goal of checking two websites for information about Hong Kong's National Day holiday on October 1st, 2024.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"fda8decd-b5e9-4c8c-9815-b61695429c49.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240918154219_488fe792-78e3-4472-85e7-7c467d8b8282","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"3a05f2adf1dc0ac0adf47bf1e702ede13d48ee972d9998bab79b55bb6c3260bb"}
{"messages":[{"role":"user","content":"Open Chrome, perform Gmail search, open Gmail, copy email address dzakwanrafiarrayyan.5@upi.edu, compose email, paste email address dzakwanrafiarrayyan.5@upi.edu, create test message, change align format to right, add confidential mode in 1 month, send email, open email, give star, delete email."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Type \\\"gmail\\\" in the Google search box and press Enter.\",\"code\":\"pyautogui.write(message='gmail')\\npyautogui.press('enter')\",\"image\":\"e292793e-ee2e-4270-bc2a-2fe2459af5c2.png\"},{\"action\":\"Click on the \\\"Email - Gmail - Google\\\" link in the first search result to access Gmail.\",\"code\":\"pyautogui.click(x=0.2385, y=0.3972)\",\"image\":\"840bf7f3-c9c9-4a31-adc1-0ac31c16e377.png\"},{\"action\":\"Click on the user profile icon located in the top right corner of the Gmail interface, next to the Universitas Pendidikan Indonesia logo.\",\"code\":\"pyautogui.click(x=0.9875, y=0.1898)\",\"image\":\"bbfa55a7-6d6d-412f-b12e-876e535e30e0.png\"},{\"action\":\"Right-click on the email address \\\"dzakwanrafiarrayyan.5@upi.edu\\\" displayed at the top of the Google account popup to access the context menu for copying options.\",\"code\":\"pyautogui.rightClick(x=0.9026, y=0.2537)\",\"image\":\"904ac9e2-f715-4730-b54c-0ed775361f1e.png\"},{\"action\":\"Click on the \\\"Copy\\\" option in the context menu to copy the email address to the clipboard.\",\"code\":\"pyautogui.click(x=0.8255, y=0.287)\",\"image\":\"44c1fb9e-5ed5-4e8b-9b0a-46c5c5730fb8.png\"},{\"action\":\"Click on the \\\"Compose\\\" button in the left sidebar of Gmail to create a new email.\",\"code\":\"pyautogui.click(x=0.1229, y=0.2611)\",\"image\":\"63708b54-fecc-424a-8287-4b21f92fcc6a.png\"},{\"action\":\"Press Ctrl+V to paste the previously copied email address into the \\\"To\\\" field of the compose window.\",\"code\":\"pyautogui.hotkey(['ctrl', 'v'])\",\"image\":\"cea0a8f6-bd78-4dec-9b7e-cb34bf90aad9.png\"},{\"action\":\"Click on the Subject field in the email compose window.\",\"code\":\"pyautogui.click(x=0.6568, y=0.5694)\",\"image\":\"d58b857d-1aaa-473d-82a7-06b0f834353b.png\"},{\"action\":\"Type \\\"Testing\\\" in the Subject field of the email compose window.\",\"code\":\"pyautogui.write(message='Testing')\",\"image\":\"b10a5b81-018a-4ac6-9984-027aef4cce55.png\"},{\"action\":\"Click on the Bold button (B) in the formatting toolbar to enable bold text formatting for the email content.\",\"code\":\"pyautogui.click(x=0.7714, y=0.8565)\",\"image\":\"a98c9319-e24c-423f-adce-d426f53f1c37.png\"},{\"action\":\"Click on the text alignment button in the formatting toolbar to open the alignment options dropdown menu.\",\"code\":\"pyautogui.click(x=0.8677, y=0.8537)\",\"image\":\"b4a08806-9f11-4fb0-a8d9-0f1267a0940e.png\"},{\"action\":\"Click on the right align option (third option) in the text alignment dropdown menu.\",\"code\":\"pyautogui.click(x=0.8688, y=0.8213)\",\"image\":\"d4fbf61b-21cb-4702-a767-94a3479403cf.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241129105740_e7cbde6d-b81a-4430-acb6-1840598b7840","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"b88a33c8a1e29d6317310e22de876d09541c678281ab8cac9d1c847748c9bd57"}
{"messages":[{"role":"user","content":"Ask GPT about how to find my public profile url in my LinkedIn, follow the instruction and open the url"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the message input field at the bottom of the screen where it says \\\"Message ChatGPT\\\" to begin typing a question about finding a LinkedIn public profile URL.\",\"code\":\"pyautogui.click(x=0.4523, y=0.8712)\",\"image\":\"d2db8c93-d66d-4d59-aac9-cca7b677789d.png\"},{\"action\":\"Type \\\"how to check my public linkedin page\\\" in the message input field and press Enter to send the message to ChatGPT.\",\"code\":\"pyautogui.write(message='how to check my public linkedin page')\\npyautogui.press('enter')\",\"image\":\"365c1837-c703-40f5-bc40-9a7004612cac.png\"},{\"action\":\"Click on the \\\"linkedin.com\\\" link in ChatGPT's response, which is part of the first instruction for accessing LinkedIn to find the public profile URL.\",\"code\":\"pyautogui.click(x=0.5254, y=0.3169)\",\"image\":\"c213a0ca-8bd8-4a15-b8f3-54ff9ebe0978.png\"},{\"action\":\"Click on the profile icon in the LinkedIn navigation bar to access the user's profile or profile options.\",\"code\":\"pyautogui.click(x=0.716, y=0.1381)\",\"image\":\"4ba63f16-b7e2-47cd-b6db-a470d45a2d85.png\"},{\"action\":\"Click on the \\\"View Profile\\\" button in the dropdown menu to navigate to Xinyuan Wang's LinkedIn profile page.\",\"code\":\"pyautogui.click(x=0.6711, y=0.2956)\",\"image\":\"b5ade32e-5f56-4dce-89b0-31bf2a5e40bc.png\"},{\"action\":\"Press Ctrl+C to copy the selected LinkedIn public profile URL to the clipboard.\",\"code\":\"pyautogui.hotkey(['ctrl', 'c'])\",\"image\":\"4c183bdb-73c5-4979-a4a3-3da7d643ff81.png\"},{\"action\":\"Click the close (X) button on the \\\"hku address - Google\\\" browser tab to close it.\",\"code\":\"pyautogui.click(x=0.8125, y=0.0406)\",\"image\":\"1374493f-573a-4038-8993-659cc3426cfc.png\"},{\"action\":\"Double-click on the browser address bar to select all the text in it, preparing to replace it with the copied LinkedIn profile URL.\",\"code\":\"pyautogui.doubleClick(x=0.7719, y=0.0644)\",\"image\":\"80146f1d-8158-4ab1-a0be-0ecc9da7fdf1.png\"},{\"action\":\"Press Ctrl+V to paste the copied LinkedIn profile URL into the address bar, then press Enter to navigate to the URL.\",\"code\":\"pyautogui.write(message='V')\\npyautogui.press('enter')\",\"image\":\"0b33ab68-eae8-4e49-a354-1e3616fbc90f.png\"},{\"action\":\"Scroll down slightly on Xinyuan Wang's LinkedIn profile page to view more content below the currently visible area.\",\"code\":\"pyautogui.moveTo(x=0.6188, y=0.5981)\\npyautogui.scroll(-20)\",\"image\":\"abb48d6d-4286-4929-9d54-52c16159d792.png\"},{\"action\":\"Terminate the task with a success status as we have successfully completed the goal of finding and opening the LinkedIn public profile URL.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"cde710ba-6bcd-4f88-8450-295e9188eda9.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240930162723_72c956be-e8a4-420a-bb37-7297e7f271e6","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"33cbc4f7cb366d4d3efc800673c4117404958fcf42ecf9a6228308fe0c95f2f6"}
{"messages":[{"role":"user","content":"Open the interface of QQ. Change the personalized signature of QQ. Click to open the chat box. Send a message. Close the chat box. Open it again. Send an emoji. Close the chat box."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Type \\\"nihao \\\" in the message composition area and press Enter to send the message.\",\"code\":\"pyautogui.write(message='nihao ')\\npyautogui.press('enter')\",\"image\":\"8103a37e-2f27-49c6-a9ee-bf00f8ff52d6.png\"},{\"action\":\"Click the \\\"关闭(Close)\\\" button in the bottom right corner of the QQ chat window to close the current conversation.\",\"code\":\"pyautogui.click(x=0.822, y=0.897)\",\"image\":\"1428606e-cf33-44fc-9539-59a4222e0f7b.png\"},{\"action\":\"Click on the blue \\\"你好\\\" (Hello) button in the chat window to send a quick emoji or message.\",\"code\":\"pyautogui.click(x=0.851, y=0.374)\",\"image\":\"e120579d-9dfa-4a0a-ae6f-642fa6814bad.png\"},{\"action\":\"Click the close (X) button in the top-right corner of the QQ chat window to close the conversation.\",\"code\":\"pyautogui.doubleClick(x=1.000, y=0.262)\",\"image\":\"5a02924f-7491-4d05-9a0b-956656de59e2.png\"},{\"action\":\"Click on the smiling emoji in the middle of the emoji selection panel to send it in the chat.\",\"code\":\"pyautogui.click(x=0.489, y=0.476)\",\"image\":\"5dcaef98-1ffe-4e29-803c-6d580104835b.png\"},{\"action\":\"Click the \\\"关闭(Close)\\\" button at the bottom right corner of the QQ chat window to close the chat.\",\"code\":\"pyautogui.click(x=0.815, y=0.900)\",\"image\":\"7a8e3d0b-e030-458e-aa70-5c24bbd7b7ea.png\"},{\"action\":\"Click the Google search bar to make it active for typing a search query.\",\"code\":\"pyautogui.click(x=0.844, y=0.375)\",\"image\":\"938cf978-f04c-4a2d-8ab0-74929328d07e.png\"},{\"action\":\"The minimize button of the Google Chrome browser window was clicked.\",\"code\":\"pyautogui.click(x=1.000, y=0.194)\",\"image\":\"6914ecd5-6aeb-4ac3-9660-eaba149f9728.png\"},{\"action\":\"Complete the task by indicating successful completion of all required steps - the QQ interface was opened, personalized signature was changed to \\\"life is beautiful\\\", chat with contact \\\"ooooo\\\" was opened, a message was sent, chat was closed and reopened, an emoji was sent, and the chat was closed again.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"af5ebbc2-967b-431b-9ba1-75ef5f9dfa26.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241010133502_8ced26dc-f215-4f6c-b90a-94e24dbaf646","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"321c9b6131f35f2ec45182b681dad4bc5273caab4b2ac6bb46a0dabd43abd1dd"}
{"messages":[{"role":"user","content":"Create a Certificate with Logos, Seals, and Signatures in Canva"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Microsoft Edge browser icon in the taskbar to open the web browser.\",\"code\":\"pyautogui.click(x=0.489, y=0.972)\",\"image\":\"b7f0d8ee-d0a3-42bd-8634-13e6ea3262cf.png\"},{\"action\":\"Click on the Canva option in the application launcher at the bottom of the screen to open Canva.\",\"code\":\"pyautogui.click(x=0.490, y=0.994)\",\"image\":\"e00a963d-5d70-47f4-b98f-12bf1bac179f.png\"},{\"action\":\"Click on the plus (+) icon in the top navigation bar to create a new design.\",\"code\":\"pyautogui.click(x=0.087, y=0.032)\",\"image\":\"69a2042f-0ca6-499a-865a-ba4c5d69611f.png\"},{\"action\":\"Type \\\"certificate\\\" in the search bar that says \\\"What would you like to create?\\\" to search for certificate templates.\",\"code\":\"pyautogui.write(message='certificate')\",\"image\":\"af8791e3-d3e8-4a07-9049-4372578e2d4e.png\"},{\"action\":\"Click the search icon (magnifying glass) next to the search text field to execute the search for certificate templates.\",\"code\":\"pyautogui.click(x=0.110, y=0.082)\",\"image\":\"7d98e08d-77cb-41bb-b599-3e0b889d00d7.png\"},{\"action\":\"Click on the \\\"Certificate 29.7×21 cm\\\" template option in the dropdown menu to select this certificate template for editing.\",\"code\":\"pyautogui.click(x=0.124, y=0.194)\",\"image\":\"55f0f891-6ed9-4d59-a2a5-db6bd9917947.png\"},{\"action\":\"Click on the \\\"Certificate of Achievement\\\" template in the bottom row of the templates panel.\",\"code\":\"pyautogui.click(x=0.201, y=0.803)\",\"image\":\"98b0fae8-ff06-453b-a5fa-cefd57aa8bdb.png\"},{\"action\":\"Double-click on the \\\"Certificate of Achievement\\\" title text to activate text editing mode and begin customizing the certificate.\",\"code\":\"pyautogui.doubleClick(x=0.556, y=0.378)\",\"image\":\"7d125eae-81aa-4d18-b88d-df5742b26355.png\"},{\"action\":\"Click on the Lorem ipsum placeholder text in the body of the certificate to begin editing it.\",\"code\":\"pyautogui.click(x=0.491, y=0.543)\",\"image\":\"ca0e4c4c-e5e3-4c20-89ee-9048b6c72cc2.png\"},{\"action\":\"Double-click on the Lorem ipsum placeholder text in the middle of the certificate to select it for editing.\",\"code\":\"pyautogui.doubleClick(x=0.495, y=0.506)\",\"image\":\"e7583d31-aae2-47c1-b0d6-a274d8e5d686.png\"},{\"action\":\"Double-click on the name \\\"Marceline Anderson\\\" to enter text editing mode and begin typing a new recipient name for the certificate.\",\"code\":\"pyautogui.doubleClick(x=0.495, y=0.500)\",\"image\":\"a7a3b87d-150e-4a70-8d1a-41d123a4d1c0.png\"},{\"action\":\"Click on the \\\"Head of Event\\\" signature line area to select it for editing.\",\"code\":\"pyautogui.click(x=0.418, y=0.663)\",\"image\":\"fbb67517-561f-4079-bae3-5891882d33ef.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114150637_206587eb-a4a7-4f44-8eab-25359dd4f631","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"c812d61ee8ffc6f3fd6bd25854c0c5ab8ac6d37a3c8ca5444d15c98aaf4617bb"}
{"messages":[{"role":"user","content":"Search and select Islamabad to London flights with the lowest fare for one passenger in business class, departure 25 Nov, arrival 3 Dec on qatarairways.com."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Double-click on the Chrome browser icon in the Windows taskbar to launch the web browser.\",\"code\":\"pyautogui.doubleClick(x=0.741, y=0.9766)\",\"image\":\"4dda29c8-eef1-467a-9175-0f2d01de768d.png\"},{\"action\":\"Click on the Qatar Airways shortcut icon in the shortcuts row at the bottom of the Google homepage.\",\"code\":\"pyautogui.click(x=0.8538, y=0.4922)\",\"image\":\"70be4636-4c32-48a9-bfb9-02bf43c2ed21.png\"},{\"action\":\"Click on the \\\"1 Passenger Economy\\\" dropdown menu in the flight booking form to change the cabin class from Economy to Business.\",\"code\":\"pyautogui.click(x=0.2049, y=0.6911)\",\"image\":\"9542a547-c6d4-40e2-b394-00ab654f933b.png\"},{\"action\":\"Click on \\\"Islamabad\\\" in the departure city dropdown menu to confirm this selection.\",\"code\":\"pyautogui.click(x=0.2149, y=0.2932)\",\"image\":\"92a63133-b42c-4698-af3a-ab30970e5695.png\"},{\"action\":\"Click on \\\"London\\\" in the destination dropdown menu to confirm London as the destination city.\",\"code\":\"pyautogui.click(x=0.3979, y=0.3203)\",\"image\":\"4e32f3a1-4555-48c8-ade7-50551aed8426.png\"},{\"action\":\"Click the \\\"Continue\\\" button to proceed with the flight search process.\",\"code\":\"pyautogui.click(x=0.7007, y=0.749)\",\"image\":\"45c92e45-598f-42fe-b0a3-af453ede4006.png\"},{\"action\":\"Click on the dropdown arrow next to \\\"1 Passenger Economy\\\" to open the passenger and cabin class selection menu.\",\"code\":\"pyautogui.click(x=0.8733, y=0.2401)\",\"image\":\"805257fd-4d33-4976-9eec-c3799076bf06.png\"},{\"action\":\"Click on the \\\"Premium(Business/First)\\\" option in the cabin class selection dropdown menu to change the class from Economy to Business.\",\"code\":\"pyautogui.click(x=0.8528, y=0.7099)\",\"image\":\"08f7f1cf-0638-4f56-b1f5-d7054b423751.png\"},{\"action\":\"Click the \\\"Confirm\\\" button at the bottom of the passenger and cabin class selection dropdown to finalize the Premium(Business/First) class selection.\",\"code\":\"pyautogui.click(x=0.8142, y=0.7833)\",\"image\":\"f734840a-67e5-4231-b3a8-9a30e3ea912c.png\"},{\"action\":\"Click on the \\\"Search flights\\\" button to initiate the flight search with the selected parameters.\",\"code\":\"pyautogui.click(x=0.826, y=0.3146)\",\"image\":\"2a4bfbec-ec5c-4fd7-845a-706d178265d6.png\"},{\"action\":\"Click on the \\\"Sort and filter\\\" button to access filtering and sorting options for the flight search results.\",\"code\":\"pyautogui.click(x=0.9243, y=0.5734)\",\"image\":\"252fafaa-9b5e-44e9-967a-fc091bfff6d6.png\"},{\"action\":\"Click on the \\\"Lowest fare\\\" sorting option in the Sort and filter panel to arrange flights from cheapest to most expensive.\",\"code\":\"pyautogui.click(x=0.6059, y=0.3182)\",\"image\":\"76371456-4f74-4837-8277-a49eb926131b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241118210354_7d0db5be-43f4-4ee8-8a68-6804eb8937ce","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"2c178b6f35ecf0599586635df0fd207df978e29633bfb87dd6cb85b319577287"}
{"messages":[{"role":"user","content":"In Google Play, go to Apps section, choose Car. Click on Google Chrome and choose Firefox Fast & Private Browser, and then share it using Facebook."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Apps\\\" tab in the Google Play Store navigation menu to switch from Games to Apps section.\",\"code\":\"pyautogui.click(x=0.219, y=0.130)\",\"image\":\"280e897e-1f54-4f29-b7b2-055fc6eded72.png\"},{\"action\":\"Click on the \\\"Phone\\\" filter button to narrow down the apps to those compatible with phone devices.\",\"code\":\"pyautogui.click(x=0.222, y=0.129)\",\"image\":\"f8ee7beb-2cb3-4ebb-9927-954c87677a93.png\"},{\"action\":\"Click on the \\\"Car\\\" filter button in the device filter options at the top of the Google Play Store page.\",\"code\":\"pyautogui.click(x=0.505, y=0.211)\",\"image\":\"39f1497b-6ddb-4600-b35a-0a41e83c2862.png\"},{\"action\":\"Scroll down the page to view more apps in the Google Play Store by scrolling the mouse wheel downward at the center of the page.\",\"code\":\"pyautogui.moveTo(x=0.505, y=0.211)\\npyautogui.scroll(-10)\",\"image\":\"f8f70bac-3527-4b95-a7cc-16181577127e.png\"},{\"action\":\"Click on the Google Chrome app in the \\\"Explore all apps\\\" section.\",\"code\":\"pyautogui.click(x=0.101, y=0.458)\",\"image\":\"4ddb7931-9124-45f4-9744-e494dfdcdab0.png\"},{\"action\":\"Scroll down on the Google Chrome app page to view more of the \\\"Similar apps\\\" section.\",\"code\":\"pyautogui.moveTo(x=0.739, y=0.642)\\npyautogui.scroll(-10)\",\"image\":\"6a2eaaa2-1edd-4b7a-8f72-0482e0fe88dd.png\"},{\"action\":\"Click on the Firefox Fast & Private Browser app in the Similar apps section.\",\"code\":\"pyautogui.click(x=0.787, y=0.559)\",\"image\":\"c877e2d9-509d-4ff9-87ac-499edf6eb950.png\"},{\"action\":\"Click on the \\\"Share\\\" button next to the Install button to open sharing options for the Firefox Fast & Private Browser app.\",\"code\":\"pyautogui.click(x=0.239, y=0.810)\",\"image\":\"a242d486-784f-4efa-9901-ab7d379bc92e.png\"},{\"action\":\"Click on the Facebook sharing option in the sharing dialog to share the Firefox Fast & Private Browser app via Facebook.\",\"code\":\"pyautogui.click(x=0.411, y=0.482)\",\"image\":\"602cf791-5c80-44c4-a638-65af92202b79.png\"},{\"action\":\"Terminate the current task with a success status as all required steps in the goal have been completed.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"c3232db1-7816-4213-89a1-6007cfb8f31c.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241122111412_f7ad406f-c412-4885-85f6-fec02cbfe39f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"fa6e92b116ac0a7e08b31e75b00143262a1bdfbf2d5ebfe18e91a6c508e8a452"}
{"messages":[{"role":"user","content":"Open OneNote, in the notebook named task go to the page titled draw, go to Draw, use the ruler, select the first of the existing pen to draw a straight line, in turn, insert the shape of the line, line arrow, line arrow double, no arrow, single arrow, double arrow, rectangle, oval, parallelogram, triangle and diamond, close the ruler."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the OneNote icon (purple icon with \\\"N\\\") in the Windows taskbar to launch the application.\",\"code\":\"pyautogui.click(x=0.532, y=0.966)\",\"image\":\"b489c382-6001-48ee-977f-bed4f9316149.png\"},{\"action\":\"Click on the \\\"draw\\\" page in the right navigation panel to navigate to that page.\",\"code\":\"pyautogui.click(x=0.947, y=0.411)\",\"image\":\"9d7423eb-c720-4d8c-a306-59c36ece90dc.png\"},{\"action\":\"Click on the Draw tab in the top ribbon menu, located between Insert and History tabs.\",\"code\":\"pyautogui.click(x=0.117, y=0.051)\",\"image\":\"42ea3332-0d1b-43de-a003-8e70badb8ab4.png\"},{\"action\":\"Click on the Ruler tool icon in the Stencils section of the ribbon to activate the ruler on the canvas.\",\"code\":\"pyautogui.click(x=0.429, y=0.104)\",\"image\":\"94e7d2c7-cdd0-4d06-8aca-b900d3e5264b.png\"},{\"action\":\"Click on the black pen tool in the Drawing Tools section of the ribbon to select it for drawing.\",\"code\":\"pyautogui.click(x=0.140, y=0.106)\",\"image\":\"385898ec-8e2a-4c14-be76-9360833f4f22.png\"},{\"action\":\"Position the cursor on the ruler's edge, click and drag from the upper left portion of the ruler down along its edge toward the bottom right to draw a straight line.\",\"code\":\"pyautogui.moveTo(x=0.219, y=0.492)\\npyautogui.dragTo(x=0.431, y=0.995, button='left')\",\"image\":\"d1771026-91b4-48d8-ba22-851dc651c095.png\"},{\"action\":\"Click on the Shapes button in the OneNote ribbon to access the shapes gallery.\",\"code\":\"pyautogui.click(x=0.564, y=0.112)\",\"image\":\"b9b47621-997f-4913-b579-b92e0fcd7709.png\"},{\"action\":\"Click on the line shape option in the Basic Shapes section of the dropdown menu to select it for drawing.\",\"code\":\"pyautogui.click(x=0.558, y=0.201)\",\"image\":\"07557d22-51d7-43e7-8385-eb09d2764f3f.png\"},{\"action\":\"Click and drag from a point in the lower left area of the canvas to a point in the upper right area to create a line shape.\",\"code\":\"pyautogui.moveTo(x=0.374, y=0.546)\\npyautogui.dragTo(x=0.554, y=0.410, button='left')\",\"image\":\"b2a65973-2d42-47a5-8c0b-e8f77e42abd6.png\"},{\"action\":\"Click on the line arrow shape in the shapes dropdown menu to select it for drawing.\",\"code\":\"pyautogui.click(x=0.569, y=0.200)\",\"image\":\"4d5da08e-9bc5-47dd-aa43-f8efc7c745de.png\"},{\"action\":\"Click in an empty area in the lower portion of the canvas, and drag diagonally upward to create a line arrow shape.\",\"code\":\"pyautogui.moveTo(x=0.394, y=0.671)\\npyautogui.dragTo(x=0.565, y=0.533, button='left')\",\"image\":\"21928fbb-d460-4104-ab1b-c8ef489909c1.png\"},{\"action\":\"Click on the Shapes dropdown button in the ribbon to access the shapes gallery.\",\"code\":\"pyautogui.click(x=0.557, y=0.094)\",\"image\":\"d3e29817-ee0a-4555-b049-3facfa32117b.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241117115609_b1e79dad-3e29-4127-9095-6a1fed5ecf88","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"681c58f1ac614bb93a2044451e32131284f3bc055bd7ce6bca96a51e5c04c021"}
{"messages":[{"role":"user","content":"Compare Adjustable Dumbbells for Home Gyms in February 2024, using Amazon website"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search box where it says \\\"Search Amazon\\\" to begin entering a search query.\",\"code\":\"pyautogui.click(x=0.246, y=0.144)\",\"image\":\"5615db7d-bd61-49e9-b7b6-7ff298b32328.png\"},{\"action\":\"Type \\\"adjustable drumbbells ideal for home gyms\\\" in the search box and press Enter to search for these products.\",\"code\":\"pyautogui.write(message='adjustable drumbbells ideal for home gyms')\\npyautogui.press('enter')\",\"image\":\"9b3c7176-a8f4-4016-8ea5-8d897fb76a9a.png\"},{\"action\":\"Click on the Amazon Basics checkbox in the Brands filter section to filter the search results by Amazon Basics products.\",\"code\":\"pyautogui.click(x=0.015, y=0.642)\",\"image\":\"cb3c6b7a-5980-4177-a96f-8d36ad429ccb.png\"},{\"action\":\"Scroll down on the search results page to view more product details and potentially additional products.\",\"code\":\"pyautogui.moveTo(x=0.302, y=0.810)\\npyautogui.scroll(-10)\",\"image\":\"44418f31-5652-4ade-b27d-65d974e4aaa8.png\"},{\"action\":\"Click on the \\\"Bowflex SelectTech 552 Dumbbells (Pair) and Stand Bundle\\\" product listing to view its detailed product page.\",\"code\":\"pyautogui.click(x=0.539, y=0.276)\",\"image\":\"21b31ce1-b178-4fdb-a673-71778d39d40e.png\"},{\"action\":\"Click on the image of the Bowflex SelectTech 552 Dumbbells (Pair) and Stand Bundle to view the detailed product page.\",\"code\":\"pyautogui.click(x=0.544, y=0.246)\",\"image\":\"01a933c8-bde7-4899-a876-75e381e45ce3.png\"},{\"action\":\"Scroll down at the current position to view more product details about the Bowflex SelectTech 552 Dumbbells bundle.\",\"code\":\"pyautogui.moveTo(x=0.483, y=0.771)\\npyautogui.scroll(-10)\",\"image\":\"3b9a1af3-8e23-4cfd-b832-aa9a726bf207.png\"},{\"action\":\"Click on the \\\"627 ratings\\\" link to view customer reviews of the Bowflex SelectTech 552 Dumbbells.\",\"code\":\"pyautogui.click(x=0.544, y=0.393)\",\"image\":\"8fa3bc69-a124-4a75-838c-a03a6cc0212a.png\"},{\"action\":\"Scroll down on the customer reviews page to view more detailed written reviews of the Bowflex SelectTech 552 Dumbbells.\",\"code\":\"pyautogui.moveTo(x=0.435, y=0.725)\\npyautogui.scroll(-10)\",\"image\":\"c311e1bd-595b-4a72-bff3-76bbb9b6deb5.png\"},{\"action\":\"Click on the \\\"About this item\\\" tab to view the product specifications and details.\",\"code\":\"pyautogui.click(x=0.268, y=0.124)\",\"image\":\"12005f97-abde-4711-b11b-c9412d0b5b13.png\"},{\"action\":\"Press Enter to execute the search for \\\"adjustable dumbbells ideal for home gyms\\\" and return to the search results page.\",\"code\":\"pyautogui.press('enter')\",\"image\":\"d22e6692-d5e3-4e27-820c-a1aa5b158d4d.png\"},{\"action\":\"Press the space key, then type \\\"brand core home fitness\\\" in the search box, and press Enter to search for Core Home Fitness branded adjustable dumbbells.\",\"code\":\"pyautogui.press('space')\\npyautogui.write(message='brand core home fitness')\\npyautogui.press('enter')\",\"image\":\"c25cfb98-f687-45bd-bc71-c500a7788ce5.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241126163440_05c1040d-b2af-4979-8237-8e9541263657","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"7c96a9ac116cc416cab31fa58c16a635a26bceccfb02ff54da6a9c173d528d6c"}
{"messages":[{"role":"user","content":"View the travel shop of VeronicaVisits on Expedia, check three items of her Niagara Falls accommodation collection, check Eurostars Monumental in her current picks, check availability from Nov to Nov 14, check room types and save the hotel to list, check the Youtube channel of  Veronica Visits"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the VeronicaVisits travel creator card to view her travel shop on Expedia.\",\"code\":\"pyautogui.click(x=0.109, y=0.4994)\",\"image\":\"27a5b1da-e16c-4800-a213-13f0fa98d715.png\"},{\"action\":\"Scroll down on the page to view more of VeronicaVisits' travel shop content.\",\"code\":\"pyautogui.moveTo(x=0.623, y=0.25)\\npyautogui.scroll(-11)\",\"image\":\"f4b377a6-6af5-44fc-b97b-2a0da30851f1.png\"},{\"action\":\"Click on the Niagara Falls destination card to view the accommodation collection for that location.\",\"code\":\"pyautogui.click(x=0.5004, y=0.5225)\",\"image\":\"2d23ad54-5069-4b97-90a3-40159d3e73df.png\"},{\"action\":\"Click on the right arrow navigation button to view the next image in the Sheraton Fallsview Hotel gallery.\",\"code\":\"pyautogui.click(x=0.7586, y=0.5381)\",\"image\":\"ff57321e-d755-41da-8403-aa24990eaac2.png\"},{\"action\":\"Click on the right arrow navigation button to view the next image in the Marriott Niagara Falls Fallsview Hotel & Spa gallery.\",\"code\":\"pyautogui.click(x=0.7574, y=0.5381)\",\"image\":\"e971d954-0793-4f18-95bc-04a93271705f.png\"},{\"action\":\"Click the X button in the top-left corner of the image gallery to close the detailed view of the Embassy Suites by Hilton Niagara Falls Fallsview.\",\"code\":\"pyautogui.click(x=0.0238, y=0.1688)\",\"image\":\"63f97b0b-d0bd-4c03-9225-af2e30da1439.png\"},{\"action\":\"Click on the Eurostars Monumental hotel card in the Barcelona section of the \\\"Current picks\\\" to view its details.\",\"code\":\"pyautogui.click(x=0.7695, y=0.3669)\",\"image\":\"3585f610-35bc-4370-aff4-66e501595fd9.png\"},{\"action\":\"Click on the blue \\\"Search\\\" button to find available room types and rates for the Eurostars Monumental hotel for the dates November 12-14.\",\"code\":\"pyautogui.click(x=0.9141, y=0.4462)\",\"image\":\"f1ddf6bc-1ae6-4a77-b120-65dfb2870f3d.png\"},{\"action\":\"Click the blue \\\"Continue\\\" button in the Eurostars Monumental hotel listing to view detailed room options and availability.\",\"code\":\"pyautogui.click(x=0.7723, y=0.8144)\",\"image\":\"29f0eb23-e2a3-4730-8d80-28c02714ad8a.png\"},{\"action\":\"Click on the \\\"Rooms\\\" tab to view the available room types at the Eurostars Monumental hotel for the selected dates.\",\"code\":\"pyautogui.click(x=0.1941, y=0.82)\",\"image\":\"cf1ee72c-09bd-4fbd-bdbb-0368ffb01357.png\"},{\"action\":\"Click on the \\\"Save\\\" button in the upper right corner of the page to save the Eurostars Monumental hotel to your list.\",\"code\":\"pyautogui.click(x=0.7855, y=0.1656)\",\"image\":\"50d450c2-dae2-4a98-84de-1814bb3f786c.png\"},{\"action\":\"Click on the \\\"Creator Shops - Traveler Shop\\\" browser tab to return to VeronicaVisits' travel shop page.\",\"code\":\"pyautogui.click(x=0.3402, y=0.0181)\",\"image\":\"9b995b91-0296-4722-8b7b-56a657c727dd.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241105152819_18a034b6-e0de-41b3-9b9a-95cd64afa387","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"87643b293da2fe5a5c49fc2509dca296aa628b777218123585e556003442ef92"}
{"messages":[{"role":"user","content":"Add Slackbot reponse on Slack, when someone says 'when does the project launch?', Slackbot responds 'Dec 3', test out the response"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the dropdown arrow next to the \\\"Yanikovski\\\" workspace name in the left sidebar to access workspace settings.\",\"code\":\"pyautogui.click(x=0.1469, y=0.2137)\",\"image\":\"e56917a0-4ecb-4f18-bdd3-25bc5878e6a5.png\"},{\"action\":\"Click on \\\"Customise workspace\\\" in the submenu to access workspace customization settings.\",\"code\":\"pyautogui.click(x=0.393, y=0.4381)\",\"image\":\"d3df0d1f-41f3-4bab-94bd-f8f516b798c4.png\"},{\"action\":\"Click on the \\\"Slackbot\\\" tab in the customization navigation menu to access Slackbot response settings.\",\"code\":\"pyautogui.click(x=0.232, y=0.4406)\",\"image\":\"e9367fcd-eb82-4591-9505-6c6cb76010b4.png\"},{\"action\":\"Click on the \\\"Add new response\\\" button to create a new Slackbot automated response.\",\"code\":\"pyautogui.click(x=0.759, y=0.63)\",\"image\":\"d995cb59-67f1-4940-b61e-472f51a8d22d.png\"},{\"action\":\"Click in the text input field under \\\"When someone says\\\" to start typing the trigger phrase.\",\"code\":\"pyautogui.click(x=0.5535, y=0.3862)\",\"image\":\"5d17ab61-ef57-4a67-8de5-6f34d89f6cc0.png\"},{\"action\":\"Type \\\"when does the project launch?\\\" in the \\\"When someone says\\\" input field.\",\"code\":\"pyautogui.write(message='when does the project launch?')\",\"image\":\"8729a155-672d-4719-9e44-d9285e959aef.png\"},{\"action\":\"Click in the empty \\\"Slackbot responds\\\" text field to position the cursor for typing the response.\",\"code\":\"pyautogui.click(x=0.466, y=0.5669)\",\"image\":\"b457e820-3178-4c4d-8801-2fdbd2b66f05.png\"},{\"action\":\"Type \\\"Dec 3\\\" in the Slackbot responds text field.\",\"code\":\"pyautogui.write(message='Dec 3')\",\"image\":\"0907d355-31a8-40b9-b75e-7fe9898664e4.png\"},{\"action\":\"Click the \\\"Save\\\" button to finalize and store the Slackbot automated response configuration.\",\"code\":\"pyautogui.click(x=0.652, y=0.8306)\",\"image\":\"90be7c04-8041-4a53-84c3-4e3999c2a82e.png\"},{\"action\":\"Click on the #sales channel in the left sidebar to navigate to that channel's conversation view.\",\"code\":\"pyautogui.click(x=0.1152, y=0.5537)\",\"image\":\"8cbe6e80-6455-41ea-895c-c86bf12440bc.png\"},{\"action\":\"Click in the message input field at the bottom of the #sales channel to begin typing a message that will test the Slackbot response.\",\"code\":\"pyautogui.click(x=0.4727, y=0.8144)\",\"image\":\"5afee797-3b65-487e-887c-d2a7270ab60e.png\"},{\"action\":\"Type \\\"when does the project launch?\\\" in the message field and press Enter to send it.\",\"code\":\"pyautogui.write(message='when does the project launch?')\\npyautogui.press('enter')\",\"image\":\"e5f47983-7edc-4e6b-947c-ae885bf9d794.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241115150437_898faa24-1b1d-442a-9ef6-3b49f061c83f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"f97e72cd52d1be92b0dd00f77d78f1f4f2501382f9dc59a2ce246e3a0a2e0670"}
{"messages":[{"role":"user","content":"Please convert the two line math equations in the opened PDF file into latex code and save it into file \"math.txt\" on Desktop. I heard Mathpix could perform this work, you can resort to this tool."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the Google Chrome icon in the macOS dock to launch the web browser.\",\"code\":\"pyautogui.click(x=0.1731, y=0.9657)\",\"image\":\"e05f9e14-5fa9-4b3d-a446-7b7e55a9202d.png\"},{\"action\":\"Type \\\"mathpix\\\" in the Google search box and press Enter to search for the Mathpix website.\",\"code\":\"pyautogui.write(message='mathpix')\\npyautogui.press('enter')\",\"image\":\"f9713633-d2eb-4bee-89d4-6e31073870e5.png\"},{\"action\":\"Click on the main heading \\\"Mathpix: Document conversion done right\\\" in the search results to navigate to the Mathpix website.\",\"code\":\"pyautogui.click(x=0.2875, y=0.3929)\",\"image\":\"1dfbbd2a-9c8f-4746-979c-4de5539dfaa0.png\"},{\"action\":\"Click on the \\\"Try for free\\\" button to begin using Mathpix's services.\",\"code\":\"pyautogui.click(x=0.1277, y=0.7014)\",\"image\":\"499b78a5-a822-4d6e-a792-472f035bdfc8.png\"},{\"action\":\"Click on the \\\"Snip\\\" option to activate the screen capture tool for converting mathematical equations to LaTeX.\",\"code\":\"pyautogui.click(x=0.7805, y=0.4065)\",\"image\":\"50aad369-9e29-4926-a2f3-79235531309c.png\"},{\"action\":\"Click on \\\"Create Snip by Image Upload\\\" option in the dropdown menu to proceed with uploading the mathematical equations from the PDF.\",\"code\":\"pyautogui.click(x=0.7464, y=0.4958)\",\"image\":\"202f9eb1-b333-4dfd-af2d-f303d081483c.png\"},{\"action\":\"Press Command+Shift+5 to open the macOS screenshot tool.\",\"code\":\"pyautogui.hotkey(['cmd', 'shift', '5'])\",\"image\":\"750d9b46-1644-47e2-9321-90595df270a6.png\"},{\"action\":\"Click on the \\\"拖拽\\\" (drag) button in the macOS screenshot toolbar to activate the area selection capture mode.\",\"code\":\"pyautogui.click(x=0.6238, y=0.9339)\",\"image\":\"44eb0d46-edfb-4375-8c76-ea9fb9d80c3a.png\"},{\"action\":\"Click on the \\\"截屏2024.10.16 13.23.20.png\\\" file in the file selection dialog to select it for uploading to Mathpix.\",\"code\":\"pyautogui.click(x=0.3049, y=0.3733)\",\"image\":\"3ea33d10-13eb-437a-8d5f-de5c15043c8c.png\"},{\"action\":\"Click the \\\"打开\\\" (Open) button in the bottom right corner of the file selection dialog to upload the selected screenshot to Mathpix for processing.\",\"code\":\"pyautogui.click(x=0.8303, y=0.7201)\",\"image\":\"0ffe442a-8800-46c3-a629-92046615fecd.png\"},{\"action\":\"Triple-click on the LaTeX code in the text field at the top of the Mathpix interface to select all of the text.\",\"code\":\"computer.tripleClick(x=0.6354, y=0.1903)\",\"image\":\"57e8cc61-ecd6-409c-9ff5-3108677895ed.png\"},{\"action\":\"Click on the \\\"log P\\\" text within the mathematical equation in the lower right portion of the display area.\",\"code\":\"pyautogui.click(x=0.7953, y=0.6651)\",\"image\":\"a6b25c0b-6296-492c-8411-cef4f3e8b343.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241016132853_a7c9ff89-1856-445e-a3b3-232d6e2c9602","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"9ccc501e25481b85976389305a62ba3a1420672e31f8dc03652455a206f9f68d"}
{"messages":[{"role":"user","content":"In the YouTube playlist, select the \"Slack\" playlist and copy the playlist's URL. On Google, search for a tool using the keyword \"YouTube playlist URL\" to extract the URLs of each video in the playlist. Choose the NimTools website, paste the playlist URL, and proceed with the extraction."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Playlists\\\" option in the YouTube sidebar navigation under the \\\"You\\\" section.\",\"code\":\"pyautogui.click(x=0.0637, y=0.3257)\",\"image\":\"04df08cf-c0a8-46f5-a11f-870509808334.png\"},{\"action\":\"Click on the \\\"View full playlist\\\" link under the Slack playlist.\",\"code\":\"pyautogui.click(x=0.5207, y=0.3729)\",\"image\":\"0395ff3e-466a-4091-8f0b-fb8b8ef0a171.png\"},{\"action\":\"Click in the address bar of the browser to give it focus, positioning the cursor at the beginning of the URL.\",\"code\":\"pyautogui.click(x=0.4863, y=0.0215)\",\"image\":\"19cb3e4c-9f54-4c85-9c85-8e9a96e8d520.png\"},{\"action\":\"Type \\\"goog\\\" in the address bar and press Enter to navigate to Google.\",\"code\":\"pyautogui.write(message='goog')\\npyautogui.press('enter')\\npyautogui.write(message='youtube playlist url')\\npyautogui.press('enter')\",\"image\":\"1d1be744-d727-4771-bdaf-52709f28e3d0.png\"},{\"action\":\"Scroll down on the Google search results page to view more results that might include the NimTools website for YouTube playlist URL extraction.\",\"code\":\"pyautogui.moveTo(x=0.4723, y=0.3736)\\npyautogui.scroll(-11)\",\"image\":\"d94c42ad-9331-437a-b36c-f648837f5a03.png\"},{\"action\":\"Click on the \\\"YouTube Playlist Video Link Extractor\\\" result from NimTools.com to access the tool for extracting individual video links from YouTube playlists.\",\"code\":\"pyautogui.click(x=0.1184, y=0.7583)\",\"image\":\"dca98b31-6d70-4c1b-a33e-b27219d4149d.png\"},{\"action\":\"Click on the input field where YouTube playlist URLs should be entered, which currently shows a placeholder example like \\\"youtube.com/playlist?list=ID\\\".\",\"code\":\"pyautogui.click(x=0.3129, y=0.6861)\",\"image\":\"152acc5b-7819-4437-8ad1-60bbcc38802a.png\"},{\"action\":\"Press Ctrl+V to paste the previously copied YouTube playlist URL into the input field.\",\"code\":\"pyautogui.hotkey(['ctrl', 'v'])\",\"image\":\"7090d86e-fde0-41be-a0b8-c8f207a4bfd8.png\"},{\"action\":\"Click the blue \\\"Get YouTube Playlist Length\\\" button to submit the playlist URL for processing.\",\"code\":\"pyautogui.click(x=0.3984, y=0.8167)\",\"image\":\"e35b0207-1c67-431e-98cc-cc25caa4c89f.png\"},{\"action\":\"Terminate the task with success status as all steps of the goal have been completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"71d455be-7849-4a0d-8023-e0e6c62aeb45.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241028211759_18d7064d-b75b-4dd3-9f6e-905dcdbf0db7","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"04366920721fa44297c665fa46c3d3980c7e858121b91d180052179b4f94464b"}
{"messages":[{"role":"user","content":"In the MS Bing settings msn interface, click Content Settings to open all options"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Content Settings\\\" tab in the navigation menu.\",\"code\":\"pyautogui.click(x=0.300, y=0.157)\",\"image\":\"d2e7de51-1ac3-408b-b2b1-8139c3619d69.png\"},{\"action\":\"Scroll down slightly in the Content Settings page to view more options below the Sports section.\",\"code\":\"pyautogui.moveTo(x=0.647, y=0.883)\\npyautogui.scroll(-10)\",\"image\":\"8f138bb1-4a5a-4087-9dcb-a31aff552886.png\"},{\"action\":\"Click on the toggle switch for \\\"Show the traffic card in my feed\\\" to enable traffic information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.655, y=0.833)\",\"image\":\"0519d514-f8a2-4957-b8aa-c02ac0ace61f.png\"},{\"action\":\"Click on the toggle switch for \\\"Show the shopping card in my feed\\\" to enable shopping information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.658, y=0.940)\",\"image\":\"76754562-2269-44ff-a321-c0e1a7e106e6.png\"},{\"action\":\"Scroll down slightly in the Content Settings page to view more options below the Shopping section.\",\"code\":\"pyautogui.moveTo(x=0.639, y=0.917)\\npyautogui.scroll(-10)\",\"image\":\"6415a157-eb87-4320-8bbc-e2addf00933c.png\"},{\"action\":\"Click on the toggle switch for \\\"Show recipes card in my feed\\\" to enable recipes information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.649, y=0.866)\",\"image\":\"3436b9bc-2644-4f90-b019-f9f46c633c0f.png\"},{\"action\":\"Scroll down in the Content Settings page to view more options below the Recipes section.\",\"code\":\"pyautogui.moveTo(x=0.649, y=0.864)\\npyautogui.scroll(-10)\",\"image\":\"655f17f4-bdae-4cba-83ad-1b5bab88535e.png\"},{\"action\":\"Click on the toggle switch for \\\"Show health card in my feed\\\" to enable health information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.655, y=0.792)\",\"image\":\"db241937-8cba-4ed6-a6bf-efaa1347d566.png\"},{\"action\":\"Click on the toggle switch for \\\"Show autos card in my feed\\\" to enable automotive information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.657, y=0.901)\",\"image\":\"2bf17e3d-0c49-4ff9-9d2d-92f924ccd681.png\"},{\"action\":\"Scroll down slightly in the Content Settings page to view the Real Estate section and other options below the Autos Marketplace section.\",\"code\":\"pyautogui.moveTo(x=0.657, y=0.897)\\npyautogui.scroll(-10)\",\"image\":\"ab057b9e-7c2e-4239-9d4f-6018a8fa9392.png\"},{\"action\":\"Click on the toggle switch for \\\"Show real estate card in my feed\\\" to enable real estate information in the MSN feed.\",\"code\":\"pyautogui.click(x=0.656, y=0.904)\",\"image\":\"fc1ec254-1ae4-4b81-9205-86d4a2ebb677.png\"},{\"action\":\"Scroll down slightly in the Content Settings page to view more options below the Real Estate section.\",\"code\":\"pyautogui.moveTo(x=0.663, y=0.881)\\npyautogui.scroll(-10)\",\"image\":\"263a8122-7434-45a1-b2a9-44f9698fcb35.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241113201801_36df1105-2c11-420b-be35-787c3c2d056f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"f5ccb82999c755e11e36d1697f984c7d82269457e606c7796010520e64a8ecd9"}
{"messages":[{"role":"user","content":"Visit the San Francisco Japanese Visa official homepage, check the guide video in youtube and like it"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the search bar to begin typing a search query for the San Francisco Japanese Visa official homepage.\",\"code\":\"pyautogui.click(x=0.324, y=0.442)\",\"image\":\"ab4db460-6646-40b2-8b9f-670d4daabef3.png\"},{\"action\":\"Type \\\"japanese \\\" in the Google search bar to begin searching for the Japanese visa information.\",\"code\":\"pyautogui.write(message='japanese ')\",\"image\":\"3e846bf6-a229-469e-b35e-3df37c21e976.png\"},{\"action\":\"Click on the \\\"japanese visa san francisco\\\" search suggestion from the dropdown menu to perform this search.\",\"code\":\"pyautogui.click(x=0.337, y=0.512)\",\"image\":\"f2df2d9b-62cc-4f98-a541-607a0b79e4d9.png\"},{\"action\":\"Click on the \\\"VISA INFORMATION | Consulate-General of Japan in San Francisco\\\" link at the top of the search results to visit the official website.\",\"code\":\"pyautogui.click(x=0.313, y=0.368)\",\"image\":\"c308cb7e-f094-4728-9683-466a736bace5.png\"},{\"action\":\"Scroll down slightly on the visa information page to see more content about Japanese visa requirements.\",\"code\":\"pyautogui.moveTo(x=0.313, y=0.368)\\npyautogui.scroll(-10)\",\"image\":\"7f62790b-d101-4349-add3-f904261c7fb5.png\"},{\"action\":\"Click on the YouTube link next to \\\"How to apply eVISA\\\" text to open the guide video about applying for Japanese visas.\",\"code\":\"pyautogui.click(x=0.172, y=0.709)\",\"image\":\"be317a07-bfe0-4800-ba38-1dbd46f6afcb.png\"},{\"action\":\"Click the play/pause button to pause the video playback.\",\"code\":\"pyautogui.click(x=0.654, y=0.673)\",\"image\":\"68a0803d-a5e8-46e3-9b18-6144f77106f7.png\"},{\"action\":\"Click the like (thumbs up) button below the video that currently shows 71 likes.\",\"code\":\"pyautogui.click(x=0.402, y=0.875)\",\"image\":\"8cc9293f-3eee-44a8-a249-407d56ddd835.png\"},{\"action\":\"Terminate the task with a success status, as all required actions have been completed successfully.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"fdc50342-159a-4796-9eb3-f7bb23e53c74.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240922002743_e1700d65-7e50-497b-9a50-473bdadfb82f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"01e9e0548b0fdca5daf9651fc18a96839861cbdb2a6f78e3f43bab213613abcb"}
{"messages":[{"role":"user","content":"Search for emails containing 'Jolly' in Outlook, locate the email from 8/28, and review the TA arrangement"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"+\\\" button in the top-right corner of the browser to open a new tab.\",\"code\":\"pyautogui.click(x=0.555, y=0.026)\",\"image\":\"7cae1c44-7b20-4418-98eb-7cc30c15a811.png\"},{\"action\":\"Type \\\"outlo\\\" in the Google search bar and press Enter twice to navigate to Outlook.\",\"code\":\"pyautogui.write(message='outlo')\\npyautogui.press('enter', presses=2)\",\"image\":\"72d431da-7019-437c-b9c9-d0b1de192dae.png\"},{\"action\":\"Click on the search bar at the top of the Outlook interface that contains the \\\"搜索\\\" (Search) placeholder text to begin searching for emails.\",\"code\":\"pyautogui.click(x=0.236, y=0.155)\",\"image\":\"f808f99c-f1ee-4b31-bc41-32b811294686.png\"},{\"action\":\"Type \\\"Jolly\\\" in the search field and press Enter twice to execute the search.\",\"code\":\"pyautogui.write(message='Jolly')\\npyautogui.press('enter', presses=2)\",\"image\":\"0a2eac3a-874d-4238-ae9a-a625eec14af3.png\"},{\"action\":\"Scroll down in the email search results to view more emails containing 'Jolly'\",\"code\":\"pyautogui.moveTo(x=0.255, y=0.585)\\npyautogui.scroll(-10)\",\"image\":\"806ed734-505e-4f01-aec0-8e4fdbf7af97.png\"},{\"action\":\"Click on the email from xywang626@connect.hku.hk dated 8/28 with subject line starting with \\\"COMP2501A mee...\\\" to open and view its contents.\",\"code\":\"pyautogui.click(x=0.263, y=0.591)\",\"image\":\"c7b42ed5-e61f-40a7-8ed8-f4fa40087be9.png\"},{\"action\":\"Scroll down in the email content area to see if there is more information about the TA arrangement below the currently visible part of the email.\",\"code\":\"pyautogui.moveTo(x=0.635, y=0.554)\\npyautogui.scroll(10)\",\"image\":\"cee197d6-754c-450d-9816-344de7b7085e.png\"},{\"action\":\"End the task with success status as we have completed our goal of searching for emails containing 'Jolly', locating the email from 8/28, and reviewing the TA arrangement details.\",\"code\":\"computer.terminate(status='success')\",\"image\":\"428e0e2f-6f55-4d69-b212-adeb34d4c6a4.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20240922002701_3a9b3752-d1d5-4d78-81f6-e2f3c304e509","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"526b75cd52b8b984ff5523f8a38f4a3487aa10c5c50b242f809017bdf339d876"}
{"messages":[{"role":"user","content":"Asking AI for the best games for Mac, then proceeding to check out War Thunder. Then I check out alternative in app store and and i convert 14.85 GB to gigabits"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the red close button in the top-left corner of the AgentNet Tool application window to close the application.\",\"code\":\"pyautogui.click(x=0.025, y=0.060)\",\"image\":\"3fc80887-399a-4955-9a22-d4ef111f6711.png\"},{\"action\":\"Click on the Chrome icon in the dock at the bottom of the screen to launch the web browser.\",\"code\":\"pyautogui.click(x=0.594, y=0.983)\",\"image\":\"ba3819c7-b9a3-40d1-924f-a02cda0cf27c.png\"},{\"action\":\"Click on the ChatGPT icon in the dock at the bottom of the screen.\",\"code\":\"pyautogui.click(x=0.594, y=0.982)\",\"image\":\"d8147258-054d-47ec-83c4-ba62f550d3c1.png\"},{\"action\":\"Type \\\"hey, can you give me a good advice on games for mac in app store?\\\" in the ChatGPT message input field and press Enter to send the message.\",\"code\":\"pyautogui.write(message='hey, can you give me a good advice on games for mac in app store?')\\npyautogui.press('enter')\",\"image\":\"d5914f01-73a0-44fe-8e69-d31506e03210.png\"},{\"action\":\"Scroll down in the ChatGPT conversation to view more game recommendations.\",\"code\":\"pyautogui.moveTo(x=0.297, y=0.509)\\npyautogui.scroll(-10)\",\"image\":\"5fb1db44-1fb8-4bc1-8176-3d2cc6857ce4.png\"},{\"action\":\"Click and drag to select the text \\\"War Thunder\\\" in the ChatGPT response to highlight it before asking a follow-up question about this specific game.\",\"code\":\"pyautogui.moveTo(x=0.213, y=0.546)\\npyautogui.dragTo(x=0.273, y=0.551, button='left')\",\"image\":\"fe2eaef4-94f9-47ce-89f1-49713e1015be.png\"},{\"action\":\"Copy the selected text \\\"War Thunder\\\" using the Command+C keyboard shortcut.\",\"code\":\"pyautogui.hotkey(['cmd', 'c'])\",\"image\":\"83d47dbf-e617-4036-9561-3ddef98fd4bc.png\"},{\"action\":\"Press Command+V to paste \\\"War Thunder\\\" from the clipboard into the Safari search bar.\",\"code\":\"pyautogui.hotkey(['cmd', 'v'])\",\"image\":\"eb298862-5f70-41af-8719-c9875073e6e8.png\"},{\"action\":\"Press the Enter key to search for \\\"War Thunder\\\" in Safari.\",\"code\":\"pyautogui.press('enter')\",\"image\":\"2a4a81df-4dd9-483a-a0c3-bf4be063c1b2.png\"},{\"action\":\"Click on the \\\"zadarmo\\\" text in the sponsored result link that says \\\"Hrej zadarmo. Připoj se - PC hra zadarmo: War Thunder\\\"\",\"code\":\"pyautogui.click(x=0.338, y=0.361)\",\"image\":\"b3824cdd-f3de-47d1-b2be-91124baac58c.png\"},{\"action\":\"Click on the search bar in the App Store sidebar to activate it for searching.\",\"code\":\"pyautogui.click(x=0.189, y=0.142)\",\"image\":\"dac21c33-1e69-48ab-9257-197481901bde.png\"},{\"action\":\"Type \\\"war thunder\\\" in the search bar and press Enter to search for the game in the App Store.\",\"code\":\"pyautogui.write(message='war thunder')\\npyautogui.press('enter')\",\"image\":\"12568d3e-1b88-44f9-b4e2-e7e0ddbebf37.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241018003607_be8ff233-27c4-4b09-a060-9f2ded016a86","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"71188a3050b085e53e04d124c954dac68436244dd1c3f53280e9f007d8925a9a"}
{"messages":[{"role":"user","content":"Log in to the GitHub website, create a new repository named 'image', add a readme file, and set it to privacy mode. Import the test. py file stored locally into this repository"},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the \\\"Sign in\\\" button in the upper right corner of the GitHub homepage to access the login page.\",\"code\":\"pyautogui.click(x=0.911, y=0.186)\",\"image\":\"5a23b5cc-2dba-43f3-b5bc-fc5b7eb1dda5.png\"},{\"action\":\"Click on the \\\"+\\\" icon (plus sign) in the top navigation bar to open the creation menu for creating a new repository.\",\"code\":\"pyautogui.click(x=0.910, y=0.094)\",\"image\":\"c913f792-e346-4940-9aaf-88cdb274c732.png\"},{\"action\":\"Click on the \\\"New repository\\\" option in the dropdown menu that appears after clicking the \\\"+\\\" icon.\",\"code\":\"pyautogui.click(x=0.879, y=0.139)\",\"image\":\"b8349aa2-dc5e-4442-a095-ae159b71315b.png\"},{\"action\":\"Click on the Repository name input field to prepare for entering \\\"image\\\" as the repository name.\",\"code\":\"pyautogui.click(x=0.450, y=0.368)\",\"image\":\"72a7a4cf-3e03-4c17-9fa4-5a4fae9cf911.png\"},{\"action\":\"Type \\\"image\\\" in the repository name field.\",\"code\":\"pyautogui.write(message='image')\",\"image\":\"19f4dcfc-ce05-4dbf-8187-236433fc1f8a.png\"},{\"action\":\"Scroll down the repository creation page to view more options like the \\\"Add a README file\\\" checkbox and \\\"Private\\\" repository setting that are currently not fully visible.\",\"code\":\"pyautogui.moveTo(x=0.288, y=0.472)\\npyautogui.scroll(-10)\",\"image\":\"8c565a97-2ce3-4915-a516-c5402eb6d2e4.png\"},{\"action\":\"Click on the checkbox for \\\"Add a README file\\\" to enable the creation of a README file with the repository.\",\"code\":\"pyautogui.click(x=0.356, y=0.489)\",\"image\":\"9ee718e4-2de9-4fbf-8476-bd47ff5dd4e2.png\"},{\"action\":\"Scroll down the repository creation page to view the \\\"Create repository\\\" button at the bottom of the form.\",\"code\":\"pyautogui.moveTo(x=0.401, y=0.457)\\npyautogui.scroll(-10)\",\"image\":\"af1937a3-a713-44d7-80d8-e99821789458.png\"},{\"action\":\"Scroll down the repository creation page to reveal the \\\"Create repository\\\" button at the bottom of the form.\",\"code\":\"pyautogui.moveTo(x=0.264, y=0.521)\\npyautogui.scroll(-10)\",\"image\":\"ca2f7af9-bd10-4761-8b49-f0bed67fa920.png\"},{\"action\":\"Click on the \\\".gitignore template: None\\\" dropdown to open the list of available .gitignore templates.\",\"code\":\"pyautogui.click(x=0.380, y=0.531)\",\"image\":\"3d88ddff-e2a2-46ca-be9d-587ca43372b3.png\"},{\"action\":\"Click on the \\\"ActionScript\\\" option in the .gitignore template dropdown menu.\",\"code\":\"pyautogui.click(x=0.358, y=0.706)\",\"image\":\"3af5b150-ff7d-4572-a6e0-2c9f1ba081d4.png\"},{\"action\":\"Click on the \\\"Create repository\\\" button to create the new repository with the configured settings.\",\"code\":\"pyautogui.click(x=0.643, y=0.784)\",\"image\":\"319949d9-c4c2-4d1e-b129-5870c7cd4e8e.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241019223607_eb14d63b-4176-460d-856b-66faa462498f","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"b02cc49cf7e4876a19e57ea8378637f3f6efeaa67deaef8819f4af16dd7c7f85"}
{"messages":[{"role":"user","content":"In the United Vacations Deals page, set the “Featured vacation deals” hotel sorting order to “Hotel Name (Ascending).” Open the hotel “Aston Waikiki Sunset” and set the departure airport to “San Francisco,” the “Number of Nights” to 7, and the departure date to (Dec 8, 2024). Scroll down to “Customize Your Vacation” and set the departure location to “Ithaca, New York (ITH)” and the destination to “San Francisco, California (SFO).” Then, search."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Scroll down on the page to view more of the Featured vacation deals section.\",\"code\":\"pyautogui.moveTo(x=0.779, y=0.501)\\npyautogui.scroll(-10)\",\"image\":\"cd89ad19-2cee-478c-a9d3-cf7c5d12d72e.png\"},{\"action\":\"Click on the \\\"Recommended\\\" dropdown menu next to \\\"Sort by\\\" to view sorting options.\",\"code\":\"pyautogui.click(x=0.792, y=0.209)\",\"image\":\"60257ee6-c976-479c-95b0-b96cce1fdf78.png\"},{\"action\":\"Click on the \\\"Hotel Name (Ascending)\\\" option in the sort dropdown menu.\",\"code\":\"pyautogui.click(x=0.772, y=0.270)\",\"image\":\"feeb029b-7a30-410b-92f4-c36faa824206.png\"},{\"action\":\"Click on the \\\"Aston Waikiki Sunset\\\" hotel name to open its detailed page.\",\"code\":\"pyautogui.click(x=0.585, y=0.321)\",\"image\":\"17e11b3b-6f34-476e-98f1-e491851c8038.png\"},{\"action\":\"Scroll down slightly on the hotel details page to view more booking options.\",\"code\":\"pyautogui.moveTo(x=0.848, y=0.716)\\npyautogui.scroll(-10)\",\"image\":\"3f0e16d0-b60c-4bcd-869a-8e03a73e4032.png\"},{\"action\":\"Click on the departure airport dropdown menu that currently displays \\\"Wichita\\\" to open the list of airport options.\",\"code\":\"pyautogui.click(x=0.742, y=0.589)\",\"image\":\"3f308e17-4d59-4a35-a6f2-38ff04ab9bd6.png\"},{\"action\":\"Click on \\\"San Francisco\\\" in the departure airport dropdown menu to select it as the departure airport.\",\"code\":\"pyautogui.click(x=0.720, y=0.297)\",\"image\":\"76fb0b4b-3534-4608-992b-3f95ae70b9d1.png\"},{\"action\":\"Scroll down slightly to better view the \\\"Number of Nights\\\" selection area and the departure date calendar section below it.\",\"code\":\"pyautogui.moveTo(x=0.797, y=0.568)\\npyautogui.scroll(-10)\",\"image\":\"b9d7829e-12e6-4329-87da-77073926306d.png\"},{\"action\":\"Click on the \\\"7\\\" option in the Number of Nights selection area to change the stay duration from 5 nights to 7 nights.\",\"code\":\"pyautogui.click(x=0.824, y=0.572)\",\"image\":\"f680c3a5-ad8b-4880-b15a-83a22f00a3d5.png\"},{\"action\":\"Click on December 8 in the calendar to select it as the departure date.\",\"code\":\"pyautogui.click(x=0.681, y=0.800)\",\"image\":\"0af0b669-b5b6-4679-b723-09c9d3056032.png\"},{\"action\":\"Scroll down on the page to view the \\\"Customize Your Vacation\\\" section that is currently not visible on the screen.\",\"code\":\"pyautogui.moveTo(x=0.855, y=0.589)\\npyautogui.scroll(-10)\",\"image\":\"c0557b00-2343-4cbc-89ba-a09b7f7bb8fc.png\"},{\"action\":\"Click on the \\\"Customize Your Vacation\\\" text/button to expand the customization options.\",\"code\":\"pyautogui.click(x=0.710, y=0.719)\",\"image\":\"4da9c6f2-1ff2-4a0a-91f6-c25ab41f11d7.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241205183507_2ea2c275-90fb-4679-bae3-6a0fd8c33183","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"1509056d018cc0870e596b5ee2d372fcd274ae1c9ce08d8f477529544dabf0cd"}
{"messages":[{"role":"user","content":"On the MS Bing website, click Personal Account to manage my account, click Services and Subscriptions, and click Demo to view the price and subscription process."},{"role":"assistant","content":"{\"action_sequence\":[{\"action\":\"Click on the bell notification icon in the top-right corner of the page, next to the user's email address.\",\"code\":\"pyautogui.click(x=0.7516, y=0.0917)\",\"image\":\"288fb884-4956-4b5d-be55-c52a86a44a15.png\"},{\"action\":\"Click on the \\\"Microsoft account\\\" text below the email address in the account dropdown menu.\",\"code\":\"pyautogui.click(x=0.8099, y=0.2343)\",\"image\":\"d6315676-6e20-4b09-8646-07553d37be92.png\"},{\"action\":\"Click on the \\\"Services & subscriptions\\\" option in the left navigation menu to access the services and subscriptions page.\",\"code\":\"pyautogui.click(x=0.1005, y=0.3231)\",\"image\":\"017602e8-3e0b-4c11-bddf-e1b8c6b9d4a1.png\"},{\"action\":\"Click on the \\\"Try for free\\\" button in the \\\"Your first month is free\\\" section to view the subscription details and pricing information.\",\"code\":\"pyautogui.click(x=0.3484, y=0.3981)\",\"image\":\"3eb4832e-4372-420c-9d9b-1847db6476fb.png\"},{\"action\":\"Click on the \\\"Buy now\\\" button for the Microsoft 365 Family subscription plan.\",\"code\":\"pyautogui.click(x=0.4229, y=0.6019)\",\"image\":\"5bb78161-189e-4f3b-9b3a-8d7012f9cd11.png\"},{\"action\":\"Click on the \\\"Add a new payment method\\\" field to proceed with viewing the payment options in the subscription process.\",\"code\":\"pyautogui.click(x=0.3609, y=0.3741)\",\"image\":\"d5e57f4c-ad1d-4306-82ea-8c19b3e4722b.png\"},{\"action\":\"Click the \\\"Cancel\\\" button at the bottom right of the payment method selection dialog to exit the subscription process.\",\"code\":\"pyautogui.click(x=0.6911, y=0.7889)\",\"image\":\"8605b91a-3903-4b9d-885d-d78be0ecabf4.png\"},{\"action\":\"Click the X button in the top-right corner of the Microsoft 365 subscription dialog to close it and return to the Services & subscriptions page.\",\"code\":\"pyautogui.click(x=0.7016, y=0.2389)\",\"image\":\"180ce927-85ed-4048-8e44-7c7fad751dd3.png\"},{\"action\":\"Click the \\\"Join Now\\\" button in the Xbox Game Pass section to view the pricing and subscription details for Game Pass.\",\"code\":\"pyautogui.click(x=0.3542, y=0.7065)\",\"image\":\"12878e69-04bc-4e3d-bbd2-80704bb03389.png\"},{\"action\":\"Click on the \\\"Console gaming\\\" tab to view Xbox Game Pass subscription options for console.\",\"code\":\"pyautogui.click(x=0.4979, y=0.4065)\",\"image\":\"8e305618-620c-42a2-9ebc-f8f227a8bf2f.png\"},{\"action\":\"Click on the \\\"Both\\\" tab in the Xbox Game Pass subscription options to view plans that cover both PC and console gaming.\",\"code\":\"pyautogui.click(x=0.5672, y=0.4111)\",\"image\":\"1cc3325d-a6c0-4b0a-a4ef-aeec7fbabc04.png\"},{\"action\":\"Click the X button in the top-right corner of the Xbox Game Pass subscription dialog to close it and return to the Services & subscriptions page.\",\"code\":\"pyautogui.click(x=0.7359, y=0.1759)\",\"image\":\"a68eb273-1e4c-41f3-a608-e42004ad6c37.png\"}],\"verified_task_completed\":true}"}],"source":"xlangai/AgentNet","source_revision":"d76ee50a63fad81cfdbe576416757d7c2091ed50","source_task_id":"20241114181540_5bfec015-6048-48c6-9538-896fae322b4e","media_archive":"evolution/basic/_assets/agentnet/win_mac_images.zip","domain":"autonomous_workflow","language":"en","row_sha256":"fab51d7b5966964823f182876b99ca0ddf0febc159db3963685ab3241cb93780"}
Xet Storage Details

Size:: 373 kB
Xet hash:: 8e841d5554cc3c2c4bf664e9d8f7ecd2a14a40ea1c85a53a5b0cd8cab24bf683
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.