Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <meta name="color-scheme" content="light"> | |
| <title>Deep RL Timeline</title> | |
| <link rel="stylesheet" href="styles.css"> | |
| </head> | |
| <body> | |
| <main class="topic timeline-page"> | |
| <a class="back" href="index.html" aria-label="Home"> | |
| <svg width="20" height="20" viewBox="0 0 24 24" fill="none" aria-hidden="true"> | |
| <path d="m14 6-6 6 6 6M8 12h12" /> | |
| </svg> | |
| </a> | |
| <h1 class="topic-title">Deep RL Timeline</h1> | |
| <ol class="timeline"> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2013">2013</time> | |
| <section class="timeline-content" aria-labelledby="atari-dqn"> | |
| <header class="timeline-heading"> | |
| <h2 id="atari-dqn"><a class="reference-link" href="https://arxiv.org/abs/1312.5602" target="_blank" rel="noopener noreferrer">Atari (DQN)</a></h2> | |
| <p>[Deepmind]</p> | |
| </header> | |
| <div class="game-gallery"> | |
| <a class="game-link" href="https://ale.farama.org/environments/pong/" target="_blank" rel="noopener noreferrer" aria-labelledby="pong-caption"> | |
| <figure class="game"> | |
| <img src="assets/atari/pong.png" alt="Pong Atari gameplay" width="160" height="210"> | |
| <figcaption id="pong-caption">Pong</figcaption> | |
| </figure> | |
| </a> | |
| <a class="game-link" href="https://ale.farama.org/environments/enduro/" target="_blank" rel="noopener noreferrer" aria-labelledby="enduro-caption"> | |
| <figure class="game"> | |
| <img src="assets/atari/enduro.png" alt="Enduro Atari gameplay" width="160" height="210"> | |
| <figcaption id="enduro-caption">Enduro</figcaption> | |
| </figure> | |
| </a> | |
| <a class="game-link" href="https://ale.farama.org/environments/beam_rider/" target="_blank" rel="noopener noreferrer" aria-labelledby="beamrider-caption"> | |
| <figure class="game"> | |
| <img src="assets/atari/beamrider.png" alt="Beamrider Atari gameplay" width="160" height="210"> | |
| <figcaption id="beamrider-caption">Beamrider</figcaption> | |
| </figure> | |
| </a> | |
| <a class="game-link" href="https://ale.farama.org/environments/qbert/" target="_blank" rel="noopener noreferrer" aria-labelledby="qbert-caption"> | |
| <figure class="game"> | |
| <img src="assets/atari/qbert.png" alt="Q*bert Atari gameplay" width="160" height="210"> | |
| <figcaption id="qbert-caption">Q*bert</figcaption> | |
| </figure> | |
| </a> | |
| </div> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2014">2014</time> | |
| <section class="timeline-content" aria-labelledby="trpo"> | |
| <header class="timeline-heading"> | |
| <h2 id="trpo"><a class="reference-link" href="https://arxiv.org/abs/1502.05477" target="_blank" rel="noopener noreferrer">2D locomotion (TRPO)</a></h2> | |
| <p>[Berkeley]</p> | |
| </header> | |
| <a class="timeline-media" href="https://www.youtube.com/watch?v=KJ15iGGJFvQ" target="_blank" rel="noopener noreferrer" aria-label="2D locomotion (TRPO) demonstration"> | |
| <img src="assets/timeline/trpo.png" alt="A red and purple two-legged walker on a checkerboard floor" width="644" height="480"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2015">2015</time> | |
| <section class="timeline-content" aria-labelledby="alphago"> | |
| <header class="timeline-heading"> | |
| <h2 id="alphago"><a class="reference-link" href="https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf" target="_blank" rel="noopener noreferrer">AlphaGo</a></h2> | |
| <p>[Deepmind]</p> | |
| </header> | |
| <a class="timeline-media" href="https://deepmind.google/research/alphago/" target="_blank" rel="noopener noreferrer" aria-label="AlphaGo official project"> | |
| <img src="assets/timeline/alphago.png" alt="AlphaGo versus Lee Sedol, with the Go board and player shown together" width="297" height="170"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2016">2016</time> | |
| <section class="timeline-content" aria-labelledby="trpo-gae"> | |
| <header class="timeline-heading"> | |
| <h2 id="trpo-gae"><a class="reference-link" href="https://arxiv.org/abs/1506.02438" target="_blank" rel="noopener noreferrer">3D locomotion (TRPO+GAE)</a></h2> | |
| <p>[Berkeley]</p> | |
| </header> | |
| <a class="timeline-media" href="https://www.youtube.com/watch?v=ATvp0Hp7RUI" target="_blank" rel="noopener noreferrer" aria-label="3D locomotion (TRPO+GAE) demonstration"> | |
| <img src="assets/timeline/trpo-gae.png" alt="A humanoid falls on a checkerboard floor at iteration 160 while learning 3D locomotion" width="640" height="480"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2016">2016</time> | |
| <section class="timeline-content" aria-labelledby="gps"> | |
| <header class="timeline-heading"> | |
| <h2 id="gps"><a class="reference-link" href="https://jmlr.org/papers/v17/15-522.html" target="_blank" rel="noopener noreferrer">Real Robot Manipulation (GPS)</a></h2> | |
| <p><a class="reference-link" href="https://rll.berkeley.edu/deeplearningrobotics/media.html" target="_blank" rel="noopener noreferrer">[Berkeley, Google]</a></p> | |
| </header> | |
| <!-- Image: UC Berkeley, Shape-sorting Cube Learning example, frame at 40.5s. | |
| Source: https://rll.berkeley.edu/deeplearningrobotics/shape_cube_learning.mp4 | |
| License: CC BY 3.0, https://creativecommons.org/licenses/by/3.0/ --> | |
| <a class="timeline-media" href="https://rll.berkeley.edu/deeplearningrobotics/shape_cube_learning.mp4" target="_blank" rel="noopener noreferrer" aria-label="Real Robot Manipulation (GPS) demonstration"> | |
| <img src="assets/timeline/gps.png" alt="A PR2 robot manipulates a wooden shape-sorting cube at iteration 15" width="720" height="400"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2017">2017</time> | |
| <section class="timeline-content" aria-labelledby="dota2"> | |
| <header class="timeline-heading"> | |
| <h2 id="dota2"><a class="reference-link" href="https://arxiv.org/abs/1707.06347" target="_blank" rel="noopener noreferrer">Dota2 (PPO)</a></h2> | |
| <p><a class="reference-link" href="https://openai.com/index/dota-2/" target="_blank" rel="noopener noreferrer">[OpenAI]</a></p> | |
| </header> | |
| <a class="timeline-media" href="https://www.youtube.com/watch?v=wpa5wyutpGc" target="_blank" rel="noopener noreferrer" aria-label="OpenAI Dota2 Learned Bot Behaviors demonstration"> | |
| <img src="assets/timeline/dota2.png" alt="OpenAI's Dota2 bot plays a one-on-one match in the middle lane" width="1280" height="720"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2018">2018</time> | |
| <section class="timeline-content" aria-labelledby="deepmimic"> | |
| <header class="timeline-heading"> | |
| <h2 id="deepmimic"><a class="reference-link" href="https://arxiv.org/abs/1804.02717" target="_blank" rel="noopener noreferrer">DeepMimic</a></h2> | |
| <p>[Berkeley]</p> | |
| </header> | |
| <a class="timeline-media timeline-media--wide" href="https://xbpeng.github.io/projects/DeepMimic/index.html" target="_blank" rel="noopener noreferrer" aria-label="DeepMimic official project"> | |
| <img src="assets/timeline/deepmimic.png" alt="A montage of DeepMimic simulated character skills compared with reference motions" width="800" height="299"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2019">2019</time> | |
| <section class="timeline-content" aria-labelledby="alphastar"> | |
| <header class="timeline-heading"> | |
| <h2 id="alphastar"><a class="reference-link" href="https://doi.org/10.1038/s41586-019-1724-z" target="_blank" rel="noopener noreferrer">AlphaStar</a></h2> | |
| <p><a class="reference-link" href="https://deepmind.google/blog/alphastar-grandmaster-level-in-starcraft-ii-using-multi-agent-reinforcement-learning/" target="_blank" rel="noopener noreferrer">[Deepmind]</a></p> | |
| </header> | |
| <a class="timeline-media timeline-media--wide" href="https://www.youtube.com/watch?v=xP7LwZxq0ss" target="_blank" rel="noopener noreferrer" aria-label="AlphaStar results official demonstration"> | |
| <img src="assets/timeline/alphastar.png" alt="AlphaStar's StarCraft II MMR results compared with human player percentiles and Grandmaster league" width="1920" height="1080"> | |
| </a> | |
| </section> | |
| </li> | |
| <li class="timeline-entry"> | |
| <time class="timeline-year" datetime="2019">2019</time> | |
| <section class="timeline-content" aria-labelledby="rubiks-cube"> | |
| <header class="timeline-heading"> | |
| <h2 id="rubiks-cube"><a class="reference-link" href="https://arxiv.org/abs/1910.07113" target="_blank" rel="noopener noreferrer">Rubik’s Cube (PPO+DR)</a></h2> | |
| <p><a class="reference-link" href="https://openai.com/index/solving-rubiks-cube/" target="_blank" rel="noopener noreferrer">[OpenAI]</a></p> | |
| </header> | |
| <a class="timeline-media" href="https://www.youtube.com/watch?v=kVmp0uGtShk" target="_blank" rel="noopener noreferrer" aria-label="OpenAI Solving Rubik's Cube with a Robot Hand uncut demonstration"> | |
| <img src="assets/timeline/rubiks-cube.png" alt="A black robotic hand with an orange and blue wrist holds a Rubik's Cube" width="1280" height="720"> | |
| </a> | |
| </section> | |
| </li> | |
| </ol> | |
| </main> | |
| </body> | |
| </html> | |