Papers

Every paper discussed in the course, grouped by week

Each week’s student-led session discusses three papers. Papers marked (from the lecture) are ones the lecture itself relies on. Each week page has a card per paper with a short summary.

Week 1 · Introduction to Robot Learning

• Open X-Embodiment: Robotic Learning Datasets and RT-X Models. Open X-Embodiment Collaboration, ICRA 2024. (from the lecture)
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. Brohan et al., CoRL 2023. (from the lecture)

Week 2 · Robot Control & MDPs

• Simple random search provides a competitive approach to reinforcement learning. Mania, Guy & Recht, NeurIPS 2018.
• Deep Reinforcement Learning Doesn’t Work Yet. Irpan, blog post 2018.
• Curiosity-driven Exploration by Self-supervised Prediction. Pathak, Agrawal, Efros & Darrell, ICML 2017.

Week 3 · Imitation Learning

• Causal Confusion in Imitation Learning. de Haan, Jayaraman & Levine, NeurIPS 2019.
• The Surprising Effectiveness of Representation Learning for Visual Imitation. Pari, Shafiullah, Arunachalam & Pinto, RSS 2022.
• Transporter Networks: Rearranging the Visual World for Robotic Manipulation. Zeng et al., CoRL 2020.
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Ross, Gordon & Bagnell, AISTATS 2011. (from the lecture)
• End to End Learning for Self-Driving Cars. Bojarski et al., arXiv 2016. (from the lecture)
• Learning Latent Plans from Play. Lynch et al., CoRL 2019. (from the lecture)

Week 4 · Reinforcement Learning I

• Evolution Strategies as a Scalable Alternative to Reinforcement Learning. Salimans, Ho, Chen, Sidor & Sutskever, arXiv 2017.
• Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning. Zeng, Song, Welker, Lee, Rodriguez & Funkhouser, IROS 2018.
• Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning. Luo, Xu, Wu & Levine, Science Robotics 2025.
• Human-level control through deep reinforcement learning. Mnih et al., Nature 2015. (from the lecture)
• Deep Reinforcement Learning with Double Q-learning. van Hasselt, Guez & Silver, AAAI 2016. (from the lecture)
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Kalashnikov et al., CoRL 2018. (from the lecture)
• Continuous control with deep reinforcement learning. Lillicrap et al., ICLR 2016. (from the lecture)

Week 5 · Reinforcement Learning II

• End-to-End Training of Deep Visuomotor Policies. Levine, Finn, Darrell & Abbeel, JMLR 2016.
• Eureka: Human-Level Reward Design via Coding Large Language Models. Ma et al., ICLR 2024.
• Latent Plans for Task-Agnostic Offline Reinforcement Learning. Rosete-Beas, Mees, Kalweit, Boedecker & Burgard, CoRL 2022.
• Simple statistical gradient-following algorithms for connectionist reinforcement learning. Williams, Machine Learning 1992. (from the lecture)
• Trust Region Policy Optimization. Schulman, Levine, Moritz, Jordan & Abbeel, ICML 2015. (from the lecture)
• Proximal Policy Optimization Algorithms. Schulman, Wolski, Dhariwal, Radford & Klimov, arXiv 2017. (from the lecture)
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Haarnoja, Zhou, Abbeel & Levine, ICML 2018. (from the lecture)

Week 6 · Generative Models

• Planning with Diffusion for Flexible Behavior Synthesis. Janner, Du, Tenenbaum & Levine, ICML 2022.
• Implicit Behavioral Cloning. Florence et al., CoRL 2021.
• Steering Your Diffusion Policy with Latent Space Reinforcement Learning. Wagenmaker et al., arXiv 2025.
• Denoising Diffusion Probabilistic Models. Ho, Jain & Abbeel, NeurIPS 2020. (from the lecture)
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Chi et al., RSS 2023. (from the lecture)
• Flow Matching for Generative Modeling. Lipman, Chen, Ben-Hamu, Nickel & Le, ICLR 2023. (from the lecture)

Week 7 · Sequence Modeling and Transformers

• Decision Transformer: Reinforcement Learning via Sequence Modeling. Chen et al., NeurIPS 2021.
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. Zhao, Kumar, Levine & Finn, RSS 2023.
• Humanoid Locomotion as Next Token Prediction. Radosavovic et al., NeurIPS 2024.
• Attention Is All You Need. Vaswani et al., NeurIPS 2017. (from the lecture)
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Dosovitskiy, Beyer et al., ICLR 2021. (from the lecture)
• Training Compute-Optimal Large Language Models. Hoffmann et al., NeurIPS 2022. (from the lecture)
• FAST: Efficient Action Tokenization for Vision-Language-Action Models. Pertsch, Stachowicz et al., RSS 2025. (from the lecture)

Week 8 · World Models

• Learning Universal Policies via Text-Guided Video Generation. Du et al., NeurIPS 2023.
• Training Agents Inside of Scalable World Models. Hafner, Yan & Lillicrap, arXiv 2025.
• World Action Models are Zero-shot Policies. Ye et al., arXiv 2026.
• Deep Visual Foresight for Planning Robot Motion. Finn & Levine, ICRA 2017. (from the lecture)
• World Models. Ha & Schmidhuber, NeurIPS 2018. (from the lecture)
• Learning Latent Dynamics for Planning from Pixels. Hafner et al., ICML 2019. (from the lecture)
• Dream to Control: Learning Behaviors by Latent Imagination. Hafner, Lillicrap, Ba & Norouzi, ICLR 2020. (from the lecture)
• DayDreamer: World Models for Physical Robot Learning. Wu, Escontrela, Hafner, Abbeel & Goldberg, CoRL 2022. (from the lecture)
• mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs. Pai, Achenbach et al., arXiv 2025. (from the lecture)

Week 9 · Generalist Robot Policies

• Language Conditioned Imitation Learning over Unstructured Data. Lynch & Sermanet, RSS 2021.
• A Generalist Agent. Reed et al., TMLR 2022.
• π*0.6: a VLA That Learns From Experience. Physical Intelligence, arXiv 2025.
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models. Open X-Embodiment Collaboration, ICRA 2024. (from the lecture)
• Octo: An Open-Source Generalist Robot Policy. Octo Model Team (Ghosh, Walke, Pertsch, Black, Mees et al.), RSS 2024. (from the lecture)
• Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation. Doshi, Walke, Mees, Dasari & Levine, CoRL 2024. (from the lecture)
• A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation. TRI LBM Team, arXiv 2025. (from the lecture)
• π0: A Vision-Language-Action Flow Model for General Robot Control. Black et al. (Physical Intelligence), RSS 2025. (from the lecture)
• π0.5: a Vision-Language-Action Model with Open-World Generalization. Physical Intelligence (Black et al.), CoRL 2025. (from the lecture)
• Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better. Driess et al., NeurIPS 2025. (from the lecture)
• Evaluating Real-World Robot Manipulation Policies in Simulation. Li, Hsu, Gu, Pertsch, Mees et al., CoRL 2024. (from the lecture)

Week 10 · Embodied Reasoning and Test-time Scaling

• In-Context Imitation Learning via Next-Token Prediction. Fu et al., ICRA 2024.
• Voyager: An Open-Ended Embodied Agent with Large Language Models. Wang et al., TMLR 2023.
• Training Strategies for Efficient Embodied Reasoning. Chen et al., CoRL 2025.
• Robotic Control via Embodied Chain-of-Thought Reasoning. Zawalski, Chen, Pertsch, Mees, Finn & Levine, CoRL 2024. (from the lecture)
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Wei, Wang, Schuurmans, Bosma, Ichter, Xia, Chi, Le & Zhou, NeurIPS 2022. (from the lecture)
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems. Li, Liu, Zhou & Ma, ICLR 2024. (from the lecture)
• Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Parameters for Reasoning. Snell, Lee, Xu & Kumar, ICLR 2025. (from the lecture)
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. Shao et al., arXiv 2024. (from the lecture)
• DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning. DeepSeek-AI (Guo et al.), Nature 2025. (from the lecture)
• Mastering the Game of Go without Human Knowledge. Silver et al., Nature 2017. (from the lecture)

Week 11 · Frontiers & Open Problems

• A Path Towards Autonomous Machine Intelligence. LeCun, position paper, OpenReview 2022.
• The Bitter Lesson. Sutton, essay 2019.
• Intelligence without Representation. Brooks, Artificial Intelligence 1991.
• Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding. Jones, Mees, Sferrazza, Stachowicz, Abbeel & Levine, ICRA 2025. (from the lecture)
• Visual Language Maps for Robot Navigation. Huang, Mees, Zeng & Burgard, ICRA 2023. (from the lecture)
• LeLaN: Learning a Language-Conditioned Navigation Policy from In-the-Wild Video. Hirose, Glossop, Sridhar, Mees & Levine, CoRL 2024. (from the lecture)
• CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks. Mees, Hermann, Rosete-Beas & Burgard, RA-L 2022. (from the lecture)
• Big Transfer (BiT): General Visual Representation Learning. Kolesnikov, Beyer, Zhai, Puigcerver, Yung, Gelly & Houlsby, ECCV 2020. (from the lecture)
• Learning Transferable Visual Models From Natural Language Supervision. Radford et al., ICML 2021. (from the lecture)
• Sigmoid Loss for Language Image Pre-Training. Zhai, Mustafa, Kolesnikov & Beyer, ICCV 2023. (from the lecture)
• Image Captioners Are Scalable Vision Learners Too. Tschannen, Kumar, Steiner, Zhai, Houlsby & Beyer, NeurIPS 2023. (from the lecture)
• PaliGemma: A Versatile 3B VLM for Transfer. Beyer, Steiner, Pinto, Kolesnikov, Wang et al., arXiv 2024. (from the lecture)

Week 12 · Guest Lectures

No papers listed yet.