Week 12 · Guest Lectures
18 May 2026 · guest: Dieter Fox
Big picture
Where this week fits
The last session of the course has no lecture. It belongs to guest speakers, and then the students show their projects on real robots at demo day, as the lecturer announced at the end of Week 11 ▶ 1:00:28. The course page lists one speaker, Dieter Fox, whose talk is not available online. So this page does what a last week can do on its own: it looks back over the course in one place, shows how the weeks fit together inside a modern robot policy, and draws out the threads that ran through all of them.
The question this week answers: having seen the whole course, how do its pieces fit together, and which lessons kept coming back?
The course in one table
| Week | The question it answered | The short answer |
|---|---|---|
| 1 · Introduction | Why can’t we just program robots for our homes? | Sense-plan-act breaks in unstructured worlds; learning from data and experience replaces hand-coding, and robotics becomes multimodal sequence modelling. |
| 2 · Control & MDPs | How does a goal become motor commands, and how do we describe decisions above that? | Kinematics, trajectories and PID turn targets into torques; the MDP and the policy \(\pi(a \mid o)\) are the language for everything learned after. |
| 3 · Imitation learning | How do demonstrations become a policy, and why does copying break? | Behavioral cloning is supervised learning, but its errors compound off the expert’s states; DAgger, history and goal-conditioning help, and averaging modes is a trap. |
| 4 · RL I | How can a robot learn from rewards alone? | Values and the Bellman equation, from dynamic programming to Q-learning and DQN, and ways to handle continuous actions. |
| 5 · RL II | How do you follow the gradient of reward through a world you can’t differentiate? | The policy gradient, made usable by baselines and advantages, then trust regions (TRPO, PPO) and off-policy actor-critics (SAC). |
| 6 · Generative models | How can a policy represent a complicated distribution over actions? | Turn noise into actions: VAEs, diffusion and flow matching, as in Diffusion Policy and π0. |
| 7 · Sequences & transformers | How do observations, words and actions become one token sequence? | Attention over tokens, scaling laws, and actions as bins, compressed chunks (FAST) or chunks predicted at once (ACT). |
| 8 · World models | How do you learn what your actions will do, and use it? | Predict consequences, plan or imagine with them, and beware compounding, exploited model errors; scale with video or predict embeddings (JEPA). |
| 9 · Generalist policies | Where do generalist policies come from, and how do you know they work? | Pool data across robots, tokenize everything, start from a VLM and protect its knowledge, and evaluate by rankings. |
| 10 · Reasoning & test-time scaling | Can a robot that thinks harder act better? | Embodied chain-of-thought helps mainly by teaching better features; test-time search pays only on problems the model sometimes solves; RL makes reasoning emerge. |
| 11 · Frontiers | What is still unsolved? | Grounded generalization, dexterity beyond vision, knowing what you don’t know, going beyond imitation, and learning for a lifetime. |
Where each week lives in a modern robot policy
Most of the course can be read as the parts of one system. Figure 2 lays out a generalist vision-language-action policy of the kind Weeks 9 to 11 discussed, and marks which week supplied each part.
The robot’s path is the course’s middle: tokens and transformers (Week 7) carry camera images and instructions into a backbone pretrained on the web (Weeks 9 and 11), which may reason first (Week 10) before a generative action expert (Week 6) samples a chunk of actions (Week 7) that a classical controller tracks (Week 2). Training comes from the course’s two families: imitation of pooled demonstrations (Weeks 3 and 9), refined with reward (Weeks 4, 5, and π*0.6 in Week 9). Evaluation, in simulation or a learned model (Weeks 8 and 9), decides whether any of it worked.
Guest spotlight
Dieter Fox is a professor in the Allen School of Computer Science & Engineering at the University of Washington, where he heads the Robotics and State Estimation Lab, and he also works at the Allen Institute for AI (Ai2). His research spans robotics, artificial intelligence and state estimation, with applications in 3-D mapping and tracking, human activity recognition, and robot manipulation; he is a fellow of AAAI, ACM and IEEE and a recipient of the IEEE RAS Pioneer Award (his homepage). With Sebastian Thrun and Wolfram Burgard he wrote Probabilistic Robotics (MIT Press, 2005), the standard text on handling uncertainty in robot perception and control, whose filters and mapping methods underlie the classical mapping and localization stacks that mobile robots use today. Week 11 asked whether a unified learned model should take over that role.
NoteNo recording
This talk is not available online: the course page lists no title, slides or recording for it. Every other guest spotlight on this site is written from the talk’s transcript, so this one stops at the speaker’s background and does not describe the talk’s content.
Connections
Six threads ran through the course. Each came back in several weeks, usually in a new form.
- Distribution shift. A policy meets the states its own actions lead to, not the ones in its data. It was compounding error in Week 3, planners exploiting a world model’s mistakes in Week 8, the gap between simulator and robot in Week 9, and a moved obstacle that an ensemble could not see in Week 11.
- Actions have many right answers. Averaging two good demonstrations drove into the tree in Week 3; generative models in Week 6 and action tokens in Week 7 are the fix that today’s policies use.
- Data and scale win, but not by volume alone. Pooling robots helped the data-poor one in Week 9, and the guest in Week 11 traced how scaling model, data and training together made vision work. Yet mixing a robot’s own rollouts naively into cloning made the Week 11 flywheel go backwards: what you do with data matters as much as how much you have.
- Imitation and reward complement each other. Imitation is fast and safe but bounded by its teachers; reward-driven learning (Weeks 4 and 5) can go beyond them, whether by stitching (Week 11), advantage conditioning (π*0.6, Week 9) or GRPO (Week 10).
- Fast and slow. Low-level controllers run fast (100 Hz in Week 2’s joint, which swung wildly at 20 Hz) while learned policies send targets a few times a second; action chunks buy smoothness at the cost of reaction (Week 7); reasoning buys generalization at the cost of speed (Week 10); and Week 11 asked when each is worth paying for.
- Evaluation is the hard part. Twenty trials barely separate two policies (Week 9’s binomial error), a demo filmed in the training room proves little, and a benchmark is only as good as what it measures (Week 11’s “look at the data”).
Going further. The glossary collects the notation of all weeks, the papers index every paper card, and the cheat sheets every week’s formulas in one place. For the decision-making theory underneath Weeks 2 to 5 (MDPs, planning, policy search, reinforcement and imitation learning), the ADM Companion goes deeper.
Resources
- Guest lecture: Dieter Fox (University of Washington & AI2)