In the classic cartoon “The Jetsons,” Rosie the robotic maid seamlessly switches from vacuuming the house to cooking dinner to taking out the trash. But in real life, training a general-purpose robot remains a major challenge.
In the current AI zeitgeist, sequence models have skyrocketed in popularity for their ability to analyze data and predict what to do next. For instance, you’ve likely used next-token prediction models like ChatGPT, which anticipate each word (token) in a sequence to form answers to users’ queries. There are also full-sequence diffusion models like Sora, which convert words into dazzling, realistic visuals by successively “denoising” an entire video sequence
The Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT) Editorial Board has awarded MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Gwangju Institute of Science and Technology (GIST) researchers with a Distinguished Paper Award for their evaluation of visual explanations in autonomous vehicles’ decision-making.
For robots, simulation is a great teacher for learning long-horizon (multi-step) tasks — especially compared to how long it takes to collect real-world training data.
Imagine you’re tasked with sending a team of football players onto a field to assess the condition of the grass (a likely task for them, of course). If you pick their positions randomly, they might cluster together in some areas while completely neglecting others. But if you give them a strategy, like spreading out uniformly across the field, you might get a far more accurate picture of the grass condition.
Are you a CSAIL entrepreneur? Are you curious about the resources that CSAIL Alliances, as well as the rest of the MIT Ecosystem can offer you? Sign up for Office Hours using the form to ask Christiana Kalfas, Sr.
On a research cruise around Hawaii in 2018, Yuening Zhang SM ’19, PhD ’24 saw how difficult it was to keep a tight ship. The careful coordination required to map underwater terrain could sometimes led to a stressful environment for team members, who might have different understandings of which tasks must be completed in spontaneously changing conditions. During these trips, Zhang considered how a robotic companion could have helped her and her crewmates achieve their goals more efficiently.
