“How do we make agents which go about exploring the world?” This question drives much of Professor Pulkit Agrawal’s research in artificial intelligence, with the goal of building artificial common sense. With work that spans computer vision, robotics, and reinforcement learning, he is developing techniques that help robots explore their environments and learn from experiments within those environments.
Prof. Agrawal, an Assistant Professor in EECS and CSAIL at MIT, completed his PhD at UC Berkeley in 2018. He had always been interested in neuroscience and the intersection of deep neural networks of the brain and AI systems, and came up with the idea of “computational sensorimotor learning” for his PhD. In CSAIL, he continues to research learning-based approaches to control as a way of helping AI agents learn common sense in the real world.
If you think about how humans learn, human babies start out by performing seemingly random actions, such as throwing a ball, pushing a toy, putting an object in their mouth. Research in psychology suggests, however, that these actions are not random, but rather a way of exploring how their actions actually affect the world around them.
“In a way, what they are doing is they are running experiments to understand how the world around them works,” says Prof. Agrawal. “So we are trying to develop techniques which allow robots to also conduct experiments, so that they can learn more about the world.”
One challenge in this type of explorative learning approach is that just like us, computers can get distracted by things that aren’t relevant to what they should be focusing on. This isn’t necessarily a bad thing — when we get curious about things we don’t know about, we often make exciting discoveries, such as Newton’s observation of an apple falling that led him to discover gravity —but knowing when to follow a line of inquiry takes some common sense.
For example, say an agent becomes curious about watching a TV screen, which is only showing static and will continue showing the same random patterns of white and black dots. If the robot understood why the television is showing these patterns, it would stop watching and go explore elsewhere, which might lead it to discover something useful about the world.
“What our agents lack is common sense,” says Prof. Agrawal. “What they have is some very shallow understanding about the world. The models that they are building are also very shallow, in the sense that although my agent can predict what might happen in the future, it doesn’t understand what is going to happen in the future.”
Ultimately, helping agents learn and better understand the world around them will help them figure out how to make new contributions. When we envision a future with robots, we imagine robots learning and working in unstructured environments.
“We really want agents which can work in a diverse collection of environments, when things are not completely predefined, as opposed to applications where you need a lot of precision for doing one task,” says Prof. Agrawal. “I think those are the kinds of tasks where the approaches that we are developing are going to be really useful.”