Image
The “PhysicsGen” system can multiply a few dozen VR demonstrations into nearly 3,000 simulations per machine for mechanical companions like robotic arms and hands (Credit: Alex Shipps/MIT CSAIL using photos from the researchers).
CSAIL article

When ChatGPT or Gemini gives what seems to be an expert response to your burning questions, you may not realize how much information it relies on to give that reply. Like other popular artificial intelligence (AI) models, these chatbots rely on backbone systems called foundation models that train on billions or even trillions of data points.

Image
A robotic arm learns to understand its own body (Credit: Courtesy of the researchers).
CSAIL article

In an office at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), a soft robotic hand carefully curls its fingers to grasp a small object. The intriguing part isn’t the mechanical design or embedded sensors – in fact, the hand contains none. Instead, the entire system relies on a single camera that watches the robot’s movements and uses that visual data to control it.

Image
Top row, left to right: Matthew Caren, April Qiu Cheng, Arav Karighattam, and Benjamin Lou. Bottom row, left to right: Isabelle Quaye, Albert Qin, Ananthan Sadagopan, and Gianfranco (Franco) Yee (Credits: Photos courtesy of the Hertz Foundation).
CSAIL article

The Hertz Foundation announced that it has awarded fellowships to eight MIT affiliates. The prestigious award provides each recipient with five years of doctoral-level research funding (up to a total of $250,000), which gives them an unusual measure of independence in their graduate work to pursue groundbreaking research.

Image
alt="SketchAgent uses a multimodal language model to turn natural language prompts into sketches in a few seconds. It can doodle on its own or through collaboration, drawing with a human or incorporating text-based input to sketch each part separately (Credits: Alex Shipps/MIT CSAIL, with AI-generated sketches from the researchers)."
CSAIL article

When you’re trying to communicate or understand ideas, words don’t always do the trick. Sometimes the more efficient approach is to do a simple sketch of that concept — for example, diagramming a circuit might help make sense of how the system works.

But what if artificial intelligence could help us explore these visualizations? While these systems are typically proficient at creating realistic paintings and cartoonish drawings, many models fail to capture the essence of sketching: its stroke-by-stroke, iterative process, which helps humans brainstorm and edit how they want to represent their ideas.