Audrey Woods, MIT CSAIL Alliances | November 2, 2026
Is the story of King Midas an alignment problem? After pleasing the Greek god Dionysus, King Midas was granted a single wish. His wish—that everything he touched be turned to gold—quickly became a curse when his food, drink, and eventually his daughter was transformed and thus destroyed. MIT CSAIL Associate Professor Dylan Hadfield-Menell says, “Midas didn't actually want his daughter to turn to gold when he gave her a hug. But based on a reasonable, or literal, interpretation of what he said, that is what he meant.”
AI agents are not Greek gods or rogue genies, but they can, when poorly hemmed in, have equally catastrophic consequences. As Agentic AI systems are rolled out and given real-world capabilities, Hadfield-Menell’s work has gone from relatively niche to front-of-mind across the market. He says, “that’s not necessarily a good thing.”
HUGGING FACE HACK & AI AGENT BEHAVIOR
In July of 2026, some 700 OpenAI agents escaped the sandbox they were supposedly isolated in and hacked into Hugging Face. These agents were digging for answers to the difficult and even impossible problems they were set, and had collectively determined (on chatboards the OpenAI engineers were not aware of) that the solutions they needed might be found in Hugging Face’s systems. Not only did the agents break free and attack a rival tech company, they also attempted to cover their tracks, making it apparent that they knew they were doing something wrong. “One of those models should have reported back and said, ‘Hey, there's a whole bunch of message board communication here that is making your tests invalid.’ That's not a thing any of them did.”
For Hadfield-Menell, this is a clear—and loud—alignment failure. In an ideal world, “the models should have had the basic wherewithal to understand that if they're in a scenario where they need to cover their tracks like this, maybe it's something they shouldn't be doing, and probably shouldn't be hiding from the humans on whose behalf they're acting.” If the OpenAI hack represents “both failure modes of overly literal interpretation of goals and prioritization,” then what does success look like? That’s exactly what Hadfield-Menell aims to answer.
One aspect of Hadfield-Menell’s work is inferring user preference, with a focus on uncertainty, or “being able to recognize that the goal you've been given, and your interpretation of it, are not necessarily accurate.” After studying how people interact with agents, it’s easy for Hadfield-Menell to imagine a user thoughtlessly asking a shopping agent to buy me eggs and then be shocked when an agent orders all the eggs in the store. On the other hand, he also knows users will be irritated by an agent which pings them with endless questions—organic or free range? Brown or white? Large or regular? Much like employees, an agent is most useful when it can strike a balance between being proactive while still checking in when it’s unsure. One area of research, led by PhD student Rachel Ma, “is about how we build systems that understand there are multiple goals a user is trying to pursue, build distributions over what they might want given what they've asked for, and then incorporate that intelligently into behavior.” The challenge they’re exploring now is how to take these theoretical ideas and weave them into modern LLMs. Currently that looks like a deployment configuration—a separate scaffold that sits around the agent, tracking your goals and providing them as context to the agent on an updated basis. The next step is to move toward post-training, building this explicitly into a model. But getting the right kind of data is difficult “because you need a person there, or a simulation of a person. For all that these models are good at, they’re actually not great at simulating people,” which is arguably part of the problem.
Hadfield-Menell has also spent a lot of time thinking about reward hacking, or the idea that the reward an intelligent system is seeking is not necessarily the correct objective for the task. Imagine a cleaning robot rewarded for sweeping dirt off the floor. “What you'll almost always see is the robot learns to scoop up dirt, but then dumps it back out so it can scoop it up again.” To solve this, Hadfield-Menell’s lab is pursuing alternatives to reinforcement learning, which relies on rewards and punishments. Instead, they’re exploring imitation learning and old-fashioned planning systems, with incorporated agents and coding assistants to speed up what would otherwise be a slower process. He explains how this method is more involved than designing a reward function, but the reward function typically involves a lot more iteration, not to mention the risk of reward hacking. “Instead of telling the system, ‘Optimize this reward and get as much reward as you can,’ we're saying, ‘Here is a behavior I want you to implement, here are examples of it,’ and we're using machine learning to make it faster to generate that behavior.” This is an extension of what Hadfield-Menell studied during his PhD for guided imitation of task and motion planning, which introduced these ideas in robotics. “Now we're going back and revisiting that idea from the perspective of language model agents.”
A “STYLE GUIDE” FOR AGENTS & IMPLICATIONS FOR DEMOCRACY
OpenAI, Anthropic, and other major model providers have a document called a model spec or model construction, essentially the rules that are supposed to govern the behavior of a system. Hadfield-Menell is excited to make that idea more mainstream and for “all users of AI to be thinking about what their model specification is. In a corporate deployment, what are the rules and principles we want to organize how AI systems behave? We can then develop systems that evaluate whether those rules are being followed, and potentially train or configure systems so the rules will be followed.”
He hopes this will have a democratic effect on the AI systems that will increasingly affect our daily lives. “In a lot of the arguments about AI right now, people are talking past each other and not clear enough about what they really want from these systems for the discussions to be productive.” Say a public school is using an AI tutoring system. “A parent-teacher association or school board can build and lay out model spec for AI tutors, basically a set of ground rules for how they behave.” If a parent is unhappy with how the AI tutor is interacting with their child, they can claim it’s not compliant with the spec or advocate for modifying the spec, which would be a political process. In a recent paper on the subject, Hadfield-Menell and his co-authors argue: “To maintain democratic resilience, it is essential to build AI agents capable of choosing behaviors that mirror those of the human agents that constitute human democracies.”
He would also like to see “AI systems that represent preference disagreements rather than averaging them out.” In the vein of capturing what users really want, one of his research goals is modifying the learning process to allow for the natural variability of human preferences. Theoretically, this might give model developers options to either train a system which is conservative and good-enough for everyone, optimizing to never offend or bring up the worst-case scenario, or create models which are creative and sometimes deliver unpopular responses. “If you're picking a model everyone has to use, you want to make everyone happy with it. If you've got a ton of models and everyone gets to pick which one they want, you want to make it so everyone can get the version that fits them best.”
ADVICE TO INDUSTRY & MOTIVATIONS
Hadfield-Menell believes AI alignment efforts are not necessarily adversarial. He says, “there are safety engineers involved in building bridges and buildings, but it's not like safety is a separate part of what you want from a building. A good building is one that doesn't fall down. It's the same for AI agents: a good AI agent is one that doesn't misbehave in these really obvious ways. A good AI system doesn't go and hack other companies.” There are business incentives at play, since such obvious misbehavior is not good for any company, although Hadfield-Menell doesn’t think that will be enough to keep the industry safe. “As we discover new capabilities, we're going to expose a lot of places where our society is vulnerable, and the trade-offs around how accessible this technology is are going to become a much more important question for us.” He’s especially worried about open-source models and the way they might enable cyber attacks or even biological weapons. “It's much easier to patch a computer system vulnerability than to patch vulnerabilities in human biology.” As he made clear in his recent Boston Globe feature, Hadfield-Menell is less worried about a Terminator-style apocalypse than “a powerful optimization-driven system pointed at some goal” which might spiral out of control with negative, real-world impacts.
How can companies protect themselves? Hadfield-Menell’s main advice is to think about scope and evaluation. “What's your actual plan for making sure [your AI agent] works?” This means thinking clearly about the safety measures and restrictions, the hierarchy of models and evaluation procedures, what they can do, what they’re hooked up to, when they need human review, and what they can do on their own. More generally, he says, “this is a good time, between now and December, for you and your family members to change your passwords, start using a password manager, and for companies to really start taking seriously the potential for much more sophisticated cyber attacks being feasible for a much broader range of actors.”
When asked what drives his research, Hadfield-Menell answers, “what gets me up in the morning is less excitement about what I'm doing, and more fear that it's not going to get done. It's really important that we build these systems to be safe and reliable.” Unfortunately, he’s witnessed how “the field of machine learning is not necessarily full of people who think about how this can go wrong, because for a long time it's been essentially a consequence-free research area. To the extent we have deployed things that touch the world, either it's in specialized areas where the people deploying the models are really thinking about how it can go wrong, or we've deployed large systems at scale that have had a bunch of negative consequences, like building really addictive engagement-optimization tools and systems." As the rest of the world asks what can be done with AI, Hadfield-Menell is asking: “how do we make sure that while we're shooting for those incredibly ambitious targets, we're actually thinking through the details of how this can go wrong if we succeed, and what it takes to actually control these systems.”
“There's a big difference between a system that's capable of programming anything you might want it to do and a system that is going to program the thing you want it to do. It's our job as system designers to make sure the outcomes people get from their systems are what they really want.”
Learn more about Professor Hadfield-Menell on his website, CSAIL Page or feature in the Boston Globe.
Every company is asking the same questions right now: what can AI agents do? Are they safe? And what will they cost at scale.
MIT CSAIL is working on answers. Our faculty lead the field in agentic security, cost, alignment, interaction, and deployment. We're inviting companies to join our Agentic AI Research Initiative and help our researchers shape what comes next.
Reach out to CSAIL Alliances Interim Managing Director Glenn Wong at glennw@mit.edu to get involved.
