Audrey Woods, MIT CSAIL Alliances | November 16, 2026
AI models, for all their capabilities, have a fatal flaw. They are huge—data-hungry and energy-intensive—requiring numerous GPUs, ample energy, and lots of space. This makes them difficult to operate on edge devices like sensors, smart watches, or medical patches. Even the simplest ChatGPT query on our phones requires an internet connection, the time for data to be sent back and forth to distant servers, and the potential exposure of private information.
Irene Tenison, a graduate student at MIT CSAIL, says “Our brain is way more capable than any of those models out there, and its continuous power consumption is less than a single household light bulb. Meanwhile, just to train one of these AI models a single time consumes more total energy than multiple cars use over their entire lifespans. The gap is so huge, and somebody has to say: it doesn't make sense.”
Working in the Decentralized Information Group (DIG), Tenison aims to make AI efficient enough to function on smaller devices like wearables and embedded devices. Her vision, she explains in an MIT News feature on her research, is “bringing AI to small devices where it is not currently possible to run these powerful models. We carry these devices around with us in our daily lives. We need AI to be able to run on these devices, not just on giant servers and GPUs.”
THE FLAW IN FEDERATED LEARNING
Tenison began her research studying federated learning, a privacy-preserving training method where, instead of sharing data, the model is sent to each device and trained locally, then sent back with updated weights. “Around 2021, people were getting excited about doing all things federated and not sending data to a central server. But then the excitement faded away because the models kept growing faster, and you don't update a device every month. You probably update it every six or seven years. So you can't keep up from the device standpoint. That's when I realized: federated learning is great, amazing even, but there's a bottleneck that needs to be solved first.”
To address that bottleneck, Tenison is approaching things from a counterintuitive angle. While most of the AI field is all-in on accuracy, Tenison is focused on other metrics. “If you get 96% instead of 98%, it's still good enough for deployment. But if you're consuming two times more energy, that's a bottleneck.” So she’s optimizing her methods for memory footprint (RAM the model needs), energy consumption, throughput (tokens per second), and latency (time until answer). “Some of my models are not the best performing—they'd be maybe within 5% of the top accuracy—but you get all of these extra memory and compute benefits, and you compute everything faster.”
Tenison is approaching this via architectures and algorithms, asking “what memory can be not saved and bypassed?” Every time a model runs, data passes through dozens of layers. At every layer, the model produces intermediate results—activations—and holds onto them. These scratch-pad values consume an enormous amount of memory that "nobody really talks about." Tenison is playing with the idea of throwing those activations away, which opens up opportunities for model usage on smaller devices. She’s also exploring parameter selection and attention compression to offer faster responses and longer battery life. Trading compute for memory as in gradient checkpointing and similar works is often a worthwhile bargain—"with these recent chips, it's okay to give up on compute to gain memory benefits"—but the ideal, she says, is a method that reduces both to create models that can function with both the smaller memory size and battery limits of edge devices.
Tenison has experimented with interventions at every stage, training a model from scratch, adapting a pre-trained model to a specific task, and optimizing a finished model right before deployment for inference. While she says none of these methods are clear winners, she believes her current work on post-training efficiency is most practical “because many companies have trained models adapted to their data and use case. Then comes the question of deployment. That's where I think most companies get stuck.”
ADVICE FOR INDUSTRY & APPLICATIONS
If there’s one key point Tenison wants companies to take away from her research, it’s that “deployment constraints should be a first-class objective along with performance, and should be optimized in a way that's as important as performance.” Many companies are building impressive models, but Tenison argues they often “don’t have deployment in mind.” Industry users planning to deploy models for specific use cases should think early on about where the model will be used, potential edge-device scenarios, and whether their model will fit on the technology available to them. Tenison says “it will be a great win if people start thinking about deployment constraints from day one.”
The potential of such technology is enormous, offering the power of LLMs in time-critical or data-sensitive scenarios. Medicine offers many applications: quick, accurate, and privacy-preserving responses to heart incidents, allergic responses, and cancer screening. Tenison also brings up factory monitoring, where if “there's a machine that isn't working, and you have a camera recording it all the time, you want the interruption to happen as soon as possible, rather than delaying the whole pipeline.” Opportunities for global accessibility inspire her because “not everyone has the latest iPhone. In many developing countries, users might have less powerful mobile phones. With our technique, we can bring the benefits of federated learning, and AI in general, to these settings."
With users across the globe increasingly consulting models every day, multiple times a day, Tenison is disturbed by the environmental cost of such widespread AI usage. But she doesn’t want to put the brakes on AI. “At this point I can't think about doing something without ChatGPT or Claude. At the same time, I want models to not consume so much energy.” With more efficient models capable of operating on smaller devices, not only can we all reap the benefits of AI in our everyday lives, but we can protect the environment and allow for a more sustainable future of technological growth.
Learn more about Irene Tenison on her CSAIL or LinkedIn Page.