WRITTEN BY: Matt Busekroos | VIDEO BY: Nate Caldwell
Originally from the Bay Area in California, Saachi Jain previously studied computer science at Stanford University. Following her studies at Stanford, Jain worked at Autopilot at Tesla for a year before coming to MIT and CSAIL. Jain is currently in the process of receiving her PhD where she is advised by Professor Aleksander Madry. Jain’s current research interests include building more robust and reliable machine learning models.
Jain’s research seeks to understand how models generalize and build the tools to train and deploy more robust machine learning models. In particular, her work focuses on situations where the training data might be sparse, or differ considerably from the testing environment. She said better tools are needed to be built in order to detect and diagnose brittleness to distribution shifts.
“One project we’re specifically working on now is a method to automatically detect ‘hard subpopulations’ in the data by analyzing the types of mistakes that models make,” Jain said. “In particular, we want to design methods to debug model failures that can actually scale to large vision level datasets with as little direct human involvement as possible.
On the flip side, once we understand the factors that contribute to model failures, we can then start to design methods to mitigate these issues. In one of our more recent works, we found that models that rely on different sets of features will make different types of mistakes even if they have the same overall accuracy. We then show that if you combine models with non-overlapping failure modes, they can correct each other to improve generalization.”
Jain added that one last priority for her is understanding how transfer learning fits into this picture. She said she is interested in understanding how transferring from other datasets changes the biases their models have and the types of mistakes they can make.
“These days, especially if your task doesn’t have enough data, people will take a model trained on a larger dataset and then fine-tune for the problem at hand,” Jain said. “We don’t have a clear understanding of what features get transferred, and whether transferring from a larger model can have (possibly negative) side-effects.”
Prior to CSAIL, Jain worked as a computer vision scientist on Tesla’s Autopilot team.
“By far, the most important consideration for self-driving models is their behavior in unexpected environments,” Jain said. “Test time conditions can vary dramatically from changes in weather to camera calibration to other drivers behaving erratically. In order to safely deploy such models, we need to shift our focus from accuracy to robustness. By developing a fuller understanding of why models fail, my goal is to lay the foundation for safer and more deployable machine learning systems.”
Jain has learned a lot working alongside Madry and her group within the Madry Lab. The lab focuses on the science of modern machine learning.
“One thing Aleksander specifically emphasizes is doing work that is impactful, rather than just interesting,” Jain said. “The best projects address a concrete issue in the field, and, either by providing further understanding or a practical solution, should be applicable to the broader ML community.”
She added that the group is extremely collaborative in that most projects are done with at least two PhD students. In general, she said they spend a lot of time brainstorming and giving feedback as a group, which she finds helpful since projects become the culmination of multiple perspectives.
As a second year PhD student, Jain isn’t quite sure where she sees herself after CSAIL, however she said she would like to continue working on robustness in some capacity, whether that's in academia or industry.