Originally from Brookline, MA, Hallee Wong studied at Williams College prior to MIT and CSAIL. Wong majored in math, but became progressively more interested in statistics, and then machine learning and computer science toward the end of her undergraduate career. She became interested in machine learning and healthcare through a class project where she collaborated with clinicians from a local hospital to develop ML models for predicting hospital readmissions.
Wong applied to MIT EECS and CSAIL because of the multitude of research groups working on machine learning for healthcare. Wong said she was confident she would be able to find a research group that fit her interests.
Wong is now a PhD candidate working alongside Professors Adrian Dalca and John Guttag in the lab’s Clinical and Applied Machine Learning Group (CAML).
“As PhD co-advisors, I think they make a wonderful team, and they co-advise several students in our group,” Wong said. “John has a lot of perspective on how to identify research problems, conduct research effectively (e.g., designing ‘minimal viable experiments’ to test an idea as quickly and simply as possible), and communicate a clear and compelling story about the work. Adrian is more enmeshed in the technical details of our work, and thus our discussions range from debugging my latest experimental results to planning how the next series of research projects will fit together. I particularly appreciate how John and Adrian promote a friendly and collegial environment.”
Wong said one of her favorite things about the CAML group is how engaged the members are at their weekly meetings. She said the group covers a wide range of research topics related to machine learning for healthcare, medical imaging, and other applications. According to Wong, everyone is dedicated to understanding each other’s work and providing kind and thoughtful feedback.
Wong and her collaborators recently wrapped up a project called ScribblePrompt.
Wong said in medical research and clinical care, scientists often need to delineate different regions of interest in biomedical images. For example, vasculature in retinal images or tumors in brain MRI. This so called “segmentation” is an essential step in a wide range of clinical research and medical care pipelines such as quantifying variations in anatomy or radiotherapy planning. It is also frequently, given existing technology, a very labor-intensive and manual step. Human annotators must manually outline the boundaries, which is both time-consuming and tedious.
Wong added this is particularly onerous for clinical researchers and clinicians that need to solve new segmentation tasks, requiring the delineation of either new regions of interest or involving new imaging types.
Wong said recent research into smarter, iterative interactive segmentation systems seeks to alleviate this burden. In these interactive segmentation systems, the annotator iterates with the AI model to complete the segmentation, providing prompts (e.g., clicks) and receiving predicted segmentation(s) that they can correct with additional prompting. However, existing methods in this space have not been widely adopted in practice, mostly because they are useful only in limited circumstances or for specific segmentation targets, and do not generalize to new targets in biomedical imaging.
Overall, the lack of tools that can alleviate the manual segmentation burden creates a substantial barrier to many medical research studies and clinical care workflows, where crucially useful delineations are either not performed, or extremely costly. Wong said her research addresses this gap by developing generalizable human-AI systems that enable users to interactively segment and analyze biomedical images.
Existing interactive segmentation systems often fail when faced with new segmentation tasks and new types of biomedical images. In response, Wong and colleagues developed ScribblePrompt, a new approach to rapid interactive image segmentation, which they believe will dramatically improve the annotation process for any new medical image segmentation task desired by clinical researchers or practitioners. The core idea is to use a single streamlined neural network model, trained to handle a diverse array of potential user interactions, biomedical domains, and regions of interest.
Wong said ScribblePrompt is an interactive segmentation system that enables human annotators to segment new structures in new images using scribbles, clicks, and bounding boxes.
“Scribbles are an intuitive form of user interaction for complex tasks, however most existing methods focus on click-based interactions,” she said. “We introduce algorithms for simulating realistic scribbles that enable training models that are amenable to multiple types of interaction. To enable the system to generalize new tasks, we built a diverse collection of 77 open-access biomedical datasets and further augmented the collection with synthetic data for training. ScribblePrompt outperforms existing methods on new datasets and it runs fast enough to be used interactively even without a GPU. The success of ScribblePrompt relies on a set of careful design decisions made with ease-of-use and practicality in mind.”
Wong said ScribblePrompt is already in use by practitioners. She developed an open-source web application, and they are working with developers to integrate it into popular open-source medical imaging software. She said they have several collaborations planned with biomedical researchers across MIT, MGH, and Harvard Medical School to put the technology into practice.
Wong said she enjoys working on ML for healthcare not only because of the potential for positive impact on the world by improving clinical care or aiding medical research, but also because of the technical challenges.
“Machine learning methods developed in other domains often fail in healthcare problems because of challenges like limited and messy data,” she said. “So as a computer scientist, healthcare is an exciting domain because these challenges inspire new method development.”
During Wong’s first two years at CSAIL, she worked in a different area of ML for health: using machine learning to optimize the allocations of hospital beds to minimize healthcare acquired infections. After she finished that project, she decided to change directions and focus on deep learning for medical imaging. Wong thinks medical imaging is an application area where computing, and more specifically AI and deep learning, have a lot of potential for positive impact.
“More broadly, I think AI has a lot of potential to help in areas of healthcare and medical research involving messy high dimensional data like unstructured text and imaging,” she said. “For example, many forms of medical imaging e.g., CT and MRI are 3D. Humans are not very good at visualizing or analyzing complex 3D data. With the right training, deep learning models excel at analyzing this kind of data.”
Wong said her goal is to develop human-centered AI systems that augment the work of clinicians and biomedical researchers, improving clinical care and aiding medical research. The motivation for ScribblePrompt was developing an AI model that would speed up a tedious and time-consuming task for biomedical researchers.
Following her time at CSAIL, Wong said she is motivated by applications, so her dream job would involve working with a diverse team on challenging technical problems with positive impact in a domain like healthcare. She enjoys building tools and developing solutions to fit the constraints of real-world problems and users.
For more information about Hallee Wong, check out her personal website: https://halleewong.github.io/