“If used carefully, this technology could improve performance in health care and potentially reduce inequities,” says Assistant Professor Marzyeh Ghassemi. “But if we’re not actually careful, technology could worsen care.”
Neural networks are sometimes called black boxes because, despite the fact that they can outperform humans on certain tasks, even the researchers who design them often don’t understand how or why they work so well. But if a neural network is used outside the lab, perhaps to classify medical images that could help diagnose heart conditions, knowing how the model works helps researchers predict how it will behave in practice.
