Filter Options
Date
Image
alt="Regina Barzilay, MIT professor, CSAIL Principal Investigator, and Jameel Clinic AI Faculty Lead (Credit: WCVB)."
CSAIL article

Regina Barzilay, School of Engineering Distinguished Professor for AI and Health at MIT, CSAIL Principal Investigator, and Jameel Clinic AI Faculty Lead, has been awarded the 2025 Frances E. Allen Medal from the Institute of Electrical and Electronics Engineers (IEEE). Barzilay’s award recognizes the impact of her machine-learning algorithms on medicine and natural language processing.

Image
alt="Transformers process and generate inputs by “attending” to all previous inputs in each layer, which becomes expensive as the sequence length grows; in contrast, linear transformers maintain a fixed-size memory that is updated recurrently at each time step, allowing it to efficiently process and generate long sequences (Credit: The researchers)."
CSAIL article

Generative AI systems like large language models rely heavily on deep learning - and, in particular, transformers. Transformers make use of an “attention mechanism” for modeling interactions among inputs, which essentially involves doing nonlinear pairwise comparison between inputs and assigning different weights to tokens in a sequence, enabling a prioritization of some over others. The empirical effectiveness of this attention mechanism has led some in the community to claim that attention is “all you need” (the title of the original 2017 Google paper that introduced transformers).