Whether you’re describing the sound of your faulty car engine or meowing like your neighbor’s cat, imitating sounds with your voice can be a helpful way to relay a concept when words don’t do the trick.
Daniela Rus, Director of CSAIL and MIT EECS Professor, was recently named a co-recipient of the 2024 John Scott Award by the Board of Directors of City Trusts. This prestigious honor, steeped in historical significance, celebrates scientific innovation at the very location where American independence was signed in Philadelphia, a testament to the enduring connection between scientific progress and human potential.
Generative AI systems like large language models rely heavily on deep learning - and, in particular, transformers. Transformers make use of an “attention mechanism” for modeling interactions among inputs, which essentially involves doing nonlinear pairwise comparison between inputs and assigning different weights to tokens in a sequence, enabling a prioritization of some over others. The empirical effectiveness of this attention mechanism has led some in the community to claim that attention is “all you need” (the title of the original 2017 Google paper that introduced transformers).
