The unreasonable effectiveness of (nonlinear) random matrix theory in artificial intelligence
Speaker:
Greg Yang, xAI
Date and Time:
Monday, November 13, 2023 - 11:30am to 12:30pm
Location:
Fields Institute, Room 230
Abstract:
By now, the so-called “hyperparameter transfer” technology has become a critical component of frontier artificial intelligence systems, including xAI’s Grok. It is made possible by Tensor Programs, a *nonlinear* random matrix theory in the same sense that neural networks are a nonlinear generalization of matrices. I will give an introduction to this theory. Still, many key mathematical questions remain, and they present a unique research area where mathematicians have direct control over the future of intelligence.