Georgia Institute of Technology

Featuring Jacob Andreas, Massachusetts Institute of Technology

Abstract: In the age of deep networks, "learning" almost invariably means "learning from examples". We train language models with human-generated text and labeled preference pairs, mage classifiers with large datasets of images, and robot policies with rollouts or demonstrations. When human learners acquire new concepts and skills, we often do so with richer supervision, especially in the form of language---we learn new concepts from examples accompanied by descriptions or definitions, and new skills from demonstrations accompanied by instructions. Current language models (LMs) support a limited form of language-based teaching via prompting, but it remains challenging to use natural language supervision to apply global, persistent changes to learned models. This talk will focus on two recent projects aimed at more effectively supervising LMs using language: first, on *eliciting* new information (by asking questions to human users of LMs); second, on *updating* language models to incorporate new information (by using LMs to automatically ask and answer questions about information implied by, but not explicitly stated in, training data). If time permits, I'll also discuss some applications of these techniques to educational settings (where we can optimize questions for human, rather than machine, learning). This is joint work with Belinda Li, Alex Tamkin, Noah Goodman, Feyza Akyürek, Ekin Akyürek, Leshem Choshen, Derry Wijaya, and Alexis Ross.

Bio: Jacob Andreas is an associate professor at MIT in the Department of Electrical Engineering and Computer Science as well as the Computer Science and Artificial Intelligence Laboratory. His research aims to understand the computational foundations of language learning, and to build intelligent systems that can learn from human guidance. Jacob earned his Ph.D. from UC Berkeley, his M.Phil. from Cambridge (where he studied as a Churchill scholar) and his B.S. from Columbia. He has received a Sloan fellowship, an NSF CAREER award, MIT's Junior Bose and Kolokotrones teaching awards, and paper awards at ACL, ICML and NAACL.