Lucas Caccia

I completed in November my Ph.D. at McGill and the Quebec Artificial Intelligence Institute (Mila), where I was advised by Joelle Pineau. Over the course of my graduate studies, I had the opportunity to intern at Meta AI where I worked with Ludovic Denoyer and Marc’Aurelio Rantazo on Continual Learning, and at Microsoft Research under the guidance of Nicolas Le Roux and Alessandro Sordoni, working on modular adaptation in LLMs.

My research centers on building agents that can accumulate knowledge and over time, enabling them to quickly generalize to new tasks in a data and compute efficient way. My work as thus been primarily focused on areas including continual,meta,transfer,multitask learning. More broadly, I believe that a viable solution to efficient adaptation is modularity. This involves creating neural networks that use specialized, independent modules to process information. To this end, I have been focusing lately on designing better algorithms to route and combine modules at a more finegrained level, and on the role of modular methods in multitask learning.

I am currently doing a post-doc at Microsoft Research Montréal, where I am leveraging these principles to adapt LLMs for massively multitask transfer. On one side, as part of project Maia, we are using these modular approaches for generative modeling of individual behavior at scale, efficiently performing stylometry over a pool of several thousand players. Another project I am involved in is the creation of an Adapter Universe, whereby using modularity to learn reusable and composable adapters over pretrained LLMs, we are aiming to build a large repertoire of such adapters that users can leverage for efficient and performant transfer to their downstream tasks of interest.