publications

2023

1. Multi-Head Adapter Routing for Cross-Task Generalization

   Lucas Caccia, Edoardo Ponti, Lucas Liu, and 3 more authors

   Advances in neural information processing systems, 2023

2022

1. On Anytime Learning at Macroscale

   Lucas Caccia, Jing Xu, Myle Ott, and 2 more authors

   In Conference on Lifelong Learning Agents, CoLLAs 2022, 22-24 August 2022, McGill University, Montréal, Québec, Canada, 2022
2. New Insights on Reducing Abrupt Representation Change in Online Continual Learning

   Lucas Caccia, Rahaf Aljundi, Nader Asadi, and 3 more authors

   In International Conference on Learning Representations, 2022

2020

1. Online Learned Continual Compression with Adaptive Quantization Modules

   Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and 1 more author

   In Proceedings of the 37th International Conference on Machine Learning, 13–18 jul 2020

   Abs PDF

   We introduce and study the problem of Online Continual Compression, where one attempts to simultaneously learn to compress and store a representative dataset from a non i.i.d data stream, while only observing each sample once. A naive application of auto-encoder in this setting encounters a major challenge: representations derived from earlier encoder states must be usable by later decoder states. We show how to use discrete auto-encoders to effectively address this challenge and introduce Adaptive Quantization Modules (AQM) to control variation in the compression ability of the module at any given stage of learning. This enables selecting an appropriate compression for incoming samples, while taking into account overall memory constraints and current progress of the learned compression. Unlike previous methods, our approach does not require any pretraining, even on challenging datasets. We show that using AQM to replace standard episodic memory in continual learning settings leads to significant gains on continual learning benchmarks with images, LiDAR, and reinforcement learning agents.

2019

1. Online Continual Learning with Maximal Interfered Retrieval

   Rahaf Aljundi, Lucas Page-Caccia, Eugene Belilovsky, and 4 more authors

   In Advances in Neural Information Processing Systems, 13–18 jul 2019