Joint BCAM-UPV/EHU Data Science and Artificial Intelligence seminar: Self-Composing Policies for Scalable Continual Reinforcement Learning
Date: Fri, May 19 2023
Location: UPV/EHU Donosti, Faculty of Computer Science, room 3.1 and Online
Speakers: Mikel Malagon (UPV/EHU)
Link to the session here
Continual reinforcement learning aims to develop agents that learn in a never-ending stream of tasks and leverage the knowledge obtained from solving previous problems to solve new ones. However, allowing such knowledge transfer while avoiding catastrophic forgetting and interference is one of the main challenges of the field. In this work, we propose a growable and modular Neural Network (NN) architecture that naturally avoids the mentioned issues by instantiating a new policy module every time a new task is introduced. Moreover, the NN architecture of each module enables selectively composing preceding policies together with its internal policy for the purpose of accelerating solving the current task. Conducted experiments show that the proposed architecture is able to transfer knowledge in sequences of continuous control problems as well as in visual control tasks, such as MuJoCo and Atari. Unlike previous growing NN approaches, we also show that the number of parameters of the proposed approach grows linearly with respect to the number of tasks, and does not sacrifice plasticity to scale. Finally, we shed some light on the possibility of using the presented method to perform knowledge transfer across different Atari games.
Non-self-adjoint operators and their spectra
Aula BCAM-UPV/EHU Seminar: Relevant phenomena in pedestrian dynamics and how to computationally model them.
Dariel Hernández (BCAM)