Reinforcement Learning & Continual AI

TSN-Affinity addresses CORL by proposing a novel architectural approach centered on TinySubNetworks and a Decision Transformer backbone. This method enables task-specific parameterization and controlled knowledge sharing through an innovative RL-aware reuse strategy called Affinity Routing. Unlike prior methods, TSN-Affinity dynamically decides whether to reuse an existing model copy or initialize a new one based on the similarity of incoming tasks, ensuring efficient knowledge transfer while preventing destructive interference and preserving performance on previously learned tasks.

At its core, TSN-Affinity leverages Action Affinity (evaluating policy-level transfer via cross-entropy or mean-squared error) and Latent Affinity (comparing tasks at the representation level using KL divergence on observation encoder outputs). These affinity scores guide the routing decision. When reusing a model copy, previously allocated weights remain frozen and protected, with new task learning confined to sparse masks and newly allocated parameters. This strict protection prevents forgetting and allows for effective functional reuse of components. The method operates on Decision Transformer objectives, adapting for discrete (Atari) and continuous (Panda) action spaces.

The method was rigorously evaluated on two distinct CORL benchmarks: discrete-control Atari games and continuous-control Franka Emika Panda robotic arm manipulation tasks. On Atari, TSN-Affinity achieved near-zero catastrophic forgetting and significantly improved multi-task performance compared to replay-based baselines. For Panda, a more challenging domain, affinity routing provided the best trade-off between transfer and retention, outperforming observation-level similarity routing. This demonstrates the robustness of similarity-guided architectural reuse across diverse control regimes.