Robotics & AI

Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning

This research reveals that large-scale pretrained Vision-Language-Action (VLA) models exhibit remarkable resistance to catastrophic forgetting in continual learning. Unlike smaller models trained from scratch, VLAs achieve near-zero or even positive backward transfer with minimal replay data, indicating that learning new tasks can sometimes improve performance on previously learned ones. This resilience is attributed to the foundational knowledge acquired during pretraining, which allows VLAs to retain task-relevant information in their internal representations even when performance degrades, enabling rapid recovery with finetuning. The findings suggest a fundamental shift in continual learning dynamics for large pretrained models, emphasizing pretraining and representation reuse over complex algorithms or large replay buffers.