Continual learning is the ability of a learning system (e.g., a neural network) to learn to recognize new concepts in subsequent learning steps without forgetting or deteriorating too much the performance on previously learned ones. It can be tackled in many different computer vision settings: in the MEDIA Lab we were among the firsts to tackle continual learning for semantic segmentation with deep networks.
Key research topics include:
- We performed the first investigation of the incremental learning problem for semantic segmentation, where the problem is formally introduced. To tackle this task we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones.
- We propose a continual learning scheme that shapes the latent space to reduce forgetting whilst improving the recognition of novel classes. Our framework, called SDR exploits prototypes matching, features sparsification and contrastive learning.
- We also tackled continual coarse-to-fine refinements in semantic segmentation: both at the spatial level (i.e, the decomposition of object-level classes into their respective parts) and at the semantic level (classes are hierarchically derived from subdividing the classes of the initial set).
- The usage of replay data obtained through Generative Approaches and web-crawling has been considered.
- The task has also been combined with Domain Adaptation.
Recent publications:
Learning with Style: Continual Semantic Segmentation Across Tasks and Domains Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Learning from the Web: Language Drives Weakly-Supervised Incremental Learning for Semantic Segmentation Proceedings Article
In: European Conference on Computer Vision, Springer 2024.
Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data Proceedings Article
In: International Conference of Computer Vision and Pattern Recognition Workshops, 2023.
Continual coarse-to-fine domain adaptation in semantic segmentation Journal Article
In: Image and Vision Computing, vol. 121, pp. 104426, 2022.
Domain adaptation and continual learning in semantic segmentation Book Section
In: Advanced Methods and Deep Learning in Computer Vision, pp. 275–303, Academic Press, 2022.
Edge-Aware Graph Matching Network for Part-Based Semantic Segmentation Journal Article
In: International Journal of Computer Vision, vol. 130, no. 11, pp. 2797–2821, 2022.
Knowledge distillation for incremental learning in semantic segmentation Journal Article
In: Computer Vision and Image Understanding, vol. 205, pp. 103167, 2021.
Continual semantic segmentation via repulsion-attraction of sparse and disentangled latent representations Proceedings Article
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1114–1124, 2021.
Recall: Replay-based continual learning in semantic segmentation Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7026–7035, 2021.
Incremental learning techniques for semantic segmentation Proceedings Article
In: Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0–0, 2019.