Continual Learning

Continual learning is the ability of a learning system (e.g., a neural network) to learn to recognize new concepts in subsequent learning steps without forgetting or deteriorating too much the performance on previously learned ones. It can be tackled in many different computer vision settings: in the MEDIA Lab we were among the firsts to tackle continual learning for semantic segmentation with deep networks.

Key research topics include:

  • We performed the first investigation of the incremental learning problem for semantic segmentation, where the problem is formally introduced. To tackle this task we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones.
  • We propose a continual learning scheme that shapes the latent space to reduce forgetting whilst improving the recognition of novel classes. Our framework, called SDR exploits prototypes matching, features sparsification and contrastive learning.
  • We also tackled continual coarse-to-fine refinements in semantic segmentation: both at the spatial level (i.e, the decomposition of object-level classes into their respective parts) and at the semantic level (classes are hierarchically derived from subdividing the classes of the initial set).
  • The usage of replay data obtained through Generative Approaches and web-crawling has been considered.
  • The task has also been combined with Domain Adaptation.

Recent publications:

Shenaj, Donald; Barbato, Francesco; Michieli, Umberto; Zanuttigh, Pietro

Continual coarse-to-fine domain adaptation in semantic segmentation Journal Article

In: Image and Vision Computing, vol. 121, pp. 104426, 2022.

Links | BibTeX

Michieli, Umberto; Toldo, Marco; Zanuttigh, Pietro

Domain adaptation and continual learning in semantic segmentation Book Section

In: Advanced Methods and Deep Learning in Computer Vision, pp. 275–303, Academic Press, 2022.

BibTeX

Michieli, Umberto; Zanuttigh, Pietro

Edge-Aware Graph Matching Network for Part-Based Semantic Segmentation Journal Article

In: International Journal of Computer Vision, vol. 130, no. 11, pp. 2797–2821, 2022.

BibTeX

Toldo, Marco; Michieli, Umberto; Zanuttigh, Pietro

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains Journal Article

In: arXiv preprint arXiv:2210.07016, 2022.

BibTeX

Michieli, Umberto; Zanuttigh, Pietro

Knowledge distillation for incremental learning in semantic segmentation Journal Article

In: Computer Vision and Image Understanding, vol. 205, pp. 103167, 2021.

BibTeX

Michieli, Umberto; Zanuttigh, Pietro

Continual semantic segmentation via repulsion-attraction of sparse and disentangled latent representations Proceedings Article

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1114–1124, 2021.

BibTeX

Maracani, Andrea; Michieli, Umberto; Toldo, Marco; Zanuttigh, Pietro

Recall: Replay-based continual learning in semantic segmentation Proceedings Article

In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7026–7035, 2021.

BibTeX

Michieli, Umberto; Zanuttigh, Pietro

Incremental learning techniques for semantic segmentation Proceedings Article

In: Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0–0, 2019.

BibTeX