Biography
Computer Graphics, Vision, and Deep Learning enthusiast.
Elena is a Postdoctoral Research Scientist and PhD Candidate at the University of Padova, where she received her MSc in ICT for Internet and Multimedia summa cum laude in 2021, and her BSc in Information Engineering in 2019.
She spent a year of her PhD working as an AI Researcher at Samsung Research UK.
Her main research interest lies in the intersection of Computer Vision and Graphics.
She focuses on foundation AI problems (continual learning, coarse-to-fine learning, domain adaptation, robustness, few-shot learning) for 3D semantic scene understanding, 3D reconstruction and extended reality. Her research interests derive from her passion for 3D computer graphics and digital arts.
Website: elenacamuffo.github.io
Research areas
Semantic Scene Understanding
Semantic Scene Understanding aims at recognizing objects. It comes as image recognition, object detection or semantic segmentation tasks and can involve different types of data: 2D (images), 3D (point clouds), or Multi-Modal data (mixed).
Foundation AI Problems
An advanced learning strategy investigates several ways to efficiently and effectively train neural networks. Scenarios can include a progressive change of class set, domain or distribution changes, or a lack of data to train or adapt the model.
Computer Graphics
Computer Graphics (CG) develops models using a computer-driven process. It is commonly used in animation, movies and video games. Research in CG includes rendering techniques, 3D reconstruction and new generative AI models.
Extended Reality
The term groups all the projections of virtual 3D objects to the user that extend its world. It comprises Augmented Reality (AR) and Virtual Reality (VR) and all Extended Realities (XR). It is usually optimized as it runs on Embedded Head Sets.
PUBLICATIONS
2024
Camuffo, Elena; Michieli, Umberto; Milani, Simone; Moon, Ji Joong; Ozay, Mete
Enhanced Model Robustness to Input Corruptions by Per-corruption Adaptation of Normalization Statistics Proceedings Article
In: IEEE, (Ed.): IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.
@inproceedings{nokey,
title = {Enhanced Model Robustness to Input Corruptions by Per-corruption Adaptation of Normalization Statistics},
author = {Elena Camuffo and Umberto Michieli and Simone Milani and Ji Joong Moon and Mete Ozay},
editor = {IEEE},
url = {https://arxiv.org/abs/2407.06450},
year = {2024},
date = {2024-06-30},
urldate = {2024-06-30},
booktitle = { IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
journal = {IEEE},
abstract = {Developing a reliable vision system is a fundamental challenge for robotic technologies (e.g., indoor service robots and outdoor autonomous robots) which can ensure reliable navigation even in challenging environments such as adverse weather conditions (e.g., fog, rain), poor lighting conditions (e.g., over/under exposure), or sensor degradation (e.g., blurring, noise), and can guarantee high performance in safety-critical functions. Current solutions proposed to improve model robustness usually rely on generic data augmentation techniques or employ costly test-time adaptation methods. In addition, most approaches focus on addressing a single vision task (typically, image recognition) utilising synthetic data. In this paper, we introduce Per-corruption Adaptation of Normalization statistics (PAN) to enhance the model robustness of vision systems. Our approach entails three key components: (i) a corruption type identification module, (ii) dynamic adjustment of normalization layer statistics based on identified corruption type, and (iii) real-time update of these statistics according to input data. PAN can integrate seamlessly with any convolutional model for enhanced accuracy in several robot vision tasks. In our experiments, PAN obtains robust performance improvement on challenging real-world corrupted image datasets (e.g., OpenLoris, ExDark, ACDC), where most of the current solutions tend to fail. Moreover, PAN outperforms the baseline models by 20-30% on synthetic benchmarks in object recognition tasks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Developing a reliable vision system is a fundamental challenge for robotic technologies (e.g., indoor service robots and outdoor autonomous robots) which can ensure reliable navigation even in challenging environments such as adverse weather conditions (e.g., fog, rain), poor lighting conditions (e.g., over/under exposure), or sensor degradation (e.g., blurring, noise), and can guarantee high performance in safety-critical functions. Current solutions proposed to improve model robustness usually rely on generic data augmentation techniques or employ costly test-time adaptation methods. In addition, most approaches focus on addressing a single vision task (typically, image recognition) utilising synthetic data. In this paper, we introduce Per-corruption Adaptation of Normalization statistics (PAN) to enhance the model robustness of vision systems. Our approach entails three key components: (i) a corruption type identification module, (ii) dynamic adjustment of normalization layer statistics based on identified corruption type, and (iii) real-time update of these statistics according to input data. PAN can integrate seamlessly with any convolutional model for enhanced accuracy in several robot vision tasks. In our experiments, PAN obtains robust performance improvement on challenging real-world corrupted image datasets (e.g., OpenLoris, ExDark, ACDC), where most of the current solutions tend to fail. Moreover, PAN outperforms the baseline models by 20-30% on synthetic benchmarks in object recognition tasks.
Barbato, Francesco; Camuffo, Elena; Milani, Simone; Zanuttigh, Pietro
Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network Proceedings Article
In: IEEE International Conference on Image Processing (ICIP), 2024.
@inproceedings{barbato2023continualb,
title = {Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network},
author = {Francesco Barbato and Elena Camuffo and Simone Milani and Pietro Zanuttigh},
url = {https://arxiv.org/pdf/2308.04702.pdf},
year = {2024},
date = {2024-06-25},
urldate = {2024-06-25},
booktitle = {IEEE International Conference on Image Processing (ICIP)},
journal = {arXiv preprint arXiv:2308.04702},
abstract = {State-of-the-art multimodal semantic segmentation approaches combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. Regrettably, this strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions (night-time, rain, fog, etc.) that make the acquired information unreliable. Moreover, these architectures tend to fail in continual learning scenarios. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly-coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. This makes our model reliable even in safetycritical settings, as is the case of autonomous driving. We evaluate our approach on the SemanticKITTI dataset, comparing it with our closest competitor. We also introduce an ad-hoc continual learning scheme and show results in a class-incremental continual learning scenario that prove the effectiveness of the approach also in this setting.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
State-of-the-art multimodal semantic segmentation approaches combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. Regrettably, this strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions (night-time, rain, fog, etc.) that make the acquired information unreliable. Moreover, these architectures tend to fail in continual learning scenarios. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly-coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. This makes our model reliable even in safetycritical settings, as is the case of autonomous driving. We evaluate our approach on the SemanticKITTI dataset, comparing it with our closest competitor. We also introduce an ad-hoc continual learning scheme and show results in a class-incremental continual learning scenario that prove the effectiveness of the approach also in this setting.
Camuffo, Elena; Michieli, Umberto; Moon, Ji Joong; Kim, Daehyun; Ozay, Mete
FFT-based Selection And Optimization Of Statistics For Robust Recognition Of Severely Corrupted Images Proceedings Article
In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.
@inproceedings{Camuffo2023d,
title = {FFT-based Selection And Optimization Of Statistics For Robust Recognition Of Severely Corrupted Images},
author = {Elena Camuffo and Umberto Michieli and Ji Joong Moon and Daehyun Kim and Mete Ozay},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10447639},
year = {2024},
date = {2024-01-26},
urldate = {2024-01-26},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
abstract = {Improving model robustness in case of corrupted images is among the key challenges to enable robust vision systems on smart devices, such as robotic agents. Particularly, robust test-time performance is imperative for most of the applications. This paper presents a novel approach to improve robustness of any classification model, especially on severely corrupted images. Our method (FROST) employs high-frequency features to detect input image corruption type, and select layer-wise feature normalization statistics. FROST provides the state-of-the-art results for different models and datasets, outperforming competitors on ImageNet-C by up to 37.1% relative gain, improving baseline of 40.9% mCE on severe corruptions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Improving model robustness in case of corrupted images is among the key challenges to enable robust vision systems on smart devices, such as robotic agents. Particularly, robust test-time performance is imperative for most of the applications. This paper presents a novel approach to improve robustness of any classification model, especially on severely corrupted images. Our method (FROST) employs high-frequency features to detect input image corruption type, and select layer-wise feature normalization statistics. FROST provides the state-of-the-art results for different models and datasets, outperforming competitors on ImageNet-C by up to 37.1% relative gain, improving baseline of 40.9% mCE on severe corruptions.
2023
Devid, Campagnolo*; Elena, Camuffo*; Umberto, Michieli; Paolo, Borin; Simone, Milani; Andrea, Giordano
Fully Automated Scan-to-BIM via Point Cloud Instance Segmentation Proceedings Article
In: International Conference on Image Processing (ICIP), IEEE 2023.
@inproceedings{Camuffo2023c,
title = {Fully Automated Scan-to-BIM via Point Cloud Instance Segmentation},
author = {Campagnolo* Devid and Camuffo* Elena and Michieli Umberto and Borin Paolo and Milani Simone and Giordano Andrea
},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10222064},
year = {2023},
date = {2023-09-13},
urldate = {2023-09-13},
booktitle = {International Conference on Image Processing (ICIP)},
organization = {IEEE},
abstract = {Digital reconstruction through Building Information Models (BIM) is a valuable methodology for documenting and analyzing existing buildings. Its pipeline starts with geometric acquisition. (e.g., via photogrammetry or laser scanning) for accurate point cloud collection. However, the acquired data are noisy and unstructured, and the creation of a semanticallymeaningful BIM representation requires a huge computational effort, as well as expensive and time-consuming human annotations. In this paper, we propose a fully automated scan-to-BIM pipeline. The approach relies on: (i) our dataset (HePIC), acquired from two large buildings and annotated at a point-wise semantic level based on existent BIM models; (ii) a novel ad hoc deep network (BIM-Net++) for semantic segmentation, whose output is then processed to extract instance information necessary to recreate BIM objects; (iii) novel model pretraining and class re-weighting to eliminate the need for a large amount of labeled data and human intervention.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Digital reconstruction through Building Information Models (BIM) is a valuable methodology for documenting and analyzing existing buildings. Its pipeline starts with geometric acquisition. (e.g., via photogrammetry or laser scanning) for accurate point cloud collection. However, the acquired data are noisy and unstructured, and the creation of a semanticallymeaningful BIM representation requires a huge computational effort, as well as expensive and time-consuming human annotations. In this paper, we propose a fully automated scan-to-BIM pipeline. The approach relies on: (i) our dataset (HePIC), acquired from two large buildings and annotated at a point-wise semantic level based on existent BIM models; (ii) a novel ad hoc deep network (BIM-Net++) for semantic segmentation, whose output is then processed to extract instance information necessary to recreate BIM objects; (iii) novel model pretraining and class re-weighting to eliminate the need for a large amount of labeled data and human intervention.
Mari, Daniele; Camuffo, Elena; Milani, Simone
CACTUS: Content-Aware Compression and Transmission Using Semantics for Automotive LiDAR Data Journal Article
In: Sensors, vol. 23, iss. 12, 2023.
@article{Mari,
title = {CACTUS: Content-Aware Compression and Transmission Using Semantics for Automotive LiDAR Data},
author = {Daniele Mari and Elena Camuffo and Simone Milani},
editor = {MDPI},
url = {https://www.mdpi.com/1424-8220/23/12/5611},
doi = {10.3390/s23125611},
year = {2023},
date = {2023-06-15},
urldate = {2023-06-15},
journal = {Sensors},
volume = {23},
issue = {12},
abstract = {Many recent cloud or edge computing strategies for automotive applications require transmitting huge amounts of Light Detection and Ranging (LiDAR) data from terminals to centralized processing units. As a matter of fact, the development of effective Point Cloud (PC) compression strategies that preserve semantic information, which is critical for scene understanding, proves to be crucial. Segmentation and compression have always been treated as two independent tasks; however, since not all the semantic classes are equally important for the end task, this information can be used to guide data transmission. In this paper, we propose Content-Aware Compression and Transmission Using Semantics (CACTUS), which is a coding framework that exploits semantic information to optimize the data transmission, partitioning the original point set into separate data streams. Experimental results show that differently from traditional strategies, the independent coding of semantically consistent point sets preserves class information. Additionally, whenever semantic information needs to be transmitted to the receiver, using the CACTUS strategy leads to gains in terms of compression efficiency, and more in general, it improves the speed and flexibility of the baseline codec used to compress the data.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Many recent cloud or edge computing strategies for automotive applications require transmitting huge amounts of Light Detection and Ranging (LiDAR) data from terminals to centralized processing units. As a matter of fact, the development of effective Point Cloud (PC) compression strategies that preserve semantic information, which is critical for scene understanding, proves to be crucial. Segmentation and compression have always been treated as two independent tasks; however, since not all the semantic classes are equally important for the end task, this information can be used to guide data transmission. In this paper, we propose Content-Aware Compression and Transmission Using Semantics (CACTUS), which is a coding framework that exploits semantic information to optimize the data transmission, partitioning the original point set into separate data streams. Experimental results show that differently from traditional strategies, the independent coding of semantically consistent point sets preserves class information. Additionally, whenever semantic information needs to be transmitted to the receiver, using the CACTUS strategy leads to gains in terms of compression efficiency, and more in general, it improves the speed and flexibility of the baseline codec used to compress the data.
Camuffo, Elena; Milani, Simone
Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data Proceedings Article
In: International Conference of Computer Vision and Pattern Recognition Workshops, 2023.
@inproceedings{Camuffo2023b,
title = {Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data},
author = {Elena Camuffo and Simone Milani},
url = {https://arxiv.org/abs/2304.03980},
doi = {https://doi.org/10.48550/arXiv.2304.03980},
year = {2023},
date = {2023-04-10},
urldate = {2023-04-10},
booktitle = {International Conference of Computer Vision and Pattern Recognition Workshops},
abstract = {During the last few years, continual learning (CL) strategies for image classification and segmentation have been widely investigated designing innovative solutions to tackle catastrophic forgetting, like knowledge distillation and self-inpainting. However, the application of continual learning paradigms to point clouds is still unexplored and investigation is required, especially using architectures that capture the sparsity and uneven distribution of LiDAR data. The current paper analyzes the problem of class incremental learning applied to point cloud semantic segmentation, comparing approaches and state-of-the-art architectures. To the best of our knowledge, this is the first example of class-incremental continual learning for LiDAR point cloud semantic segmentation. Different CL strategies were adapted to LiDAR point clouds and tested, tackling both classic fine-tuning scenarios and the Coarse-to-Fine learning paradigm. The framework has been evaluated through two different architectures on SemanticKITTI, obtaining results in line with state-of-the-art CL strategies and standard offline learning.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
During the last few years, continual learning (CL) strategies for image classification and segmentation have been widely investigated designing innovative solutions to tackle catastrophic forgetting, like knowledge distillation and self-inpainting. However, the application of continual learning paradigms to point clouds is still unexplored and investigation is required, especially using architectures that capture the sparsity and uneven distribution of LiDAR data. The current paper analyzes the problem of class incremental learning applied to point cloud semantic segmentation, comparing approaches and state-of-the-art architectures. To the best of our knowledge, this is the first example of class-incremental continual learning for LiDAR point cloud semantic segmentation. Different CL strategies were adapted to LiDAR point clouds and tested, tackling both classic fine-tuning scenarios and the Coarse-to-Fine learning paradigm. The framework has been evaluated through two different architectures on SemanticKITTI, obtaining results in line with state-of-the-art CL strategies and standard offline learning.
Camuffo, Elena; Michieli, Umberto; Milani, Simone
Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation Journal Article
In: IEEE Transactions on Multimedia, pp. 1-11, 2023.
@article{Camuffo2023,
title = {Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation},
author = {Elena Camuffo and Umberto Michieli and Simone Milani},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10368362},
doi = {10.1109/TMM.2023.3345152},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Multimedia},
pages = {1-11},
abstract = {Recent advances in autonomous robotic technologies have highlighted the growing need for precise environmental analysis. LiDAR semantic segmentation has gained attention to accomplish fine-grained scene understanding by acting directly on raw content provided by sensors. Recent solutions showed how different learning techniques can be used to improve the performance of the model, without any architectural or dataset change. Following this trend, we present a coarse-to-fine setup that LEArns from classification mistaKes (LEAK) derived from a standard model. First, classes are clustered into macro groups according to mutual prediction errors; then, the learning process is regularized by: (1) aligning class-conditional prototypical feature representation for both fine and coarse classes, (2) weighting instances with a per-class fairness index. Our LEAK approach is very general and can be seamlessly applied on top of any segmentation architecture; indeed, experimental results showed that it enables state-of-the-art performances on different architectures, datasets and tasks, while ensuring more balanced class-wise results and faster convergence.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent advances in autonomous robotic technologies have highlighted the growing need for precise environmental analysis. LiDAR semantic segmentation has gained attention to accomplish fine-grained scene understanding by acting directly on raw content provided by sensors. Recent solutions showed how different learning techniques can be used to improve the performance of the model, without any architectural or dataset change. Following this trend, we present a coarse-to-fine setup that LEArns from classification mistaKes (LEAK) derived from a standard model. First, classes are clustered into macro groups according to mutual prediction errors; then, the learning process is regularized by: (1) aligning class-conditional prototypical feature representation for both fine and coarse classes, (2) weighting instances with a per-class fairness index. Our LEAK approach is very general and can be seamlessly applied on top of any segmentation architecture; indeed, experimental results showed that it enables state-of-the-art performances on different architectures, datasets and tasks, while ensuring more balanced class-wise results and faster convergence.
2022
Camuffo, Elena; Battisti, Federica; Pham, Francesco; Milani, Simone
Deep 3D Model Optimization for Immersive and Interactive Applications Proceedings Article
In: 2022 10th European Workshop on Visual Information Processing (EUVIP), pp. 1–6, IEEE 2022.
@inproceedings{Camuffo2022a,
title = {Deep 3D Model Optimization for Immersive and Interactive Applications},
author = {Elena Camuffo and Federica Battisti and Francesco Pham and Simone Milani},
url = {https://ieeexplore.ieee.org/abstract/document/9922742},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 10th European Workshop on Visual Information Processing (EUVIP)},
pages = {1--6},
organization = {IEEE},
abstract = {The growing diffusion of immersive and interactive applications is posing new challenges in the multimedia processing chain. When dealing with AR and VR applications, the most relevant aspects to consider are the (1) quality of the visualized 3D objects and (2) the fluidity in the visualization in case the user is moving in the environment. In this framework, we propose a deep learning based approach that estimates the optimal model parameters to be used in relation to the viewer’s movement and the model characteristics and quality. The performed tests show the effectiveness of the proposed approach.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The growing diffusion of immersive and interactive applications is posing new challenges in the multimedia processing chain. When dealing with AR and VR applications, the most relevant aspects to consider are the (1) quality of the visualized 3D objects and (2) the fluidity in the visualization in case the user is moving in the environment. In this framework, we propose a deep learning based approach that estimates the optimal model parameters to be used in relation to the viewer’s movement and the model characteristics and quality. The performed tests show the effectiveness of the proposed approach.
Camuffo, Elena; Mari, Daniele; Milani, Simone
Recent advancements in learning algorithms for point clouds: An updated overview Journal Article
In: Sensors, vol. 22, no. 4, pp. 1357, 2022.
@article{Camuffo2022,
title = {Recent advancements in learning algorithms for point clouds: An updated overview},
author = {Elena Camuffo and Daniele Mari and Simone Milani},
url = {https://www.mdpi.com/1424-8220/22/4/1357},
doi = {https://doi.org/10.3390/s22041357},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Sensors},
volume = {22},
number = {4},
pages = {1357},
publisher = {MDPI},
abstract = {Recent advancements in self-driving cars, robotics, and remote sensing have widened the range of applications for 3D Point Cloud (PC) data. This data format poses several new issues concerning noise levels, sparsity, and required storage space; as a result, many recent works address PC problems using Deep Learning (DL) solutions thanks to their capability to automatically extract features and achieve high performances. Such evolution has also changed the structure of processing chains and posed new problems to both academic and industrial researchers. The aim of this paper is to provide a comprehensive overview of the latest state-of-the-art DL approaches for the most crucial PC processing operations, i.e., semantic scene understanding, compression, and completion. With respect to the existing reviews, the work proposes a new taxonomical classification of the approaches, taking into account the characteristics of the acquisition set up, the peculiarities of the acquired PC data, the presence of side information (depending on the adopted dataset), the data formatting, and the characteristics of the DL architectures. This organization allows one to better comprehend some final performance comparisons on common test sets and cast a light on the future research trends.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent advancements in self-driving cars, robotics, and remote sensing have widened the range of applications for 3D Point Cloud (PC) data. This data format poses several new issues concerning noise levels, sparsity, and required storage space; as a result, many recent works address PC problems using Deep Learning (DL) solutions thanks to their capability to automatically extract features and achieve high performances. Such evolution has also changed the structure of processing chains and posed new problems to both academic and industrial researchers. The aim of this paper is to provide a comprehensive overview of the latest state-of-the-art DL approaches for the most crucial PC processing operations, i.e., semantic scene understanding, compression, and completion. With respect to the existing reviews, the work proposes a new taxonomical classification of the approaches, taking into account the characteristics of the acquisition set up, the peculiarities of the acquired PC data, the presence of side information (depending on the adopted dataset), the data formatting, and the characteristics of the DL architectures. This organization allows one to better comprehend some final performance comparisons on common test sets and cast a light on the future research trends.