Our paper “SalFormer360: A Transformer-Based Saliency Estimation Model for 360-Degree Videos” has been recently published in IEEE Transactions on Broadcasting.
We propose a new method for saliency estimation of 360-degree videos built on a transformer-based architecture. Our approach is based on the combination of an existing encoder architecture, SegFormer, and a custom decoder. The SegFormer model was originally developed for 2D segmentation tasks, and it has been fine-tuned to adapt it to 360-degree content. To further enhance prediction accuracy in our model, we incorporated a viewing center bias to reflect user attention in 360-degree environments. Extensive experiments on the three largest benchmark datasets for saliency estimation demonstrate that SalFormer360 outperforms existing state-of-the-art methods.