In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address this problem with a novel framework called Pixel-By-Pixel Cross-Domain Alignment (PixDA). We propose a novel pixel-by-pixel domain adversarial loss following three criteria: (i) align the source and the target domain for each pixel, (ii) avoid negative transfer on the correctly represented pixels, and (iii) regularize the training of infrequent classes to avoid overfitting. The pixel-wise adversarial training is assisted by a novel sample selection procedure, that handles the imbalance between source and target data, and a knowledge distillation strategy, that avoids overfitting towards the few target images. We demonstrate on standard synthetic-to-real benchmarks that PixDA outperforms previous state-of-the-art methods in (1-5)-shot settings.

Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation / Tavera, Antonio; Cermelli, Fabio; Masone, Carlo; Caputo, Barbara. - ELETTRONICO. - (2022), pp. 1959-1968. (Intervento presentato al convegno 2022 IEEE Winter Conference on Applications of Computer Vision (WACV) tenutosi a Waikoloa, Hawaii nel 04/01/2022-08/01/2022) [10.1109/WACV51458.2022.00202].

Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation

Tavera, Antonio;Cermelli, Fabio;Masone, Carlo;Caputo, Barbara
2022

Abstract

In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address this problem with a novel framework called Pixel-By-Pixel Cross-Domain Alignment (PixDA). We propose a novel pixel-by-pixel domain adversarial loss following three criteria: (i) align the source and the target domain for each pixel, (ii) avoid negative transfer on the correctly represented pixels, and (iii) regularize the training of infrequent classes to avoid overfitting. The pixel-wise adversarial training is assisted by a novel sample selection procedure, that handles the imbalance between source and target data, and a knowledge distillation strategy, that avoids overfitting towards the few target images. We demonstrate on standard synthetic-to-real benchmarks that PixDA outperforms previous state-of-the-art methods in (1-5)-shot settings.
File in questo prodotto:
File Dimensione Formato  
PixDA___WACV2022_Submission.pdf

accesso riservato

Descrizione: Articolo principale
Tipologia: 1. Preprint / submitted version [pre- review]
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 4.77 MB
Formato Adobe PDF
4.77 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pixel-by-Pixel_Cross-Domain_Alignment_for_Few-Shot_Semantic_Segmentation.pdf

accesso riservato

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 2.49 MB
Formato Adobe PDF
2.49 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2929172