In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address this problem with a novel framework called Pixel-By-Pixel Cross-Domain Alignment (PixDA). We propose a novel pixel-by-pixel domain adversarial loss following three criteria: (i) align the source and the target domain for each pixel, (ii) avoid negative transfer on the correctly represented pixels, and (iii) regularize the training of infrequent classes to avoid overfitting. The pixel-wise adversarial training is assisted by a novel sample selection procedure, that handles the imbalance between source and target data, and a knowledge distillation strategy, that avoids overfitting towards the few target images. We demonstrate on standard synthetic-to-real benchmarks that PixDA outperforms previous state-of-the-art methods in (1-5)-shot settings.
Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation / Tavera, Antonio; Cermelli, Fabio; Masone, Carlo; Caputo, Barbara. - ELETTRONICO. - (2022), pp. 1959-1968. (Intervento presentato al convegno 2022 IEEE Winter Conference on Applications of Computer Vision (WACV) tenutosi a Waikoloa, Hawaii nel 04/01/2022-08/01/2022) [10.1109/WACV51458.2022.00202].
Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation
Tavera, Antonio;Cermelli, Fabio;Masone, Carlo;Caputo, Barbara
2022
Abstract
In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address this problem with a novel framework called Pixel-By-Pixel Cross-Domain Alignment (PixDA). We propose a novel pixel-by-pixel domain adversarial loss following three criteria: (i) align the source and the target domain for each pixel, (ii) avoid negative transfer on the correctly represented pixels, and (iii) regularize the training of infrequent classes to avoid overfitting. The pixel-wise adversarial training is assisted by a novel sample selection procedure, that handles the imbalance between source and target data, and a knowledge distillation strategy, that avoids overfitting towards the few target images. We demonstrate on standard synthetic-to-real benchmarks that PixDA outperforms previous state-of-the-art methods in (1-5)-shot settings.File | Dimensione | Formato | |
---|---|---|---|
PixDA___WACV2022_Submission.pdf
accesso riservato
Descrizione: Articolo principale
Tipologia:
1. Preprint / submitted version [pre- review]
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
4.77 MB
Formato
Adobe PDF
|
4.77 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pixel-by-Pixel_Cross-Domain_Alignment_for_Few-Shot_Semantic_Segmentation.pdf
accesso riservato
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
2.49 MB
Formato
Adobe PDF
|
2.49 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2929172