State-of-the-art speech models may exhibit suboptimal performance in specific population subgroups. Detecting these challenging cohorts is crucial for enhancing model robustness and fairness. Traditional methods for subgroup identification typically rely on demographic information such as age, gender, and origin. However, collecting such sensitive data at deployment time can be impractical or unfeasible due to privacy concerns.This paper introduces a novel Problematic Subgroup Identification model (PSI) to (i) automatically predict if an utterance belongs to problematic subgroups and (ii) provide an interpretable representation of these subgroups. PSI exploits confidence models (CMs) to encode information about sources of errors. CM fine-tuning based on problematic subgroup identification techniques allows accurate subgroup identification. PSI leverages demographic features only during its training, avoiding the need for sensitive data collection at deployment time. Experimental results on automatic speech recognition and intent classification datasets show PSI’s effectiveness in both identifying challenging subgroups and providing an interpretable subgroup description. These findings underscore the potential of PSI as a valuable tool for improving the robustness and fairness of speech models in real-world applications.

Leveraging confidence models for identifying challenging data subgroups in speech models / Koudounas, Alkis; Pastor, Eliana; Mazzia, Vittorio; Giollo, Manuel; Gueudre, Thomas; Reale, Elisa; Attanasio, Giuseppe; Cagliero, Luca; Cumani, Sandro; de Alfaro, Luca; Baralis, Elena; Amberti, Daniele. - ELETTRONICO. - (2024), pp. 134-138. (Intervento presentato al convegno 2024 IEEE International Conference on Acoustics, Speech and Signal Processing Workshop (ICASSPW) tenutosi a Seoul (KOR) nel 14-19 April, 2024) [10.1109/ICASSPW62465.2024.10626001].

Leveraging confidence models for identifying challenging data subgroups in speech models

Koudounas, Alkis;Pastor, Eliana;Mazzia, Vittorio;Gueudre, Thomas;Attanasio, Giuseppe;Cagliero, Luca;Cumani, Sandro;de Alfaro, Luca;Baralis, Elena;
2024

Abstract

State-of-the-art speech models may exhibit suboptimal performance in specific population subgroups. Detecting these challenging cohorts is crucial for enhancing model robustness and fairness. Traditional methods for subgroup identification typically rely on demographic information such as age, gender, and origin. However, collecting such sensitive data at deployment time can be impractical or unfeasible due to privacy concerns.This paper introduces a novel Problematic Subgroup Identification model (PSI) to (i) automatically predict if an utterance belongs to problematic subgroups and (ii) provide an interpretable representation of these subgroups. PSI exploits confidence models (CMs) to encode information about sources of errors. CM fine-tuning based on problematic subgroup identification techniques allows accurate subgroup identification. PSI leverages demographic features only during its training, avoiding the need for sensitive data collection at deployment time. Experimental results on automatic speech recognition and intent classification datasets show PSI’s effectiveness in both identifying challenging subgroups and providing an interpretable subgroup description. These findings underscore the potential of PSI as a valuable tool for improving the robustness and fairness of speech models in real-world applications.
2024
979-8-3503-7451-3
File in questo prodotto:
File Dimensione Formato  
leveraging-confidence-models-for-identifying-challenging-data-subgroups-in-speech-models.pdf

accesso aperto

Descrizione: Leveraging Confidence Models for Identifying Challenging Data Subgroups in Speech Models
Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Pubblico - Tutti i diritti riservati
Dimensione 194.1 kB
Formato Adobe PDF
194.1 kB Adobe PDF Visualizza/Apri
Leveraging_Confidence_Models_for_Identifying_Challenging_Data_Subgroups_in_Speech_Models.pdf

accesso riservato

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 880.67 kB
Formato Adobe PDF
880.67 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2986418