Our daily life is profoundly affected by the adoption of au- tomated decision making (ADM) systems due to the ongoing tendency of humans to delegate machines to take decisions. The unleashed usage of ADM systems was facilitated by the availability of large-scale data, alongside with the deployment of devices and equipment. This trend re- sulted in an increasing influence of ADM systems’ output over several aspects of our life, with possible discriminatory consequences towards certain individuals or groups. In this context, we focus on input data by investigating measurable characteristics which can lead to discriminating automated decisions. In particular, we identified two indexes of hetero- geneity and diversity, and tested them on two datasets. A limitation we found is the index sensitivity to a large number of categories, but on the whole results show that the indexes reflect well imbalances in the input data. Future work is required to further assess the reliability of these indexes as indicators of discrimination risks in the context of ADM, in order to foster a more conscious and responsible use of ADM systems through an immediate investigation on input data.

Identifying Risks in Datasets for Automated Decision–Making / Mecati, Mariachiara; Cannavò, Flavio Emanuele; Vetrò, Antonio; Torchiano, Marco. - STAMPA. - 12219:(2020), pp. 332-344. (Intervento presentato al convegno EGOV2020 – IFIP EGOV-CeDEM-EPART 2020 tenutosi a Linköping University (Sweden) nel 31 August (Monday) – 2 September (Wednesday), 2020) [10.1007/978-3-030-57599-1_25].

Identifying Risks in Datasets for Automated Decision–Making

Mecati, Mariachiara;Cannavò, Flavio Emanuele;Vetrò, Antonio;Torchiano, Marco
2020

Abstract

Our daily life is profoundly affected by the adoption of au- tomated decision making (ADM) systems due to the ongoing tendency of humans to delegate machines to take decisions. The unleashed usage of ADM systems was facilitated by the availability of large-scale data, alongside with the deployment of devices and equipment. This trend re- sulted in an increasing influence of ADM systems’ output over several aspects of our life, with possible discriminatory consequences towards certain individuals or groups. In this context, we focus on input data by investigating measurable characteristics which can lead to discriminating automated decisions. In particular, we identified two indexes of hetero- geneity and diversity, and tested them on two datasets. A limitation we found is the index sensitivity to a large number of categories, but on the whole results show that the indexes reflect well imbalances in the input data. Future work is required to further assess the reliability of these indexes as indicators of discrimination risks in the context of ADM, in order to foster a more conscious and responsible use of ADM systems through an immediate investigation on input data.
2020
978-3-030-57598-4
978-3-030-57599-1
File in questo prodotto:
File Dimensione Formato  
DEFINITIVO_Paper_EGOV2020.pdf

accesso aperto

Descrizione: post print egov 2020
Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 295.45 kB
Formato Adobe PDF
295.45 kB Adobe PDF Visualizza/Apri
PUB-2020-egov-risks.pdf

non disponibili

Descrizione: versione editoriale egov 2020
Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 448.4 kB
Formato Adobe PDF
448.4 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2843005