Developing deep learning techniques for super-resolving bursts of images acquired by mobile cameras is a topic that has recently gained significant interest. This topic fits the general problem of learning-based multi-image super-resolution (SR), which, contrary to its sibling single-image SR, has so far received little attention despite its potential. In this work, we introduce a neural network architecture for burst SR, called MLB-FuseNet (Multi-Level Burst Fusion Network), that is capable of extracting features in a manner that is invariant to permutations in the burst and to progressively condition features extracted from a reference image. Permutation invariance is desirable as it is known that the order of images in a burst does not matter in this problem, but its study has so far been neglected. Moreover, we also introduce a module exploiting a polyphase decomposition to improve feature extraction from mosaiced raw images. Results show an improvement over the state of the art on the BurstSR dataset – a recent and popular benchmark for this problem.

Multi-Level Fusion for Burst Super-Resolution with Deep Permutation-Invariant Conditioning / Cilia, Martina; Valsesia, Diego; Fracastoro, Giulia; Magli, Enrico. - ELETTRONICO. - (2023), pp. 1-5. (Intervento presentato al convegno ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) tenutosi a Rhodes Island, Greece nel 04-10 June 2023) [10.1109/ICASSP49357.2023.10096252].

Multi-Level Fusion for Burst Super-Resolution with Deep Permutation-Invariant Conditioning

Cilia, Martina;Valsesia, Diego;Fracastoro, Giulia;Magli, Enrico
2023

Abstract

Developing deep learning techniques for super-resolving bursts of images acquired by mobile cameras is a topic that has recently gained significant interest. This topic fits the general problem of learning-based multi-image super-resolution (SR), which, contrary to its sibling single-image SR, has so far received little attention despite its potential. In this work, we introduce a neural network architecture for burst SR, called MLB-FuseNet (Multi-Level Burst Fusion Network), that is capable of extracting features in a manner that is invariant to permutations in the burst and to progressively condition features extracted from a reference image. Permutation invariance is desirable as it is known that the order of images in a burst does not matter in this problem, but its study has so far been neglected. Moreover, we also introduce a module exploiting a polyphase decomposition to improve feature extraction from mosaiced raw images. Results show an improvement over the state of the art on the BurstSR dataset – a recent and popular benchmark for this problem.
2023
978-1-7281-6327-7
File in questo prodotto:
File Dimensione Formato  
cilia_icassp.pdf

non disponibili

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 5.86 MB
Formato Adobe PDF
5.86 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
cilia_icassp_auhor.pdf

accesso aperto

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 5.17 MB
Formato Adobe PDF
5.17 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2982655