While monotone operator theory is often studied on Hilbert spaces, many interesting problems in machine learning and optimization arise naturally in finite-dimensional vector spaces endowed with non-Euclidean norms, such as diagonally-weighted `1 or `∞ norms. This paper provides a natural generalization of monotone operator theory to finitedimensional non-Euclidean spaces. The key tools are weak pairings and logarithmic norms. We show that the resolvent and reflected resolvent operators of non-Euclidean monotone mappings exhibit similar properties to their counterparts in Hilbert spaces. Furthermore, classical iterative methods and splitting methods for finding zeros of monotone operators are shown to converge in the non-Euclidean case. We apply our theory to equilibrium computation and Lipschitz constant estimation of recurrent neural networks, obtaining novel iterations and tighter upper bounds via forward-backward splitting
Non-Euclidean Monotone Operator Theory and Applications / Davydov, Alexander; Jafarpour, Saber; Proskurnikov, Anton V.; Bullo, Francesco. - In: JOURNAL OF MACHINE LEARNING RESEARCH. - ISSN 1532-4435. - 2024:25(2024), pp. 1-33.
Non-Euclidean Monotone Operator Theory and Applications
Anton V. Proskurnikov;Francesco Bullo
2024
Abstract
While monotone operator theory is often studied on Hilbert spaces, many interesting problems in machine learning and optimization arise naturally in finite-dimensional vector spaces endowed with non-Euclidean norms, such as diagonally-weighted `1 or `∞ norms. This paper provides a natural generalization of monotone operator theory to finitedimensional non-Euclidean spaces. The key tools are weak pairings and logarithmic norms. We show that the resolvent and reflected resolvent operators of non-Euclidean monotone mappings exhibit similar properties to their counterparts in Hilbert spaces. Furthermore, classical iterative methods and splitting methods for finding zeros of monotone operators are shown to converge in the non-Euclidean case. We apply our theory to equilibrium computation and Lipschitz constant estimation of recurrent neural networks, obtaining novel iterations and tighter upper bounds via forward-backward splittingFile | Dimensione | Formato | |
---|---|---|---|
23-0805-2.pdf
accesso aperto
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Creative commons
Dimensione
663.95 kB
Formato
Adobe PDF
|
663.95 kB | Adobe PDF | Visualizza/Apri |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2994482