Equilibrium propagation is a learning technique conceived for training continuous-time recurrent neural networks. It offers some notable advantages when compared to conventional back-propagation-based algorithms and to classical design methods. From an implementation perspective, it demands only a single computational circuit. Theoretically, although it seeks to minimize a cost function, it exhibits similarities to spike- timing-dependent plasticity (STDP), rendering it, to a certain extent, biologically plausible. This paper explores the global dynamic behavior of continuous-time piecewise linear networks trained through equilibrium point propagation. We examine a network in which the target patterns are presented as external inputs rather than as initial conditions. We first show that the learning rules, which extend equilibrium propagation to gradient- like and non-symmetric networks, can be derived as a suitable approximation of Lagrangian optimization. Then, by studying a relatively simple but thoroughly significant case, we demonstrate that a detailed analysis of the equilibrium point distribution yields a profound understanding of the network’s fundamental proper- ties and provides a valuable tool for quantitatively evaluating the network’s accuracy. Compared to classical synthesis techniques, our approach, where patterns are introduced as external inputs, in most cases, circumvents the impractical task of estimating the basins of attraction for sets of multiple equilibrium points. Furthermore, preliminary extensive simulations indicate that the primary dynamic features observed in relatively small networks closely resemble those ensuring the performance and accuracy of large-scale networks.

Exploring the global dynamics of networks trained through equilibrium propagation / Zoppo, Gianluca; Corinto, Fernando; Gilli, Marco.. - ELETTRONICO. - (2024). (Intervento presentato al convegno 2024 IEEE International Symposium on Circuits and Systems, ISCAS 2024 tenutosi a Singapore nel 19-22/05/2024) [10.1109/ISCAS58744.2024.10557843].

Exploring the global dynamics of networks trained through equilibrium propagation.

Zoppo, Gianluca;Corinto, Fernando;Gilli, Marco.
2024

Abstract

Equilibrium propagation is a learning technique conceived for training continuous-time recurrent neural networks. It offers some notable advantages when compared to conventional back-propagation-based algorithms and to classical design methods. From an implementation perspective, it demands only a single computational circuit. Theoretically, although it seeks to minimize a cost function, it exhibits similarities to spike- timing-dependent plasticity (STDP), rendering it, to a certain extent, biologically plausible. This paper explores the global dynamic behavior of continuous-time piecewise linear networks trained through equilibrium point propagation. We examine a network in which the target patterns are presented as external inputs rather than as initial conditions. We first show that the learning rules, which extend equilibrium propagation to gradient- like and non-symmetric networks, can be derived as a suitable approximation of Lagrangian optimization. Then, by studying a relatively simple but thoroughly significant case, we demonstrate that a detailed analysis of the equilibrium point distribution yields a profound understanding of the network’s fundamental proper- ties and provides a valuable tool for quantitatively evaluating the network’s accuracy. Compared to classical synthesis techniques, our approach, where patterns are introduced as external inputs, in most cases, circumvents the impractical task of estimating the basins of attraction for sets of multiple equilibrium points. Furthermore, preliminary extensive simulations indicate that the primary dynamic features observed in relatively small networks closely resemble those ensuring the performance and accuracy of large-scale networks.
2024
979-8-3503-3099-1
File in questo prodotto:
File Dimensione Formato  
GILLI_ISCAS_2024.pdf

accesso aperto

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Pubblico - Tutti i diritti riservati
Dimensione 200.64 kB
Formato Adobe PDF
200.64 kB Adobe PDF Visualizza/Apri
Exploring_the_Global_Dynamics_of_Networks_Trained_through_Equilibrium_Propagation.pdf

accesso riservato

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 816.42 kB
Formato Adobe PDF
816.42 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2996257