Equilibrium propagation is a learning technique conceived for training continuous-time recurrent neural networks. It offers some notable advantages when compared to conventional back-propagation-based algorithms and to classical design methods. From an implementation perspective, it demands only a single computational circuit. Theoretically, although it seeks to minimize a cost function, it exhibits similarities to spike- timing-dependent plasticity (STDP), rendering it, to a certain extent, biologically plausible. This paper explores the global dynamic behavior of continuous-time piecewise linear networks trained through equilibrium point propagation. We examine a network in which the target patterns are presented as external inputs rather than as initial conditions. We first show that the learning rules, which extend equilibrium propagation to gradient- like and non-symmetric networks, can be derived as a suitable approximation of Lagrangian optimization. Then, by studying a relatively simple but thoroughly significant case, we demonstrate that a detailed analysis of the equilibrium point distribution yields a profound understanding of the network’s fundamental proper- ties and provides a valuable tool for quantitatively evaluating the network’s accuracy. Compared to classical synthesis techniques, our approach, where patterns are introduced as external inputs, in most cases, circumvents the impractical task of estimating the basins of attraction for sets of multiple equilibrium points. Furthermore, preliminary extensive simulations indicate that the primary dynamic features observed in relatively small networks closely resemble those ensuring the performance and accuracy of large-scale networks.
Exploring the global dynamics of networks trained through equilibrium propagation / Zoppo, Gianluca; Corinto, Fernando; Gilli, Marco.. - ELETTRONICO. - (2024). (Intervento presentato al convegno 2024 IEEE International Symposium on Circuits and Systems, ISCAS 2024 tenutosi a Singapore nel 19-22/05/2024) [10.1109/ISCAS58744.2024.10557843].
Exploring the global dynamics of networks trained through equilibrium propagation.
Zoppo, Gianluca;Corinto, Fernando;Gilli, Marco.
2024
Abstract
Equilibrium propagation is a learning technique conceived for training continuous-time recurrent neural networks. It offers some notable advantages when compared to conventional back-propagation-based algorithms and to classical design methods. From an implementation perspective, it demands only a single computational circuit. Theoretically, although it seeks to minimize a cost function, it exhibits similarities to spike- timing-dependent plasticity (STDP), rendering it, to a certain extent, biologically plausible. This paper explores the global dynamic behavior of continuous-time piecewise linear networks trained through equilibrium point propagation. We examine a network in which the target patterns are presented as external inputs rather than as initial conditions. We first show that the learning rules, which extend equilibrium propagation to gradient- like and non-symmetric networks, can be derived as a suitable approximation of Lagrangian optimization. Then, by studying a relatively simple but thoroughly significant case, we demonstrate that a detailed analysis of the equilibrium point distribution yields a profound understanding of the network’s fundamental proper- ties and provides a valuable tool for quantitatively evaluating the network’s accuracy. Compared to classical synthesis techniques, our approach, where patterns are introduced as external inputs, in most cases, circumvents the impractical task of estimating the basins of attraction for sets of multiple equilibrium points. Furthermore, preliminary extensive simulations indicate that the primary dynamic features observed in relatively small networks closely resemble those ensuring the performance and accuracy of large-scale networks.File | Dimensione | Formato | |
---|---|---|---|
GILLI_ISCAS_2024.pdf
accesso aperto
Tipologia:
2. Post-print / Author's Accepted Manuscript
Licenza:
Pubblico - Tutti i diritti riservati
Dimensione
200.64 kB
Formato
Adobe PDF
|
200.64 kB | Adobe PDF | Visualizza/Apri |
Exploring_the_Global_Dynamics_of_Networks_Trained_through_Equilibrium_Propagation.pdf
accesso riservato
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
816.42 kB
Formato
Adobe PDF
|
816.42 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2996257