We consider the generalization problem for a perceptron with binary synapses, implementing the Stochastic Belief-Propagation-Inspired (SBPI) learning algorithm which we proposed earlier, and perform a mean-field calculation to obtain a differential equation which describes the behaviour of the device in the limit of a large number of synapses N. We show that the solving time of SBPI is of order N√(logN) , while the similar, well-known clipped perceptron (CP) algorithm does not converge to a solution at all in the time frame we considered. The analysis gives some insight into the ongoing process and shows that, in this context, the SBPI algorithm is equivalent to a new, simpler algorithm, which only differs from the CP algorithm by the addition of a stochastic, unsupervised meta-plastic reinforcement process, whose rate of application must be less than 2/√(πN) for the learning to be achieved effectively. The analytical results are confirmed by simulations.
Generalization Learning in a Perceptron with Binary Synapses / Carlo Baldassi. - In: JOURNAL OF STATISTICAL PHYSICS. - ISSN 0022-4715. - ELETTRONICO. - 136(2009), pp. 902-916. [10.1007/s10955-009-9822-1]
|Titolo:||Generalization Learning in a Perceptron with Binary Synapses|
|Data di pubblicazione:||2009|
|Digital Object Identifier (DOI):||http://dx.doi.org/10.1007/s10955-009-9822-1|
|Appare nelle tipologie:||1.1 Articolo in rivista|
File in questo prodotto: