PubMed · 10324448
The source did not provide an abstract. Follow the original record for more information.
Explore connections, maps & timelines
R Gourlay. 1986-11-27. Learning in action.. https://pubmed.ncbi.nlm.nih.gov/10324448/
Cite the original work for its findings. Save a collection to share your selection of sources.
KEEP EXPLORING
The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).
We analyze online gradient descent learning from finite training sets at noninfinitesimal learning rates eta. Exact results are obtained for the time-dependent generalization error of a simple model system: a linear network with a large number of weights N, trained on p = alphaN examples. This allows us to study in detail the effects of finite training set size alpha on, for example, the optimal choice of learning rate eta. We also compare online and offline learning, for respective optimal settings of eta at given final learning time. Online learning turns out to be much more robust to input bias and actually outperforms offline learning when such bias is present; for unbiased inputs, online and offline learning perform almost equally well.
The basis for this neuronal model is that the properties of excitable membranes are controlled by biochemical reactions occurring in the nerve cells. The kinetics of these supposed chemical reactions is described by a set of first order differential equations. We considered the effect of regulation of the properties of sodium channels. This allows the simulation of the changes in the neuron's electrical activity parameters occurring during learning, associated with its excitability. The neuronal model exhibits different excitability after the learning procedure relative to the different input signals that corresponds to the experimental data.