PubMed · 10196580
The source did not provide an abstract. Follow the original record for more information.
Explore connections, maps & timelines
D Whitney, I Murakami. 1998. Latency difference, not spatial extrapolation.. https://doi.org/10.1038/3659
Cite the original work for its findings. Save a collection to share your selection of sources.
KEEP EXPLORING
Irregularly timed action potentials, or spikes, are pervasively observed in the brain activity of awake mammals. However, the role of this temporal irregularity in neural computation is still not well understood. In canonical network models irregular spiking emerges via balanced, fluctuating input currents, leading to collective responses that track inputs linearly. How networks characterized by irregular spiking could support flexible nonlinear dynamics needed for general-purpose computation remains under ongoing debate. Here we characterize the dynamics of networks whose elementary unit is not a single neuron but a small group of neurons, with distinct tunings, that compete at each timestep via a winner-take-all (WTA) interaction. While WTA has long been proposed as an elementary functional motif in the brain and represents a powerful computational primitive, how large networks of such units behave has received less investigation. We show that these networks, like classic excitatory-inhibitory balanced networks, exhibit a chaotic fluctuation-driven regime characterized by sustained irregular activity resembling realistic cortical spiking, which we interpret as a multidimensional balance spread over several competing neural populations with different tunings. We develop a mean-field theory for the network, which shows how irregular spiking sustained by time-varying input fluctuations can support flexible nonlinear collective dynamics. Using the theory we predict and verify network regimes in which input fluctuations alone yield multistability, stable sequence generation, or complex heterogeneous firing rate dynamics-three core dynamical primitives thought to underlie memory-dependent neural computation-via consistent Poisson-like spiking produced through chaos. Thus, networks of WTA units support a chaotic fluctuation-driven regime characterized by irregular spiking that can power complex nonlinear collective dynamics. This represents a new model of brain activity capable of simultaneously reproducing realistic spike trains and diverse nonlinear firing rate patterns well posed for flexible computation, and which can be trained or fit to data.
Local minima and plateaus pose a serious problem in learning of neural networks. We investigate the hierarchical geometric structure of the parameter space of three-layer perceptrons in order to show the existence of local minima and plateaus. It is proved that a critical point of the model with H - 1 hidden units always gives many critical points of the model with H hidden units. These critical points consist of many lines in the parameter space, which can cause plateaus in learning of neural networks. Based on this result, we prove that a point in the critical lines corresponding to the global minimum of the smaller model can be a local minimum or a saddle point of the larger model. We give a necessary and sufficient condition for this, and show that this kind of local minima exist as a line segment if any. The results are universal in the sense that they do not require special properties of the target, loss functions and activation functions, but only use the hierarchical structure of the model.
A neural network model is proposed for computation of the convex-hull of a finite planar set. The model is self-organizing in that it adapts itself to the hull-vertices of the convex-hull in an orderly fashion without any supervision. The proposed network consists of three layers of processors. The bottom layer computes the activation functions, the outputs of which are passed onto the middle layer. The middle layer is used for winner selection. These information are passed onto the topmost layer as well as fed back to the bottom layer. The network in the topmost layer self-organizes by labeling the hull-processors in an orderly fashion so that the final convex-hull is obtained from the topmost layer. Time complexities of the proposed model are analyzed and are compared with existing models of similar nature.