Regularising neural networks using flexible multivariate activation function.

Mirko Solazzi; Aurelio Uncini

doi:10.1016/s0893-6080(03)00189-8

Home/Archive/PubMed

PubMed · 15036342

Regularising neural networks using flexible multivariate activation function.

Mirko Solazzi·Aurelio Uncini

View original source ↗Export metadata ↓

Abstract

This paper presents a new general neural structure based on nonlinear flexible multivariate function that can be viewed in the framework of the generalised regularisation networks theory. The proposed architecture is based on multi-dimensional adaptive cubic spline basis activation function that collects information from the previous network layer in aggregate form. In other words, each activation function represents a spline function of a subset of previous layer outputs so the number of network connections (structural complexity) can be very low with respect to the problem complexity. A specific learning algorithm, based on the adaptation of local parameters of the activation function, is derived. This fact improve the network generalisation capabilities and speed up the convergence of the learning process. At last, some experimental results demonstrating the effectiveness of the proposed architecture, are presented.

Explore related subjects

Learning Multivariate Analysis Neural Networks, Computer Journal Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Mirko Solazzi, Aurelio Uncini. 2004. Regularising neural networks using flexible multivariate activation function.. https://doi.org/10.1016/s0893-6080(03)00189-8

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2004

On-line learning through simple perceptron learning with a margin.

We analyze a learning method that uses a margin kappa a la Gardner for simple perceptron learning. This method corresponds to the perceptron learning when kappa = 0 and to the Hebbian learning when kappa = infinity. Nevertheless, we found that the generalization ability of the method was superior to that of the perceptron and the Hebbian methods at an early stage of learning. We analyzed the asymptotic property of the learning curve of this method through computer simulation and found that it was the same as for perceptron learning. We also investigated an adaptive margin control method.

Learning↗

PubMed2004

Bifurcating neuron: computation and learning.

The ability of bifurcating processing units and their networks to rapidly switch between different dynamic modes has been used in recent research efforts to model new computational properties of neural systems. In this spirit, we devise a bifurcating neuron based on control of chaos collapsing to a period-3 orbit in the dynamics of a quadratic logistic map (QLM). Proposed QLM3 neuron is constructed with the third iterate of QLM and uses an external input, which governs its dynamics. The input shifts the neuron's dynamics from chaos to one of the stable fixed points. This way the inputs from certain ranges (clusters) are mapped to stable fixed points, while the rest of the inputs is mapped to chaotic or periodic output dynamics. It has been shown that QLM3 neuron is able to learn a specific mapping by adaptively adjusting its bifurcation parameter, the idea of which is based on the principles of parametric control of logistic maps [Proceedings of the International Symposium on Nonlinear Theory and its Applications (NOLTA'97), Honolulu, HI, 1997; Proceedings of SPIE, 2000]. Learning algorithm for the bifurcation parameter is proposed, which employs the error gradient descent method.

Learning↗

PubMed2004

A hierarchical self-organizing approach for learning the patterns of motion trajectories.

The understanding and description of object behaviors is a hot topic in computer vision. Trajectory analysis is one of the basic problems in behavior understanding, and the learning of trajectory patterns that can be used to detect anomalies and predict object trajectories is an interesting and important problem in trajectory analysis. In this paper, we present a hierarchical self-organizing neural network model and its application to the learning of trajectory distribution patterns for event recognition. The distribution patterns of trajectories are learnt using a hierarchical self-organizing neural network. Using the learned patterns, we consider anomaly detection as well as object behavior prediction. Compared with the existing neural network structures that are used to learn patterns of trajectories, our network structure has smaller scale and faster learning speed, and is thus more effective. Experimental results using two different sets of data demonstrate the accuracy and speed of our hierarchical self-organizing neural network in learning the distribution patterns of object trajectories.

Learning↗