Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “recurrent neural network”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources
PubMed2002

Dynamic stability conditions for Lotka-Volterra recurrent neural networks with delays.

The Lotka-Volterra model of neural networks, derived from the membrane dynamics of competing neurons, have found successful applications in many "winner-take-all" types of problems. This paper studies the dynamic stability properties of general Lotka-Volterra recurrent neural networks with delays. Conditions for nondivergence of the neural networks are derived. These conditions are based on local inhibition of networks, thereby allowing these networks to possess a multistability property. Multistability is a necessary property of a network that will enable important neural computations such as those governing the decision making process. Under these nondivergence conditions, a compact set that globally attracts all the trajectories of a network can be computed explicitly. If the connection weight matrix of a network is symmetric in some sense, and the delays of the network are in L2 space, we can prove that the network will have the property of complete stability.

Journal Article↗
PubMed2006

Recurrent neural network as a linear attractor for pattern association.

We propose a linear attractor network based on the observation that similar patterns form a pipeline in the state space, which can be used for pattern association. To model the pipeline in the state space, we present a learning algorithm using a recurrent neural network. A least-squares estimation approach utilizing the interdependency between neurons defines the dynamics of the network. The region of convergence around the line of attraction is defined based on the statistical characteristics of the input patterns. Performance of the learning algorithm is evaluated by conducting several experiments in benchmark problems, and it is observed that the new technique is suitable for multiple-valued pattern association.

Letter↗
PubMed2005

Prediction of debacle points for robustness of biological pathways by using recurrent neural networks.

Living organisms have ingenious control mechanisms in which many molecular interactions work for keeping their normal activities against disturbances inside and outside of them. However, at the same time, the control mechanism has debacle points at which the stability can be broken easily. This paper proposes a new method which uses recurrent neural network for predicting debacle points in an hybrid functional Petri net model of a biological pathway. Evaluation on an apoptosis signaling pathway indicates that the rates of 96.5 % of debacle points and 65.5 % of non-debacle points can be predicted by the proposed method.

Apoptosis↗
PubMed2007

Modelling memory functions with recurrent neural networks consisting of input compensation units: I. Static situations.

Humans are able to form internal representations of the information they process -- a capability which enables them to perform many different memory tasks. Therefore, the neural system has to learn somehow to represent aspects of the environmental situation; this process is assumed to be based on synaptic changes. The situations to be represented are various as for example different types of static patterns but also dynamic scenes. How are neural networks consisting of mutually connected neurons capable of performing such tasks? Here we propose a new neuronal structure for artificial neurons. This structure allows one to disentangle the dynamics of the recurrent connectivity from the dynamics induced by synaptic changes due to the learning processes. The error signal is computed locally within the individual neuron. Thus, online learning is possible without any additional structures. Recurrent neural networks equipped with these computational units cope with different memory tasks. Examples illustrate how information is extracted from environmental situations comprising fixed patterns to produce sustained activity and to deal with simple algebraic relations.

Humans↗
PubMed2006

Recognition of the physiological actions of the triphasic EMG pattern by a dynamic recurrent neural network.

Triphasic electromyographic (EMG) patterns with a sequence of activity in agonist (AG1), antagonist (ANT) and again in agonist (AG2) muscles are characteristic of ballistic movements. They have been studied in terms of rectangular pulse-width or pulse-height modulation. In order to take into account the complexity of the EMG signal within the bursts, we used a dynamic recurrent neural network (DRNN) for the identification of this pattern in subjects performing fast elbow flexion movements. Biceps and triceps EMGs were fed to all 35 fully-connected hidden units of the DRNN for mapping onto elbow angular acceleration signals. DRNN training was supervised, involving learning rule adaptations of synaptic weights and time constants of each unit. We demonstrated that the DRNN is able to perfectly reproduce the acceleration profile of the ballistic movements. Then we tested the physiological plausibility of all the networks that reached an error level below 0.001 by selectively increasing the amplitude of each burst of the triphasic pattern and evaluating the effects on the simulated accelerating profile. Nineteen percent of these simulations reproduced the physiological action classically attributed to the 3 EMG bursts: AG1 increase showed an increase of the first accelerating pulse, ANT an increase of the braking pulse and AG2 an increase of the clamping pulse. These networks also recognized the physiological function of the time interval between AG1 and ANT, reproducing the linear relationship between time interval and movement amplitude. This task-dynamics recognition has implications for the development of DRNN as diagnostic tools and prosthetic controllers.

Action Potentials↗
PubMed2002

Prediction of contact maps by GIOHMMs and recurrent neural networks using lateral propagation from all four cardinal corners.

MOTIVATION: Accurate prediction of protein contact maps is an important step in computational structural proteomics. Because contact maps provide a translation and rotation invariant topological representation of a protein, they can be used as a fundamental intermediary step in protein structure prediction. RESULTS: We develop a new set of flexible machine learning architectures for the prediction of contact maps, as well as other information processing and pattern recognition tasks. The architectures can be viewed as recurrent neural network implemantations of a class of Bayesian networks we call generalized input-output HMMs (GIOHMMs). For the specific case of contact maps, contextual information is propagated laterally through four hidden planes, one for each cardinal corner. We show that these architectures can be trained from examples and yield contact map predictors that outperform previously reported methods. While several extensions and improvements are in progress, the current version can accurately predict 60.5% of contacts at a distance cutoff of 8 A and 45% of distant contacts at 10 A, for proteins of length up to 300.

Algorithms↗
PubMed2004

Constrained motion control of flexible robot manipulators based on recurrent neural networks.

In this paper, a neural network approach is presented for the motion control of constrained flexible manipulators, where both the contact force everted by the flexible manipulator and the position of the end-effector contacting with a surface are controlled. The dynamic equations for vibration of flexible link and constrained force are derived. The developed control, scheme can adaptively estimate the underlying dynamics of the manipulator using recurrent neural networks (RNNs). Based on the error dynamics of a feedback controller, a learning rule for updating the connection weights of the adaptive RNN model is obtained. Local stability properties of the control system are discussed. Simulation results are elaborated on for both position and force trajectory tracking tasks in the presence of varying parameters and unknown dynamics, which show that the designed controller performs remarkably well.

Algorithms↗
PubMed2006

Parameter space structure of continuous-time recurrent neural networks.

A fundamental challenge for any general theory of neural circuits is how to characterize the structure of the space of all possible circuits over a given model neuron. As a first step in this direction, this letter begins a systematic study of the global parameter space structure of continuous-time recurrent neural networks (CTRNNs), a class of neural models that is simple but dynamically universal. First, we explicitly compute the local bifurcation manifolds of CTRNNs. We then visualize the structure of these manifolds in net input space for small circuits. These visualizations reveal a set of extremal saddle node bifurcation manifolds that divide CTRNN parameter space into regions of dynamics with different effective dimensionality. Next, we completely characterize the combinatorics and geometry of an asymptotically exact approximation to these regions for circuits of arbitrary size. Finally, we show how these regions can be used to calculate estimates of the probability of encountering different kinds of dynamics in CTRNN parameter space.

Animals↗
PubMed2006

The chaotic nature of temper in humans: a long short-term memory recurrent neural network model.

In mathematics and physics, chaos theory deals with the behavior of certain nonlinear dynamical systems that under certain conditions exhibit a phenomenon known as chaos, which is characterised by a sensitivity to initial conditions. Mathematicians paradoxically call such states of order chaos and distinguish them from randomness. New models for describing and predicting different aspects of behavior are being created which once seemed unpredictable. This is done by focusing on the overall patterns of behavior, showing how stable or unstable they are and identifying the circumstances that make them change. In this paper, we indicate why human temper and mood changes have a chaotic nature. Then, we develop a chaotic model based on a long short-term memory recurrent neural network with irregular embeddings derived by the gamma test to model temper tantrum. We finally use a feedback delay controller to stabilize its chaotic behavior, because it is a plausible method for stabilizing biological neural systems. A lot of aspects of this model are analogous to the human counterpart. The model might suggest, for example, that if a particular person had stronger ego defenses, and attended a little less vigilantly to the external world, he or she might find a stable attractor amidst of a broader landscape of chaotic attractors. A therapist or self-control would be analogous to the delayed feedback controller, by specifically encouraging those changes, he might help the person reach a stable behavior. Finally, some comments are proposed to facilitate the normal behavior.

Computer Simulation↗
PubMed2002

Application of recurrent neural network to predict bacterial growth in dynamic conditions.

A combination of a factorial design and two central composite designs was used to assess quantitatively the effects of acid pH (5.6-7.0) or alkaline pH (7.0-9.5) and NaCl (0-8%) variations on the growth of Listeria monocytogenes in a meat broth, at 20 degrees C and lower temperature 10 degrees C. Two principal phenomena were observed when bacteria were submitted to abrupt change of pH and a(w) during growth, whatever the growth temperature: (i) large environmental variations induced a lag phase following the fluctuation, and (ii) the growth continued with a generation time value different from that observed before the change or that associated to the new environment. A dynamic model, based on recurrent neural network (RNN), was developed to describe the growth of L. monocytogenes as a function of temperature and fluctuating conditions of acid pH, alkaline pH and concentration of NaCl. The results showed that the neural network model can be used to represent the complex effects of environmental variable conditions on the microorganism behaviour.

Hydrogen-Ion Concentration↗
PubMed1998

Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network.

We study unsupervised Hebbian learning in a recurrent network in which synapses have a finite number of stable states. Stimuli received by the network are drawn at random at each presentation from a set of classes. Each class is defined as a cluster in stimulus space, centred on the class prototype. The presentation protocol is chosen to mimic the protocols of visual memory experiments in which a set of stimuli is presented repeatedly in a random way. The statistics of the input stream may be stationary, or changing. Each stimulus induces, in a stochastic way, transitions between stable synaptic states. Learning dynamics is studied analytically in the slow learning limit, in which a given stimulus has to be presented many times before it is memorized, i.e. before synaptic modifications enable a pattern of activity correlated with the stimulus to become an attractor of the recurrent network. We show that in this limit the synaptic matrix becomes more correlated with the class prototypes than with any of the instances of the class. We also show that the number of classes that can be learned increases sharply when the coding level decreases, and determine the speeds of learning and forgetting of classes in the case of changes in the statistics of the input stream.

Generalization, Stimulus↗
PubMed1998

A recurrent neural network for modelling dynamical systems.

We introduce a recurrent network architecture for modelling a general class of dynamical systems. The network is intended for modelling real-world processes in which empirical measurements of the external and state variables are obtained at discrete time points. The model can learn from multiple temporal patterns, which may evolve on different timescales and be sampled at non-uniform time intervals. We demonstrate the application of the model to a synthetic problem in which target data are only provided at the final time step. Despite the sparseness of the training data, the network is able not only to make good predictions at the final time step for temporal processes unseen in training, but also to reproduce the sequence of the state variables at earlier times. Moreover, we show how the network can infer the existence and role of state variables for which no target information is provided. The ability of the model to cope with sparse data is likely to be useful in a number of applications, including, in particular, the modelling of metal forging.

Models, Theoretical↗
PubMed2001

Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions.

We establish two conditions that ensure the nondivergence of additive recurrent networks with unsaturating piecewise linear transfer functions, also called linear threshold or semilinear transfer functions. As Hahnloser, Sarpeshkar, Mahowald, Douglas, and Seung (2000) showed, networks of this type can be efficiently built in silicon and exhibit the coexistence of digital selection and analog amplification in a single circuit. To obtain this behavior, the network must be multistable and nondivergent, and our conditions allow determining the regimes where this can be achieved with maximal recurrent amplification. The first condition can be applied to nonsymmetric networks and has a simple interpretation of requiring that the strength of local inhibition match the sum over excitatory weights converging onto a neuron. The second condition is restricted to symmetric networks, but can also take into account the stabilizing effect of nonlocal inhibitory interactions. We demonstrate the application of the conditions on a simple example and the orientation-selectivity model of Ben-Yishai, Lev Bar-Or, and Sompolinsky (1995). We show that the conditions can be used to identify in their model regions of maximal orientation-selective amplification and symmetry breaking.

Animals↗
PubMedDate not supplied

Abductive reasoning with recurrent neural networks.

Abduction is the process of proceeding from data describing a set of observations or events, to a set of hypotheses which best explains or accounts for the data. Cost-based abduction (CBA) is a formalism in which evidence to be explained is treated as a goal to be proven, proofs have costs based on how much needs to be assumed to complete the proof, and the set of assumptions needed to complete the least-cost proof are taken as the best explanation for the given evidence. In previous work, we presented a method for using high order recurrent networks to find least cost proofs for CBA instances. Here, we present a method that significantly reduces the size of the neural network that is produced for a given CBA instance. We present experimental results describing the performance of this method and comparing its performance to that of the previous method.

Artificial Intelligence↗
PubMed1996

Hebbian learning of context in recurrent neural networks.

Single electrode recording in the inferotemporal cortex of monkeys during delayed visual memory tasks provide evidence for attractor dynamics in the observed region. The persistent elevated delay activities could be internal representations of features of the learned visual stimuli shown to the monkey during training. When uncorrelated stimuli are presented during training in a fixed sequence, these experiments display significant correlations between the internal representations. Recently a simple model of attractor neural network has reproduced quantitatively the measured correlations. An underlying assumption of the model is that the synaptic matrix formed during the training phase contains in its efficacies information about the contiguity of persistent stimuli in the training sequence. We present here a simple unsupervised learning dynamics that produces such a synaptic matrix if sequences of stimuli are repeatedly presented to the network at fixed order. The resulting matrix is then shown to convert temporal correlations during training into spatial correlations between attractors. The scenario is that, in the presence of selective delay activity, at the presentation of each stimulus, the activity distribution in the neural assembly contain information of both the current stimulus and the previous one (carried by the attractor). Thus the recurrent synaptic matrix can code not only for each of the stimuli presented to the network but also for their context. We combine the idea that for learning to be effective, synaptic modification should be stochastic, with the fact that attractors provide learnable information about two consecutive stimuli. We calculate explicitly the probability distribution of synaptic efficacies as a function of training protocol, that is, the order in which stimuli are presented to the network. We then solve for the dynamics of a network composed of integrate-and-fire excitatory and inhibitory neurons with a matrix of synaptic collaterals resulting from the learning dynamics. The network has a stable spontaneous activity, and stable delay activity develops after a critical learning stage. The availability of a learning dynamics makes possible a number of experimental predictions for the dependence of the delay activity distributions and the correlations between them, on the learning stage and the learning protocol. In particular it makes specific predictions for pair-associates delay experiments.

Animals↗
PubMedDate not supplied

Recurrent neural networks with backtrack-points and negative reinforcement applied to cost-based abduction.

Abduction is the process of proceeding from data describing a set of observations or events, to a set of hypotheses which best explains or accounts for the data. Cost-based abduction (CBA) is an AI formalism in which evidence to be explained is treated as a goal to be proven, proofs have costs based on how much needs to be assumed to complete the proof, and the set of assumptions needed to complete the least-cost proof are taken as the best explanation for the given evidence. In this paper, we present two techniques for improving the performance of high order recurrent networks (HORN) applied to cost-based abduction. In the backtrack-points technique, we use heuristics to recognize early that the network trajectory is moving in the wrong direction; we then restore the network state to a previously stored point, and apply heuristic perturbations to nudge the network trajectory in a different direction. In the negative reinforcement technique, we add hyperedges to the network to reduce the attractiveness of local minima. We apply these techniques to a suite of six large CBA instances, systematically generated to be difficult.

Algorithms↗
PubMed2007

Modelling memory functions with recurrent neural networks consisting of input compensation units: II. Dynamic situations.

Modelling the cognitive abilities of humans or animals or building agents that are supposed to behave cognitively requires modelling a memory system that is able to store and retrieve various contents. The content to be stored is assumed to comprise information about more or less invariant environmental objects as well as information about movements. A combination of information about both objects and movements may be called a situation model. Here we focus, in part, on models storing dynamic patterns. In particular, two abilities of humans in representing dynamical systems receive special focus: the capability of representing the acceleration of objects, as can be found in the movement of a pendulum or freely falling objects, and the capability of representing actions of transfer, i.e. motion from one point to another, have been modelled using recurrent networks consisting of input compensation units. In addition, possibilities of combining static and dynamic properties within a single model are studied.

Cognition↗
PubMed1997

Emergence of clusters in the hidden layer of a dynamic recurrent neural network.

The neural integrator of the oculomotor system is a privileged field for artificial neural network simulation. In this paper, we were interested in an improvement of the biologically plausible features of the Arnold-Robinson network. This improvement was done by fixing the sign of the connection weights in the network (in order to respect the biological Dale's Law). We also introduced a notion of distance in the network in the form of transmission delays between its units. These modifications necessitated the introduction of a general supervisor in order to train the network to act as a leaky integrator. When examining the lateral connection weights of the hidden layer, the distribution of the weights values was found to exhibit a conspicuous structure: the high-value weights were grouped in what we call clusters. Other zones are quite flat and characterized by low-value weights. Clusters are defined as particular groups of adjoining neurons which have strong and privileged connections with another neighborhood of neurons. The clusters of the trained network are reminiscent of the small clusters or patches that have been found experimentally in the nucleus prepositus hypoglossi, where the neural integrator is located. A study was conducted to determine the conditions of emergence of these clusters in our network: they include the fixation of the weight sign, the introduction of a distance, and a convergence of the information from the hidden layer to the motoneurons. We conclude that this spontaneous emergence of clusters in artificial neural networks; performing a temporal integration, is due to computational constraints, with a restricted space of solutions. Thus, information processing could induce the emergence of iterated patterns in biological neural networks.

Animals↗