Search PubMed⌕ Search

Biomedical subjects

Peter Dayan

Publications and source records attributed to Peter Dayan.

At least 37 records · Page 2Linked to original sources
PubMed2005

Dopamine, learning, and impulsivity: a biological account of attention-deficit/hyperactivity disorder.

BACKGROUND: Attention-deficit/hyperactivity disorder (ADHD) affects up to 10% of school-age children. The impulsivity which is seen as its core feature persists over years, yet experimental measures of impulsivity can be altered in a single session. In this study, we tested the theory that both the persistence and the variability of impulsivity could be the result of abnormalities in learning mechanisms and environment. METHOD: We extended an existing model of the role of dopamine in operant conditioning to address the delayed response time task, which is one of the standard tests for impulsivity in ADHD. In this task, subjects choose between immediate responding for a small reinforcer and later responding for a larger one. We studied the influence on impulsivity of four key parameters of the model: The learning rate, discount factor, brittleness, and action bias. RESULTS: The behavior of the model is broadly comparable with electrophysiological (monkey) and behavioral (ADHD and normal) data. Variations in any of the parameters can cause impulsivity. All parameters except the discount factor show inverted U-shaped curves for their effects on impulsivity, suggesting, for example, how either hyper- or hypofunctioning of dopamine can cause impulsivity. The model suggests how decision making can be affected by environmental unpredictability, and thus offers an account of one aspect of the natural history of ADHD. CONCLUSIONS: Some types of ADHD may be caused by specific deficits in reinforcement learning and in the use of learned lessons. Environmental factors can interact with these deficits to delay maturation.

Attention Deficit Disorder with Hyperactivity↗
PubMed2005

Controversies in rapid sequence intubation in children.

PURPOSE OF REVIEW: Rapid sequence intubation is the method of choice for intubation of the emergency department patient. The purpose of the present review is to address several controversies pertaining to emergency department rapid sequence intubation of children. RECENT FINDINGS: The topics covered in this review include the determination of the appropriate clinician to perform emergency department intubation, the use of atropine and lidocaine as premedications, the choice of sedative agents depending upon the clinical scenario, and the choice of neuromuscular blockade agent. Concerning these topics: The literature supports that emergency department physicians, with appropriate training, successfully perform intubation in most patients. Limited data exist to determine the appropriate use of atropine and lidocaine for rapid sequence intubation. Etomidate has clearly become a preferred sedative for rapid sequence intubation with a low risk of cardiovascular side effects. Thiopental and propofol may more readily provide adequate sedation as compared with etomidate but both have the potential to reduce blood pressure. Succinylcholine arguably remains the preferred neuromuscular blockade agent for rapid sequence intubation in most children. The side effects of succinylcholine occur in relatively predictable circumstances. Rocuronium is a commonly used nondepolarizing paralytic agent but its prolonged duration of action must be weighed against the risk of side effects associated with succinylcholine. SUMMARY: Though more research is needed, the available data allow for the development of protocols that will result in a rational, scenario-based approach to rapid sequence intubation in children.

Androstanols↗
PubMed2004

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Conditioning, Classical↗
PubMed2004

Dissociable roles of ventral and dorsal striatum in instrumental conditioning.

Instrumental conditioning studies how animals and humans choose actions appropriate to the affective structure of an environment. According to recent reinforcement learning models, two distinct components are involved: a "critic," which learns to predict future reward, and an "actor," which maintains information about the rewarding outcomes of actions to enable better ones to be chosen more frequently. We scanned human participants with functional magnetic resonance imaging while they engaged in instrumental conditioning. Our results suggest partly dissociable contributions of the ventral and dorsal striatum, with the former corresponding to the critic and the latter corresponding to the actor.

Adult↗
PubMed2004

Off-line replay maintains declarative memories in a model of hippocampal-neocortical interactions.

During sleep, neural activity in the hippocampus and neocortex seems to recapitulate aspects of its earlier, awake form. This replay may be a substrate for the consolidation of long-term declarative memories, whereby they become independent of the hippocampus and are stored in neocortex. In contrast to storage, other crucial facets of competent long-term memory, such as maintenance of access to stored traces and preservation of their correct interpretation, have received little attention. We investigate long-term episodic and semantic memory in a theoretical model of neocortical-hippocampal interaction. We find that, in the absence of regular hippocampal reactivation, even supposedly consolidated episodic memories are fragile in the face of cortical semantic plasticity. Replay allows access to episodes stored in the hippocampus to be maintained, by keeping them in appropriate register with changing neocortical representations. Hippocampal storage and replay also has a constructive role in the recall of structured, semantic information.

Brain Injuries↗
PubMed2003

Temporal difference models and reward-related learning in the human brain.

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.

Adolescent↗
PubMed2003

Inference and computation with population codes.

In the vertebrate nervous system, sensory stimuli are typically encoded through the concerted activity of large populations of neurons. Classically, these patterns of activity have been treated as encoding the value of the stimulus (e.g., the orientation of a contour), and computation has been formalized in terms of function approximation. More recently, there have been several suggestions that neural computation is akin to a Bayesian inference process, with population activity patterns representing uncertainty about stimuli in the form of probability distributions (e.g., the probability density function over the orientation of a contour). This paper reviews both approaches, with a particular emphasis on the latter, which we see as a very promising framework for future modeling and experimental work.

Animals↗
PubMed2003

Doubly distributional population codes: simultaneous representation of uncertainty and multiplicity.

Perceptual inference fundamentally involves uncertainty, arising from noise in sensation and the ill-posed nature of many perceptual problems. Accurate perception requires that this uncertainty be correctly represented, manipulated, and learned about. The choices subjects make in various psychophysical experiments suggest that they do indeed take such uncertainty into account when making perceptual inferences, posing the question as to how uncertainty is represented in the activities of neuronal populations. Most theoretical investigations of population coding have ignored this issue altogether; the few existing proposals that address it do so in such a way that it is fatally conflated with another facet of perceptual problems that also needs correct handling: multiplicity (that is, the simultaneous presence of multiple distinct stimuli). We present and validate a more powerful proposal for the way that population activity may encode uncertainty, both distinctly from and simultaneously with multiplicity.

Action Potentials↗
PubMed2003

Nonlinear ideal observation and recurrent preprocessing in perceptual learning.

Residual micro-saccades, tremor and fixation errors imply that, on different trials in visual tasks, stimulus arrays are inevitably presented at different positions on the retina. Positional variation is likely to be specially important for tasks involving visual hyperacuity, because of the severe demands that these tasks impose on spatial resolution. In this paper, we show that small positional variations lead to a structural change in the nature of the ideal observer's solution to a hyperacuity-like visual discrimination task such that the optimal discriminator depends quadratically rather than linearly on noisy neural activities. Motivated by recurrent models of early visual processing, we show how a recurrent preprocessor of the noisy activities can produce outputs which, when passed through a linear discriminator, lead to better discrimination even when the positional variations are much larger than the threshold acuity of the task. Since, psychophysically, hyperacuity typically improves greatly over the course of perceptual learning, we discuss our model in the light of results on the speed and nature of learning.

Animals↗
PubMed2002

Reward, motivation, and reinforcement learning.

There is substantial evidence that dopamine is involved in reward learning and appetitive conditioning. However, the major reinforcement learning-based theoretical models of classical conditioning (crudely, prediction learning) are actually based on rules designed to explain instrumental conditioning (action learning). Extensive anatomical, pharmacological, and psychological data, particularly concerning the impact of motivational manipulations, show that these models are unreasonable. We review the data and consider the involvement of a rich collection of different neural systems in various aspects of these forms of conditioning. Dopamine plays a pivotal, but complicated, role.

Animals↗
PubMed2002

Matters temporal.

Current evidence suggests that neural Hebbian learning in cortical and hippocampal synapses is fundamentally predictive rather than conventionally correlational. Much attention is focussing on what sort of predictions are acquired, and in what neural architectures. A recent paper by Rao and Sejnowski has suggested an interesting interpretation in terms of a popular predictive algorithm that has roots in psychology, computer science and engineering.

Journal Article↗
PubMed2002

Acquisition and extinction in autoshaping.

C. R. Gallistel and J. Gibbon (2000) presented quantitative data on the speed with which animals acquire behavioral responses during autoshaping, together with a statistical model of learning intended to account for them. Although this model captures the form of the dependencies among critical variables, its detailed predictions are substantially at variance with the data. In the present article, further key data on the speed of acquisition are used to motivate an alternative model of learning, in which animals can be interpreted as paying different amounts of attention to stimuli according to estimates of their differential reliabilities as predictors.

Animals↗
PubMed1996

Varieties of Helmholtz Machine.

The Helmholtz machine is a new unsupervised learning architecture that uses top-down connections to build probability density models of input and bottom-up connections to build inverses to those models. The wake-sleep learning algorithm for the machine involves just the purely local delta rule. This paper suggests a number of different varieties of Helmholtz machines, each with its own strengths and weaknesses, and relates them to cortical information processing. Copyright 1996 Elsevier Science Ltd.

Journal Article↗
PubMedDate not supplied

Pattern formation and cortical maps.

The response selectivities of neurons in adult primary sensory cortices depend on intricate patterns of synaptic connections; these selectivities are arranged over cortex in equally rich fashion. Characterising these patterns, and particularly the activity-dependence (and independence) of their developmental trajectories, has been a major task for experimental and theoretical neuroscience. Here, we describe and analyse a paradigmatic algorithm for activity-dependent development of the refinement and generation of neuronal selectivities, and relate it to some of the wealth of suggestions in the literature.

Animals↗
PubMedDate not supplied

Dopamine: generalization and bonuses.

In the temporal difference model of primate dopamine neurons, their phasic activity reports a prediction error for future reward. This model is supported by a wealth of experimental data. However, in certain circumstances, the activity of the dopamine cells seems anomalous under the model, as they respond in particular ways to stimuli that are not obviously related to predictions of reward. In this paper, we address two important sets of anomalies, those having to do with generalization and novelty. Generalization responses are treated as the natural consequence of partial information; novelty responses are treated by the suggestion that dopamine cells multiplex information about reward bonuses, including exploration bonuses and shaping bonuses. We interpret this additional role for dopamine in terms of the mechanistic attentional and psychomotor effects of dopamine, having the computational role of guiding exploration.

Animals↗
PubMedDate not supplied

Opponent interactions between serotonin and dopamine.

Anatomical and pharmacological evidence suggests that the dorsal raphe serotonin system and the ventral tegmental and substantia nigra dopamine system may act as mutual opponents. In the light of the temporal difference model of the involvement of the dopamine system in reward learning, we consider three aspects of motivational opponency involving dopamine and serotonin. We suggest that a tonic serotonergic signal reports the long-run average reward rate as part of an average-case reinforcement learning model; that a tonic dopaminergic signal reports the long-run average punishment rate in a similar context; and finally speculate that a phasic serotonin signal might report an ongoing prediction error for future punishment.

Animals↗
PubMedDate not supplied

Acetylcholine in cortical inference.

Acetylcholine (ACh) plays an important role in a wide variety of cognitive tasks, such as perception, selective attention, associative learning, and memory. Extensive experimental and theoretical work in tasks involving learning and memory has suggested that ACh reports on unfamiliarity and controls plasticity and effective network connectivity. Based on these computational and implementational insights, we develop a theory of cholinergic modulation in perceptual inference. We propose that ACh levels reflect the uncertainty associated with top-down information, and have the effect of modulating the interaction between top-down and bottom-up processing in determining the appropriate neural representations for inputs. We illustrate our proposal by means of an hierarchical hidden Markov model, showing that cholinergic modulation of contextual information leads to appropriate perceptual inference.

Acetylcholine↗