Biomedical subjects

Peter Dayan

Publications and source records attributed to Peter Dayan.

11 recordsLinked to original sources

PubMed2003

Temporal difference models and reward-related learning in the human brain.

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.

Adolescent↗

PubMed2003

Inference and computation with population codes.

In the vertebrate nervous system, sensory stimuli are typically encoded through the concerted activity of large populations of neurons. Classically, these patterns of activity have been treated as encoding the value of the stimulus (e.g., the orientation of a contour), and computation has been formalized in terms of function approximation. More recently, there have been several suggestions that neural computation is akin to a Bayesian inference process, with population activity patterns representing uncertainty about stimuli in the form of probability distributions (e.g., the probability density function over the orientation of a contour). This paper reviews both approaches, with a particular emphasis on the latter, which we see as a very promising framework for future modeling and experimental work.

Animals↗

PubMed2003

Nonlinear ideal observation and recurrent preprocessing in perceptual learning.

Residual micro-saccades, tremor and fixation errors imply that, on different trials in visual tasks, stimulus arrays are inevitably presented at different positions on the retina. Positional variation is likely to be specially important for tasks involving visual hyperacuity, because of the severe demands that these tasks impose on spatial resolution. In this paper, we show that small positional variations lead to a structural change in the nature of the ideal observer's solution to a hyperacuity-like visual discrimination task such that the optimal discriminator depends quadratically rather than linearly on noisy neural activities. Motivated by recurrent models of early visual processing, we show how a recurrent preprocessor of the noisy activities can produce outputs which, when passed through a linear discriminator, lead to better discrimination even when the positional variations are much larger than the threshold acuity of the task. Since, psychophysically, hyperacuity typically improves greatly over the course of perceptual learning, we discuss our model in the light of results on the speed and nature of learning.

Animals↗

PubMed2002

Reward, motivation, and reinforcement learning.

There is substantial evidence that dopamine is involved in reward learning and appetitive conditioning. However, the major reinforcement learning-based theoretical models of classical conditioning (crudely, prediction learning) are actually based on rules designed to explain instrumental conditioning (action learning). Extensive anatomical, pharmacological, and psychological data, particularly concerning the impact of motivational manipulations, show that these models are unreasonable. We review the data and consider the involvement of a rich collection of different neural systems in various aspects of these forms of conditioning. Dopamine plays a pivotal, but complicated, role.

Animals↗

PubMed2002

Matters temporal.

Current evidence suggests that neural Hebbian learning in cortical and hippocampal synapses is fundamentally predictive rather than conventionally correlational. Much attention is focussing on what sort of predictions are acquired, and in what neural architectures. A recent paper by Rao and Sejnowski has suggested an interesting interpretation in terms of a popular predictive algorithm that has roots in psychology, computer science and engineering.

Journal Article↗

PubMed2002

Acquisition and extinction in autoshaping.

C. R. Gallistel and J. Gibbon (2000) presented quantitative data on the speed with which animals acquire behavioral responses during autoshaping, together with a statistical model of learning intended to account for them. Although this model captures the form of the dependencies among critical variables, its detailed predictions are substantially at variance with the data. In the present article, further key data on the speed of acquisition are used to motivate an alternative model of learning, in which animals can be interpreted as paying different amounts of attention to stimuli according to estimates of their differential reliabilities as predictors.

Animals↗

PubMed1996

Varieties of Helmholtz Machine.

The Helmholtz machine is a new unsupervised learning architecture that uses top-down connections to build probability density models of input and bottom-up connections to build inverses to those models. The wake-sleep learning algorithm for the machine involves just the purely local delta rule. This paper suggests a number of different varieties of Helmholtz machines, each with its own strengths and weaknesses, and relates them to cortical information processing. Copyright 1996 Elsevier Science Ltd.

Journal Article↗

PubMedDate not supplied

Dopamine: generalization and bonuses.

In the temporal difference model of primate dopamine neurons, their phasic activity reports a prediction error for future reward. This model is supported by a wealth of experimental data. However, in certain circumstances, the activity of the dopamine cells seems anomalous under the model, as they respond in particular ways to stimuli that are not obviously related to predictions of reward. In this paper, we address two important sets of anomalies, those having to do with generalization and novelty. Generalization responses are treated as the natural consequence of partial information; novelty responses are treated by the suggestion that dopamine cells multiplex information about reward bonuses, including exploration bonuses and shaping bonuses. We interpret this additional role for dopamine in terms of the mechanistic attentional and psychomotor effects of dopamine, having the computational role of guiding exploration.

Animals↗

PubMedDate not supplied

Opponent interactions between serotonin and dopamine.

Anatomical and pharmacological evidence suggests that the dorsal raphe serotonin system and the ventral tegmental and substantia nigra dopamine system may act as mutual opponents. In the light of the temporal difference model of the involvement of the dopamine system in reward learning, we consider three aspects of motivational opponency involving dopamine and serotonin. We suggest that a tonic serotonergic signal reports the long-run average reward rate as part of an average-case reinforcement learning model; that a tonic dopaminergic signal reports the long-run average punishment rate in a similar context; and finally speculate that a phasic serotonin signal might report an ongoing prediction error for future punishment.

Animals↗

PubMedDate not supplied

Acetylcholine in cortical inference.

Acetylcholine (ACh) plays an important role in a wide variety of cognitive tasks, such as perception, selective attention, associative learning, and memory. Extensive experimental and theoretical work in tasks involving learning and memory has suggested that ACh reports on unfamiliarity and controls plasticity and effective network connectivity. Based on these computational and implementational insights, we develop a theory of cholinergic modulation in perceptual inference. We propose that ACh levels reflect the uncertainty associated with top-down information, and have the effect of modulating the interaction between top-down and bottom-up processing in determining the appropriate neural representations for inputs. We illustrate our proposal by means of an hierarchical hidden Markov model, showing that cholinergic modulation of contextual information leads to appropriate perceptual inference.

Acetylcholine↗

PubMedDate not supplied

Test characteristics of the respiratory syncytial virus enzyme-linked immunoabsorbent assay in febrile infants < or = 60 days of age.

The test characteristics of rapid tests for respiratory syncytial virus (RSV) in infants may differ from older children secondary to a lower likelihood of previous illness with RSV. Our main goal was to establish the test characteristics of the RSV Abbott Testpack (TP) enzyme-linked immunoabsorbent assay (EIA) in febrile infants < or = 60 days of age. Our secondary goal was to determine the likelihood of RSV given a particular clinical syndrome and a negative or positive EIA. A prospective sample of infants with a temperature > or = 38.0 degrees C was evaluated during 2 successive RSV seasons. Conventional tissue and shell vial viral cultures were utilized as the reference standard. The RSV Abbott Testpack EIA had a sensitivity of 75% (95% CI 60-90%), a specificity of 98% (95% CI 96-100%), a positive predictive value of 89% (95% CI 77-100%), a negative predictive value of 95% (95% CI 91-98%), a likelihood ratio for a positive test of 35.5 (95% CI 11.4-110.7), and a likelihood ratio for a negative test of 0.26 (95% CI 0.14-0.47). Even with a negative EIA, patients with lower and upper respiratory tract illness still had a 22.3% and 5.5% chance of harboring RSV, respectively. The RSV Abbott Testpack is a useful diagnostic tool in the detection of RSV in febrile infants but has limitations. During months typically associated with RSV disease, a positive RSV TP indicates a high likelihood of illness, but clinicians should be wary of false negatives.

Child, Preschool↗