Perceptions of science. Tap into science 24-7.
Explore the source record for details and available documents.
Access to Information↗
Biomedical subjects
Terrence J Sejnowski
Publications and source records attributed to Terrence J Sejnowski.
Self-organizing neural systems based on predictive learning.
The ability to predict future events based on the past is an important attribute of organisms that engage in adaptive behaviour. One prominent computational method for learning to predict is called temporal-difference (TD) learning. It is so named because it uses the difference between successive predictions to learn to predict correctly. TD learning is well suited to modelling the biological phenomenon of conditioning, wherein an organism learns to predict a reward even though the reward may occur later in time. We review a model for conditioning in bees based on TD learning. The model illustrates how the TD-learning algorithm allows an organism to learn an appropriate sequence of actions leading up to a reward, based solely on reinforcement signals. The second part of the paper describes how TD learning can be used at the cellular level to model the recently discovered phenomenon of spike-timing-dependent plasticity. Using a biophysical model of a neocortical neuron, we demonstrate that the shape of the spike-timing-dependent learning windows found in biology can be interpreted as a form of TD learning occurring at the cellular level. We conclude by showing that such spike-based TD-learning mechanisms can produce direction selectivity in visual-motion-sensitive cells and can endow recurrent neocortical circuits with the powerful ability to predict their inputs at the millisecond time-scale.
Action Potentials↗
Independent sources of quantal variability at single glutamatergic synapses.
Variability in the size of single postsynaptic responses is a feature of most central neurons, although the source of this variability is not completely understood. The dominant source of variability could be either intersynaptic or intrasynaptic. To quantitatively examine this question, a biophysically realistic model of an idealized central axospinous synapse was used to assess mechanisms underlying synaptic variability measurements. Three independent sources of variability were considered: stochasticity of postsynaptic receptors ("channel noise"), variations of glutamate concentration in the synaptic cleft (Deltaq), and differences in the potency of vesicles released from different locations on the active zone [release-location dependence (RLD)]. As expected, channel noise was small (8% of the total variance) and Deltaq was the dominant source of variability (58% of total variance). Surprisingly, RLD accounted for a significant amount of variability (36%). Our simulations show that potency of release sites decreased with a length constant of approximately 100 nm, and that receptors were not activated by release events >300 nm away, which is consistent with the observation that single active zones are rarely >300 nm. RLD also predicts that the manner in which receptors are added or removed from synapses can dramatically affect the nature of the synaptic response, with increasing receptor density being more efficient than merely increasing synaptic area. Saturation levels and synaptic geometry were also important in determining the size and shape of the distribution of amplitudes recorded at different synapses.
Biophysics↗
Representation of color stimuli in awake macaque primary visual cortex.
We investigated the responses of single neurons in primary visual cortex (area V1) of awake monkeys to chromatic stimuli. Chromatic tuning properties, determined for homogeneous color patches presented on a neutral gray background, varied strongly between cells. The continuum of preferred chromaticities and tuning widths indicated a distributed representation of color signals in V1. When stimuli were presented on colored backgrounds, chromatic tuning was different in most neurons, and the changes in tuning were consistent with some degree of sensitivity of the neurons to the chromatic contrast between stimulus and background. Quantitatively, the average response changes matched the magnitudes of color induction effects measured in human subjects under corresponding stimulus conditions.
Animals↗
Dictionary learning algorithms for sparse representation.
Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).
Algorithms↗
Spatiochromatic receptive field properties derived from information-theoretic analyses of cone mosaic responses to natural scenes.
Neurons in the early stages of processing in the primate visual system efficiently encode natural scenes. In previous studies of the chromatic properties of natural images, the inputs were sampled on a regular array, with complete color information at every location. However, in the retina cone photoreceptors with different spectral sensitivities are arranged in a mosaic. We used an unsupervised neural network model to analyze the statistical structure of retinal cone mosaic responses to calibrated color natural images. The second-order statistical dependencies derived from the covariance matrix of the sensory signals were removed in the first stage of processing. These decorrelating filters were similar to type I receptive fields in parvo- or konio-cellular LGN in both spatial and chromatic characteristics. In the subsequent stage, the decorrelated signals were linearly transformed to make the output as statistically independent as possible, using independent component analysis. The independent component filters showed luminance selectivity with simple-cell-like receptive fields, or had strong color selectivity with large, often double-opponent, receptive fields, both of which were found in the primary visual cortex (V1). These results show that the "form" and "color" channels of the early visual system can be derived from the statistics of sensory signals.
Humans↗
The initiation of bursts in thalamic neurons and the cortical control of thalamic sensitivity.
Thalamic neurons generate high-frequency bursts of action potentials when a low-threshold (T-type) calcium current, located in soma and dendrites, becomes activated. Computational models were used to investigate the bursting properties of thalamic relay and reticular neurons. These two types of thalamic cells differ fundamentally in their ability to generate bursts following either excitatory or inhibitory events. Bursts generated with excitatory inputs in relay cells required a high degree of convergence from excitatory inputs, whereas moderate excitation drove burst discharges in reticular neurons from hyperpolarized levels. The opposite holds for inhibitory rebound bursts, which are more difficult to evoke in reticular neurons than in relay cells. The differences between the reticular neurons and thalamocortical neurons were due to different kinetics of the T-current, different electrotonic properties and different distribution patterns of the T-current in the two cell types. These properties enable the cortex to control the sensitivity of the thalamus to inputs and are also important for understanding states such as absence seizures.
Action Potentials↗
Model of thalamocortical slow-wave sleep oscillations and transitions to activated States.
During natural slow-wave sleep (SWS) in nonanesthetized cats, silent (down) states alternate with active (up) states; the down states are absent during rapid-eye-movement sleep and waking. Oscillations (<1 Hz) in SWS and transformation to an activated awake state were investigated with intracellular recordings in vivo and with computational models of the corticothalamic system. Occasional summation of the miniature EPSPs during the hyperpolarized (silent) phase of SWS oscillation activated the persistent sodium current and depolarized the membrane of cortical pyramidal (PY) cells sufficiently for spike generation. In the model, this triggered the active phase, which was maintained by lateral PY-PY excitation and persistent sodium current. Progressive depression of the excitatory interconnections and activation of Ca2+-dependent K+ current led to termination of the 20-25 Hz activity after 500-1000 msec. Including thalamocortical (TC) and thalamic reticular neurons in the model increased the duration of the active epochs up to 1-1.5 sec and introduced waning spindle sequences. An increase in acetylcholine activity, which is associated with activated states, was modeled by the reduction in the K+ leak current in PY and TC cells and by a decrease in intracortical PY-PY synaptic conductances. These changes eliminated the hyperpolarizing phases of network activity and transformed cortical neurons to tonic firing at 15-20 Hz. During the transition from SWS to the activated state, the input resistance of cortical neurons gradually increased and, in a fully activated state, reached the same or even higher values as during silent phases of SWS oscillations. The model describes many essential features of SWS and activated states in the thalamocortical system as well as the transition between them.
Action Potentials↗
Frequency-selective augmenting responses by short-term synaptic depression in cat neocortex.
Thalamic stimulation at frequencies between 5 and 15 Hz elicits incremental or 'augmenting' cortical responses. Augmenting responses can also be evoked in cortical slices and isolated cortical slabs in vivo. Here we show that a realistic network model of cortical pyramidal cells and interneurones including short-term plasticity of inhibitory and excitatory synapses replicates the main features of augmenting responses as obtained in isolated slabs in vivo. Repetitive stimulation of synaptic inputs at frequencies around 10 Hz produced postsynaptic potentials that grew in size and carried an increasing number of action potentials resulting from the depression of inhibitory synaptic currents. Frequency selectivity was obtained through the relatively weak depression of inhibitory synapses at low frequencies, and strong depression of excitatory synapses together with activation of a calcium-activated potassium current at high frequencies. This network resonance is a consequence of short-term synaptic plasticity in a network of neurones without intrinsic resonances. These results suggest that short-term plasticity of cortical synapses could shape the dynamics of synchronized oscillations in the brain.
Animals↗
Short- and medium-term plasticity associated with augmenting responses in cortical slabs and spindles in intact cortex of cats in vivo.
Plastic changes in the synaptic responsiveness of neocortical neurones, which occur after rhythmic stimuli within the frequency range of sleep spindles (10 Hz), were investigated in isolated neocortical slabs and intact cortex of anaesthetized cats by means of single, dual and triple simultaneous intracellular recordings in conjunction with recordings of local field potential responses. In isolated cortical slabs (10 mm long, 6 mm wide and 4-5 mm deep), augmenting responses to pulse-trains at 10 Hz (responses with growing amplitudes from the second stimulus in a train) were elicited only by relatively high-intensity stimuli. At low intensities, responses were decremental. The largest augmenting responses were evoked in neurones located close to the stimulation site. Quantitative analyses of the number of action potentials and the amplitude and area of depolarization during augmenting responses in a population of neurones recorded from slabs showed that the most dramatic increases in the number of spikes with successive stimuli, and the greatest increase in depolarization amplitude, were found in conventional fast-spiking (FS) neurones. The largest increase in the area of depolarization was found in regular-spiking (RS) neurones. Dual intracellular recordings from a pair of FS and RS neurones in the slab revealed more action potentials in the FS neurone during augmenting responses and a significant increase in the depolarization area of the RS neurone that was dependent on the firing of the FS neurone. Self-sustained seizures could occur in the slab after rhythmic stimuli at 10 Hz. In the intact cortex, repeated sequences of stimuli generating augmenting responses or spontaneous spindles could induce an increased synaptic responsiveness to single stimuli, which lasted for several minutes. A similar time course of increased responsiveness was obtained with induction of cellular plasticity. These data suggest that augmenting responses elicited by stimulation, as well as spontaneously occurring spindles, may induce short- and medium-term plasticity of neuronal responses.
Animals↗
Complexity of calcium signaling in synaptic spines.
Long-term potentiation and long-term depression are thought to be cellular mechanisms contributing to learning and memory. Although the physiological phenomena have been well characterized, little consensus of their underlying molecular mechanisms has emerged. One reason for this may be the under-appreciated complexity of the signaling pathways that can arise if key signaling molecules are discretely localized within the synapse. Recent findings suggest an unanticipated degree of structural organization at the synapse, and improved methods in cellular imaging of living tissue have provided much-needed information about the intracellular dynamics of Ca(2+), thought to be critical for both LTP and LTD. In this review, we briefly summarize some of these developments, and show that a more complete understanding of cellular signaling depends on the successful integration of traditional biochemistry and molecular biology with the spatial and temporal details of synaptic ultrastructure. Biophysically realistic computer simulations can have an important role in bridging these disciplines.
Animals↗
Single-trial variability in event-related BOLD signals.
Most current analysis methods for fMRI data assume a priori knowledge of the time course of the hemodynamic response (HR) to experimental stimuli or events in brain areas of interest. In addition, they typically assume homogeneity of both the HR and the non-HR "noise" signals, both across brain regions and across similar experimental events. When HRs vary unpredictably, from area to area or from trial to trial, an alternative approach is needed. Here, we use Infomax independent component analysis (ICA) to detect and visualize variations in single-trial HRs in event-related fMRI data. Six subjects participated in four fMRI sessions each in which ten bursts of 8-Hz flickering-checkerboard stimulation were presented for 0.5-s (short) or 3-s (long) durations at 30-s intervals. Five axial slices were acquired by a Bruker 3-T magnetic resonance imager at interscan intervals of 500 ms (TR). ICA decomposition of the resulting blood oxygenation level-dependent (BOLD) data from each session produced an independent component active in primary visual cortex (V1) and, in several sessions, another active in medial temporal cortex (MT/V5). Visualizing sets of BOLD response epochs with novel BOLD-image plots demonstrated that component HRs varied substantially and often systematically across trials as well as across sessions, subjects, and brain areas. Contrary to expectation, in four of the six subjects the V1 component HR contained two positive peaks in response to short-stimulus bursts, while components with nearly identical regions of activity in long-stimulus sessions from the same subjects were associated with single-peaked HRs. Thus, ICA combined with BOLD-image visualization can reveal dramatic and unforeseen HR variations not apparent to researchers analyzing their data with event-related response averaging and fixed HR templates.
Adult↗
Spike propagation synchronized by temporally asymmetric Hebbian learning.
Synchronously spiking neurons have been observed in the cerebral cortex and the hippocampus. In computer models, synchronous spike volleys may be propagated across appropriately connected neuron populations. However, it is unclear how the appropriate synaptic connectivity is set up during development and maintained during adult learning. We performed computer simulations to investigate the influence of temporally asymmetric Hebbian synaptic plasticity on the propagation of spike volleys. In addition to feedforward connections, recurrent connections were included between and within neuron populations and spike transmission delays varied due to axonal, synaptic and dendritic transmission. We found that repeated presentations of input volleys decreased the synaptic conductances of intragroup and feedback connections while synaptic conductances of feedforward connections with short delays became stronger than those of connections with longer delays. These adaptations led to the synchronization of spike volleys as they propagated across neuron populations. The findings suggests that temporally asymmetric Hebbian learning may enhance synchronized spiking within small populations of neurons in cortical and hippocampal areas and familiar stimuli may produce synchronized spike volleys that are rapidly propagated across neural tissue.
Action Potentials↗
A Monte Carlo model reveals independent signaling at central glutamatergic synapses.
We have developed a biophysically realistic model of receptor activation at an idealized central glutamatergic synapse that uses Monte Carlo techniques to simulate the stochastic nature of transmission following release of a single synaptic vesicle. For the a synapse with 80 AMPA and 20 NMDA receptors, a single quantum, with 3000 glutamate molecules, opened approximately 3 NMDARs and 20 AMPARs. The number of open receptors varied directly with the total number of receptors, and the fraction of open receptors did not depend on the ratio of co-localized AMPARs and NMDARs. Variability decreased with increases in either total receptor number or quantal size, and differences between the variability of AMPAR and NMDAR responses were due solely to unequal numbers of receptors at the synapse. Despite NMDARs having a much higher affinity for glutamate than AMPARs, quantal release resulted in similar occupancy levels in both receptor types. Receptor activation increased with number of transmitter molecules released or total receptor number, whereas occupancy levels were only dependent on quantal size. Tortuous diffusion spaces reduced the extent of spillover and the activation of extrasynaptic receptors. These results support the conclusion that signaling is spatially independent within and between central glutamatergic synapses.
Animals↗
Color opponency is an efficient representation of spectral properties in natural scenes.
The human visual system encodes the chromatic signals conveyed by the three types of retinal cone photoreceptors in an opponent fashion. This opponency is thought to reduce redundant information by decorrelating the photoreceptor signals. Correlations in the receptor signals are caused by the substantial overlap of the spectral sensitivities of the receptors, but it is not clear to what extent the properties of natural spectra contribute to the correlations. To investigate the influences of natural spectra and photoreceptor spectral sensitivities, we attempted to find linear codes with minimal redundancy for trichromatic images assuming human cone spectral sensitivities, or hypothetical non-overlapping cone sensitivities, respectively. The resulting properties of basis functions are similar in both cases. They are non-orthogonal, show strong opponency along an achromatic direction (luminance edges) and along chromatic directions, and they achieve a highly efficient encoding of natural chromatic signals. Thus, color opponency arises for the encoding of human cone signals, i.e. with strongly overlapping spectral sensitivities, but also under the assumption of non-overlapping spectral sensitivities. Our results suggest that color opponency may in part be a result of the properties of natural spectra and not solely a consequence of the cone spectral sensitivities.
Color Perception↗
Untangling spatial from temporal illusions.
Explore the source record for details and available documents.
Animals↗
Comparison of machine learning and traditional classifiers in glaucoma diagnosis.
Glaucoma is a progressive optic neuropathy with characteristic structural changes in the optic nerve head reflected in the visual field. The visual-field sensitivity test is commonly used in a clinical setting to evaluate glaucoma. Standard automated perimetry (SAP) is a common computerized visual-field test whose output is amenable to machine learning. We compared the performance of a number of machine learning algorithms with STATPAC indexes mean deviation, pattern standard deviation, and corrected pattern standard deviation. The machine learning algorithms studied included multilayer perceptron (MLP), support vector machine (SVM), and linear (LDA) and quadratic discriminant analysis (QDA), Parzen window, mixture of Gaussian (MOG), and mixture of generalized Gaussian (MGG). MLP and SVM are classifiers that work directly on the decision boundary and fall under the discriminative paradigm. Generative classifiers, which first model the data probability density and then perform classification via Bayes' rule, usually give deeper insight into the structure of the data space. We have applied MOG, MGG, LDA, QDA, and Parzen window to the classification of glaucoma from SAP. Performance of the various classifiers was compared by the areas under their receiver operating characteristic curves and by sensitivities (true-positive rates) at chosen specificities (true-negative rates). The machine-learning-type classifiers showed improved performance over the best indexes from STATPAC. Forward-selection and backward-elimination methodology further improved the classification rate and also has the potential to reduce testing time by diminishing the number of visual-field location measurements.
Artificial Intelligence↗
Slow feature analysis: unsupervised learning of invariances.
Invariant features of temporally varying signals are useful for analysis and classification. Slow feature analysis (SFA) is a new method for learning invariant or slowly varying features from a vectorial input signal. It is based on a nonlinear expansion of the input signal and application of principal component analysis to this expanded signal and its time derivative. It is guaranteed to find the optimal solution within a family of functions directly and can learn to extract a large number of decorrelated features, which are ordered by their degree of invariance. SFA can be applied hierarchically to process high-dimensional input signals and extract complex features. SFA is applied first to complex cell tuning properties based on simple cell output, including disparity and motion. Then more complicated input-output functions are learned by repeated application of SFA. Finally, a hierarchical network of SFA modules is presented as a simple model of the visual system. The same unstructured network can learn translation, size, rotation, contrast, or, to a lesser degree, illumination invariance for one-dimensional objects, depending on only the training stimulus. Surprisingly, only a few training objects suffice to achieve good generalization to new objects. The generated representation is suitable for object recognition. Performance degrades if the network is trained to learn multiple invariances simultaneously.
Algorithms↗