Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “representation learning”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6Linked to original sources
PubMed2002

Associative learning and elemental representation: II. Generalization and discrimination.

This paper follows on from an earlier companion paper (McLaren & Mackintosh, 2000), in which we further developed the elemental associative theory put forward in McLaren, Kaye, and Mackintosh (1989). Here, we begin by explicating the idea that stimuli can be represented as patterns of activation distributed across a set of units and that different stimuli activate partially overlapping sets (the degree of overlap being proportional to the similarity of the stimuli). A consequence of this view is that the overall level of activity of some of the units representing a stimulus may be dependent on the nature of the other stimuli present at the same time. This allows an elemental analysis in which provision for the representation of configurations of stimuli is made. A selective review of studies of generalization and discrimination learning, including peak shift, transfer along a continuum, configural discrimination, and summation, suggests that the principles embodied in this class of theory deserve careful consideration and will form part of any successful model of associative learning in humans or animals. There are some phenomena that require an elemental/associative explanation.

Association Learning↗
PubMed1992

Combining exemplar-based category representations and connectionist learning rules.

Adaptive network and exemplar-similarity models were compared on their ability to predict category learning and transfer data. An exemplar-based network (Kruschke, 1990a, 1990b, 1992) that combines key aspects of both modeling approaches was also tested. The exemplar-based network incorporates an exemplar-based category representation in which exemplars become associated to categories through the same error-driven, interactive learning rules that are assumed in standard adaptive networks. Experiment 1, which partially replicated and extended the probabilistic classification learning paradigm of Gluck and Bower (1988a), demonstrated the importance of an error-driven learning rule. Experiment 2, which extended the classification learning paradigm of Medin and Schaffer (1978) that discriminated between exemplar and prototype models, demonstrated the importance of an exemplar-based category representation. Only the exemplar-based network accounted for all the major qualitative phenomena; it also achieved good quantitative predictions of the learning and transfer data in both experiments.

Concept Formation↗
PubMed1990

Hippocampal representation in place learning.

The generality of the place-learning impairment associated with hippocampal system damage was challenged using methods of training that permitted subjects to form an individual association between the place of escape and a particular navigational route in an open-field water maze. Both normal rats and rats with fornix lesions (FX rats) acquired this task rapidly, although FX rats were slightly slower in achieving minimum escape latencies. In postcriterion testing, FX rats occasionally made near misses but, more often, their escape performance was indistinguishable from that of intact rats. Results from a variety of probe tests indicated that FX rats, like normal rats, had based their performance on a representation of multiple distal cues but their representation, unlike that of normal rats, was inflexible in that it could not be used to guide performance when the cues or starting position were altered. These results parallel those from other studies of hippocampal function in animals and humans: The learning deficit consequent to hippocampal system damage (1) is not specific to a particular category of learning materials, but is dependent on the representational demands of the task; (2) is observed when task demands encourage a representation based on relations among multiple cues, but not when the task encourages adaptation to an individual (or compound) stimulus; (3) spares acquisition of fundamental procedures needed to perform the task; and (4) impairs the flexible use of learned information in tests other than repetition of the learning experience.

Animals↗
PubMed2001

A variational method for learning sparse and overcomplete representations.

An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variational approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectation-maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients.

Journal Article↗
PubMed2003

Dictionary learning algorithms for sparse representation.

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).

Algorithms↗
PubMedDate not supplied

Learning of somatosensory representations for texture discrimination using a temporal coherence principle.

In order to perform appropriate actions, animals need to quickly and reliably classify their sensory input. How can representations suitable for classification be acquired from statistical properties of the animal's natural environment? Akin to behavioural studies in rats, we investigate this question using texture discrimination by the vibrissae system as a model. To account for the rat's active sensing behaviour, we record whisker movements in a hardware model. Based on these signals, we determine the response of primary neurons, modelled as spatio-temporal filters. Using their output, we train a second layer of neurons to optimise a temporal coherence objective function. The performance in classifying textures using a single cell strongly correlates with the cell's temporal coherence; hence output cells outperform primary cells. Using a simple, unsupervised classifier, the performance on the output cell population is same as if using a sophisticated supervised classifier on the primary cells. Our results demonstrate that the optimisation of temporal coherence yields a representation that facilitates subsequent classification by selectively conveying relevant information.

Animals↗
PubMed2001

Altering object representations through category learning.

Previous research has shown that objects that are grouped together in the same category become more similar to each other and that objects that are grouped in different categories become increasingly dissimilar, as measured by similarity ratings and psychophysical discriminations. These findings are consistent with two theories of the influence of concept learning on similarity. By a Strategic Judgment Bias account, the categories associated with objects are explicitly used as cues for determining similarity, and objects that are categorized together are judged to be more similar because similarity is not only a function of the objects themselves, but also the objects' category labels. By a Changed Object Description account, category learning alters the description of the objects themselves, emphasizing properties that are relevant for categorization. A new method for distinguishing between these accounts is introduced which measures the difference between the similarity ratings of categorized objects to a neutral object. The results indicate both strategic biases based on category labels and genuine representational change, with the strategic bias affecting mostly objects belonging to different categories and the representational change affecting mostly objects belonging to the same category.

Adult↗
PubMed2004

Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value.

In extinction, an animal learns that a previously conditioned stimulus (CS+) no longer predicts delivery of a salient reinforcer (unconditioned stimulus, UCS). Rodent studies indicate that extinction relies on amygdala-prefrontal interactions and involves formation of memories that inhibit, without actually erasing, the original conditioning trace. Whether extinction learning in humans follows similar neurobiological principles is unknown. We used functional magnetic resonance imaging to measure human brain activity evoked during olfactory aversive conditioning and extinction learning. Neural responses in orbitofrontal cortex and amygdala were preferentially enhanced during extinction, suggesting potential cross-species preservation of learning mechanisms that oppose conditioning. Moreover, by manipulating UCS aversiveness via reinforcer inflation, we showed that a CS+ retains access to representations of UCS value in distinct regions of ventral prefrontal cortex, even as extinction proceeds.

Adolescent↗
PubMed2001

Rapid regulation of GAD67 mRNA and protein level in cortical neurons after sensory learning.

Cortical representations of different modalities can be modified by sensory learning. Our previous studies in the barrel cortex showed that expansion of the cortical representation of a row of vibrissae could be induced by pairing stimulation of a row of vibrissae with a tail shock. The plastic change in cortical reactivity to the input used during the training was accompanied by increased density of GABA immunoreactive neurons in the involved row of cortical barrels. Using the same paradigm, the present study examined the pathway of GABA synthesis-expression of GAD67 mRNA and immunoreactivity of GAD67 isoenzyme in the barrel cortex of mice after sensory learning. In situ hybridization revealed that the GAD67 mRNA level was elevated in one row of barrels in the trained group as well as in controls receiving vibrissae stimulation alone. In contrast, elevation of immunoreactivity of the GAD67 protein occurred only in the trained group. The density of GABA-immunoreactive neurons in the hollows of barrels representing the row of vibrissae activated during the training was increased by 50%. These data indicated that sensory stimulation alone affected expression of the 67 kDa glutamate decarboxylase isoenzyme synthesis pathway, whereas the processes involved in cortical plasticity induced by associative learning modified this pathway additionally at the level of translation.

Animals↗
PubMed2005

Category representation for classification and feature inference.

This research's purpose was to contrast the representations resulting from learning of the same categories by either classifying instances or inferring instance features. Prior inference learning research, particularly T. Yamauchi and A. B. Markman (1998), has suggested that feature inference learning fosters prototype representation, whereas classification learning encourages exemplar representation. Experiment 1 supported this hypothesis. Averaged and individual participant data from transfer after inference training were better fit by a prototype than by an exemplar model. However, Experiment 2, with contrasting inference learning conditions, indicated that the prototype model was mimicking a set of label-based bidirectional rules, as determined by the inference learning task demands in Experiment 1. Only the set of rules model accounted for all the inference learning conditions in these experiments.

Humans↗
PubMedDate not supplied

Vector representation of associative learning.

I. P. Pavlov has shown that conditioned reflexes are selective both with respect to conditioned stimuli and to conditioned reflexes elicited by those conditioned stimuli. At the neuronal level, selective aspects of conditional stimuli are based on detectors selectively tuned to the respective stimuli. The selective aspects of conditioned reflexes are due to command neurons representing specific unconditioned reflexes. It can be assumed that conditioned reflexes result from association between selective detectors and specific command neurons. The detectors activated by a conditional stimulus constitute a combination of excitations--a detector excitation vector. The detector excitation vector acts on a command neuron via a set of plastic synapses--a synaptic weight vector. Plastic synapses are modified in the process of learning, making the command neuron selectively tuned to a specific conditioned stimulus. The selective tuning of a particular command neuron to a specific excitation vector referred to a conditioned stimulus is the basis of associative learning. The probabilities of conditioned reflexes elicited by conditional and differential stimuli implicitly contain information concerning excitation vectors that encode the respective stimuli. The contribution of the vector code to associative learning was explored combining differential color conditioning with intracellular recording from color-coding neurons. It is shown that colors in carps and monkeys are represented on a hypersphere in the four-dimensional space similar with human color space. The basis of the color space is constituted by red-green, blue-yellow, brightness, and darkness neurons.

Association Learning↗
PubMedDate not supplied

Vector representation of associative learning.

I. P. Pavlov [12] has shown that conditioned reflexes are selective both with respect to conditioned stimuli and to conditioned reflexes elicited by those conditioned stimuli. At the neuronal level selective aspects of conditioned stimuli are based on detectors selectively tuned to respective stimuli. The selective aspects of conditioned reflexes are due to command neurons representing specific unconditioned reflexes. It can be assumed that conditioned reflexes result from association between selective detectors and specific command neurons. The detectors activated by a conditioned stimulus constitute a combination of excitations--a detector excitation vector. The detector excitation vector acts on a command neuron via a set of plastic synapses--a synaptic weight vector. Plastic synapses are modified in the process of learning making command neuron selectively tuned to a specific conditioned stimulus. The selective tuning of a particular command neuron to a specific excitation vector referred to a conditioned stimulus is a basis of associative learning. The probabilities of conditioned reflexes elicited by conditioned and differential stimuli implicitly contain information concerning excitation vectors that encode respective stimuli. Contribution of the vector code to associative learning was explored combining differential color conditioning with intracellular recording from color-coding neurons. It was shown that colors in carps and monkeys are represented on a hypersphere in the four-dimensional space similar to human color space. The basis of the color space is constituted by red-green, blue-yellow, brightness and darkness neurons.

Animals↗
PubMed2005

Learning viewpoint invariant perceptual representations from cluttered images.

In order to perform object recognition, it is necessary to form perceptual representations that are sufficiently specific to distinguish between objects, but that are also sufficiently flexible to generalize across changes in location, rotation, and scale. A standard method for learning perceptual representations that are invariant to viewpoint is to form temporal associations across image sequences showing object transformations. However, this method requires that individual stimuli be presented in isolation and is therefore unlikely to succeed in real-world applications where multiple objects can co-occur in the visual input. This paper proposes a simple modification to the learning method that can overcome this limitation and results in more robust learning of invariant representations.

Algorithms↗
PubMed2006

Episodic representations support early semantic learning: evidence from midazolam induced amnesia.

Current controversy exists regarding the role of episodic representations in the formation of long-term semantic memories. Using the drug midazolam to induce temporary amnesia we tested participants' memories for newly learned facts in a semantic cue condition or an episodic and semantic cue condition. Following midazolam administration, memory performance was superior in the episodic and semantic condition, suggesting early semantic learning is supported by episodic representations.

Adult↗
PubMed2005

Rapid modulation of GABA concentration in human sensorimotor cortex during motor learning.

Movement representations within the human primary motor and somatosensory cortices can be altered by motor learning. Decreases in local GABA concentration and its release may facilitate this plasticity. Here we use in vivo magnetic resonance spectroscopy (MRS) to noninvasively measure serial changes in GABA concentration in humans in a brain region including the primary sensorimotor cortex contralateral to the hand used for an isometric motor sequence learning task. Thirty minutes of motor sequence learning reduced the mean GABA concentration within a 2 x 2 x 2-cm3 voxel by almost 20%. This reduction was specific to motor learning: 30 min of similar, movements with an unlearnable, nonrepetitive sequence were not associated with changes in GABA concentration. No significant changes in GABA concentration were found in the primary sensorimotor cortex ipsilateral to the hand used for learning. These changes suggest remarkably rapid, regionally specific short-term presynaptic modulation of GABAergic input that should facilitate motor learning. Although apparently confined to the contralateral hemisphere, the magnitude of changes seen within a large spectroscopic voxel suggests that these changes occur over a wide local neocortical field.

Adult↗
PubMed2004

Representation of well-learned information in the monkey hippocampus.

In the neocortex, extensive training results in enhanced neuronal selectivity for learned stimuli relative to novel stimuli. This enhanced selectivity has been taken as evidence for learning-related plasticity. Much less is known, in contrast, about the representation of well-learned information in the hippocampus. In this study, we examined the responses of individual hippocampal neurons to well-learned and novel stimuli presented in the context of an associative learning task. There was no difference in the response magnitude or visual response latency of hippocampal neurons to the well-learned and novel stimuli. In contrast, hippocampal neurons responded significantly more selectively to the well-learned stimuli relative to the novel stimuli. These findings show that hippocampal cells, like neocortical cells, show greater selectivity to well-learned stimuli compared to novel stimuli.

Animals↗
PubMed2025

Orthrus: Towards Evolutionary and Functional RNA Foundation Models.

In the face of rapidly accumulating genomic data, our ability to accurately predict key mature RNA properties that underlie transcript function and regulation remains limited. Pre-trained genomic foundation models offer an avenue to adapt learned RNA representations to biological prediction tasks. However, existing genomic foundation models are trained using strategies borrowed from textual domains that do not leverage biological domain knowledge. Here, we introduce Orthrus, a Mamba-based mature RNA foundation model pre-trained using a novel self-supervised contrastive learning objective with biological augmentations. Orthrus is trained by maximizing embedding similarity between curated pairs of RNA transcripts, where pairs are formed from splice isoforms of 10 model organisms and transcripts from orthologous genes in 400+ mammalian species from the Zoonomia Project. This training objective results in a latent representation that clusters RNA sequences with functional and evolutionary similarities. We find that the generalized mature RNA isoform representations learned by Orthrus significantly outperform genomic foundation models on mRNA property prediction tasks, and requires only a fraction of fine-tuning data to do so. Finally, we show that Orthrus is capable of capturing divergent biological function of individual transcript isoforms.

Journal Article↗