Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “representation learning”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7Linked to original sources
PubMed2025

A multi-modal transformer for cell type-agnostic regulatory predictions.

Sequence-based deep learning models have emerged as powerful tools for deciphering the cis-regulatory grammar of the human genome but cannot generalize to unobserved cellular contexts. Here, we present EpiBERT, a multi-modal transformer that learns generalizable representations of genomic sequence and cell type-specific chromatin accessibility through a masked accessibility-based pre-training objective. Following pre-training, EpiBERT can be fine-tuned for gene expression prediction, achieving accuracy comparable to the sequence-only Enformer model, while also being able to generalize to unobserved cell states. The learned representations are interpretable and useful for predicting chromatin accessibility quantitative trait loci (caQTLs), regulatory motifs, and enhancer-gene links. Our work represents a step toward improving the generalization of sequence-based deep neural networks in regulatory genomics.

Humans↗
PubMed2004

Reinforcement learning with via-point representation.

In this paper, we propose a new learning framework for motor control. This framework consists of two components: reinforcement learning and via-point representation. In the field of motor control, conventional reinforcement learning has been used to acquire control sequences such as cart-pole or stand-up robot control. Recently, researchers have become interested in hierarchical architecture, such as multiple levels, and multiple temporal and spatial scales. Our new framework contains two levels of hierarchical architecture. The higher level is implemented using via-point representation, which corresponds to macro-actions or multiple time scales. The lower level is implemented using a trajectory generator that produces primitive actions. Our framework can modify the ongoing movement by means of temporally localized via-points and trajectory generation. Successful results are obtained in computer simulation of the cart-pole swing up task.

Computer Simulation↗
PubMed1992

A sensorimotor basis for motor learning: evidence indicating specificity of practice.

Our previous work (Proteau, Marteniuk, Girouard, & Dugas, 1987) was concerned with determining whether with relatively extensive practice on a movement aiming task, as the skill theoretically starts becoming open-loop, there would be evidence for a decreasing emphasis on visual feedback for motor control. We eliminated vision of the moving limb after moderate and extensive practice and found that the movement became more dependent on this feedback with greater amounts of practice. In the present study, we wished to test the hypothesis, developed from our previous work, that at the base of movement learning is a sensorimotor representation that consists of integrated information from central processes and sensory feedback derived from previous experiences on the movement task. A strong test of this hypothesis would be the prediction that for an aiming task, the addition of vision, after moderate and relatively extensive practice without vision, would lead to an increasingly large movement decrement, relative to appropriate controls. We found good support for this prediction. From these and our previous results, and the idea of the sensorimotor representation underlying learning, we develop the idea that learning is specific to the conditions that prevail during skill acquisition. This has implications for the ideas of the generalized motor program and schema theory.

Feedback↗
PubMed1998

Basing categorization on individuals and events.

Exemplar, prototype, and connectionist models typically assume that events constitute the basic unit of learning and representation in categorization. In these models, each learning events updates a statistical representation of a category independently of other learning events. An implication is that events involving the same individual affect learning independently and are not integrated into a single structure that represents the individual in an internal model of the world. A series of experiments demonstrates that human subjects track individuals across events, establish representations of them, and use these representations in categorization. These findings are consistent with "representationalism," the view that an internal model of the world constitutes a physical level of representation in the brain, and that the brain does not simply capture the statistical properties of events in an undifferentiated dynamical system. Although categorization is an inherently statistical process that produces generalization, pattern completion, frequency effects, and adaptive learning, it is also an inherently representational process that establishes an internal model of the world. As a result, representational structures evolve in memory to track the histories of individuals, accumulate information about them, and simulate them in events.

Humans↗
PubMed2003

Spatial representation of predictive motor learning.

A key feature of skilled motor behavior is the ability of the CNS to predict the consequences of its actions. Such prediction occurs when one hand pulls on an object held in the other hand; the restraining hand generates an anticipatory increase in grip force, thereby preventing the object from slipping. When manipulating a novel object, the CNS adapts its predictive response to ensure that predictions are accurately tuned to the dynamics of the object. Here we examine whether learning to predict the consequences of an action on a novel object is restricted to the actions performed during manipulation or generalizes to novel actions. A bimanual task in which subjects held an object in each hand and the relationship between actions on one object and the motion of the other could be computer controlled from trial-to-trial was used. In four conditions we varied the spatial relationship between the direction of force subjects applied to the left-hand object and the consequent direction of motion of an object held in their right hand, which subjects were required to restrain. The results show that predictive learning was local to the direction of forces experienced during learning and that the magnitude of predictive responses was greatly reduced for novel directions of action of the left hand. The pattern of generalization shows that the representation of predictive learning is spatially local and can be approximated as having a spatially narrow Gaussian basis function.

Conditioning, Psychological↗
PubMed1998

Efficient Learning of VAM-Based Representation of 3D Targets and its Active Vision Applications.

There has been a considerable interest in using active vision for various applications. This interest is primarily because active vision can enhance machine vision capabilities by dynamically changing the camera parameters based on the content of the scene. An important issue in active vision is that of representing 3D targets in a manner that is invariant to changing camera configurations. This paper addresses this representation issue for a robotic active vision system. An efficient Vector Associative Map (VAM)-based learning scheme is proposed to learn a joint-based representation. Computer simulations and experiments are first performed to evaluate the effectiveness of this scheme using the University of Illinois Active Vision System (UIAVS). The invariance property of the learned representation is then exploited to develop several robotic applications. These include, detecting moving targets, saccade control, planning saccade sequences and controlling a robot manipulator.

Journal Article↗
PubMed2002

Are there representational shifts during category learning?

Early theories of categorization assumed that either rules, or prototypes, or exemplars were exclusively used to mentally represent categories of objects. More recently, hybrid theories of categorization have been proposed that variously combine these different forms of category representation. Our research addressed the question of whether there are representational shifts during category learning. We report a series of experiments that tracked how individual subjects generalized their acquired category knowledge to classifying new critical transfer items as a function of learning. Individual differences were observed in the generalization patterns exhibited by subjects, and those generalizations changed systematically with experience. Early in learning, subjects generalized on the basis of single diagnostic dimensions, consistent with the use of simple categorization rules. Later in learning, subjects generalized in a manner consistent with the use of similarity-based exemplar retrieval, attending to multiple stimulus dimensions. Theoretical modeling was used to formally corroborate these empirical observations by comparing fits of rule, prototype, and exemplar models to the observed categorization data. Although we provide strong evidence for shifts in the kind of information used to classify objects as a function of categorization experience, interpreting these results in terms of shifts in representational systems underlying perceptual categorization is a far thornier issue. We provide a discussion of the challenges of making claims about category representation, making reference to a wide body of literature suggesting different kinds of representational systems in perceptual categorization and related domains of human cognition.

Analysis of Variance↗
PubMed1995

Representational consequences of two modes of learning.

Two experiments tested the hypothesis that a non-strategic mode of rule learning results in atomic representations. In each case subjects were taught concepts under two different conditions, designed to favour either non-strategic or strategic learning. Following training, subjects demonstrated an equivalent ability to discriminate exemplars from non-exemplars of the concepts acquired under each of these two learning conditions. However, performance on a decompositional inference task, which required access to critical constituent elements within the rule representations, was disproportionately poor for a concept acquired under the training condition that favoured non-strategic learning. These findings lend support to the view that rule acquisition can be mediated by either of two modes of learning, and that the format of knowledge representations is not equivalent across these two learning modes.

Adult↗
PubMed1984

Building spatial representations through primary and secondary learning.

The current study examined the conditions under which cognitive representations of spatial information are stored in orientation-specific ways (like pictures) versus orientation-free ways. College students learned simple paths by viewing a map of them (map condition), by walking them while blindfolded (walk condition), or by directly viewing the route from a single vantage point (look condition). Blindfolded subjects then stood on the route in various orientations and made directional judgments to other locations on the route. When subjects learned the route by indirect, symbolic means (map), judgments were quite easy when aligned with the learned map orientation and were difficult if the judgment was not aligned with the learned map orientation. However, when subjects learned the route by more direct, primary spatial learning (walk or look), there were no alignment effects. Thus, subjects treat information from primary (direct) and secondary (symbolic) spatial learning in distinct ways. Learning from a map results in a figural representation that has great precision but a specific orientation. Learning the route more directly results in a representation that is less precise but one that can be used in more flexible ways.

Cognition↗
PubMed2000

Scale-invariant superiority of foveal vision in perceptual categorization.

The recognition of objects is exceedingly difficult in indirect view. This complication cannot be explained in terms of retino-cortical magnification, as size scaling fails to establish position invariance both for character recognition [Strasburger, H. & Rentschler, I. (1996) Eur. J. Neurosci., 8 1787-1791] and pattern classification [Jüttner, M. & Rentschler, I. (1996) Vision Res., 36, 1007-1021]. Thus we compared, for two tasks of discrimination learning and category learning with respect to a common set of grey-level patterns, how humans perform in foveal and extrafoveal vision. Observers learnt to discriminate (size-scaled) images equally well in foveal and extrafoveal view, whereas they displayed profound deficiencies in extrafoveal category learning for the same patterns. From the behavioural learning data, internal representations of the learning signals were reconstructed by means of computer simulations. For foveal view, these representations were found to be veridical to their physical counterparts for both learning tasks. For extrafoveal view, they were severely distorted for category learning but not for discrimination learning. A variance reduction of the pattern classes by a factor of 100 reduced the dissociation between extrafoveal categorization and discrimination but did not remove it. These observations suggest a scale-invariant superiority of foveal vision for learning object categories. This implies a high degree of space variance of visual cognition which is vastly underestimated by classical measures of visual performance, e.g. acuity, visual field and contrast sensitivity.

Adult↗
PubMed2002

Learning of sequences of finger movements and timing: frontal lobe and action-oriented representation.

Motor sequence learning involves learning of a sequence of effectors with which to execute a series of movements and learning of a sequence of timings at which to execute the movements. In this study, we have segregated the neural correlates of the two learning mechanisms. Moreover, we have found an interaction between the two learning mechanisms in the frontal areas, which we claim as suggesting action-oriented coding in the frontal lobe. We used positron emission tomography and compared three learning conditions with a visuo-motor control condition. In two learning conditions, the subjects learned either a sequence of finger movements with random timing or a sequence of timing with random use of fingers. In the third condition the subjects learned to execute a sequence of specific finger movements at specific timing; we argue that it was only in this condition that the motor sequence was coded as an action-oriented representation. By looking for condition by session interactions (learning vs. control conditions over sessions), we have removed nonspecific time effects and identified areas that showed a learning-related increment of activation during learning. Learning of a finger sequence was associated with an increment of activation in the right intraparietal sulcus region and medial parietal cortex, whereas learning of a timing sequence was associated with an increment of activation in the lateral cerebellum, suggesting separate mechanisms for learning effector and temporal sequences. The left intraparietal sulcus region showed an increment of activation in learning of both finger and timing sequences, suggesting an overlap between the two learning mechanisms. We also found that the mid-dorsolateral prefrontal cortex, together with the medial and lateral premotor areas, became increasingly active when subjects learned a sequence that specified both fingers and timing, that is, when subjects were able to prepare specific motor action. These areas were not active when subjects learned a sequence that specified fingers or timing alone, that is, when subjects were still dependent on external stimuli as to the timing or fingers with which to execute the movements. Frontal areas may integrate the effector and temporal information of a motor sequence and implement an action-oriented representation so as to perform a motor sequence accurately and quickly. We also found that the mid-dorsolateral prefrontal cortex was distinguished from the ventrolateral prefrontal cortex and anterior fronto-polar cortex, which showed sustained activity throughout learning sessions and did not show either an increment or decrement of activation.

Adult↗
PubMed2002

Challenging the widespread assumption that connectionism and distributed representations go hand-in-hand.

One of the central claims associated with the parallel distributed processing approach popularized by D.E. Rumelhart, J.L. McClelland and the PDP Research Group is that knowledge is coded in a distributed fashion. Localist representations within this perspective are widely rejected. It is important to note, however, that connectionist networks can learn localist representations and many connectionist models depend on localist coding for their functioning. Accordingly, a commitment to distributed representations should be considered a specific theoretical claim regarding the structure of knowledge rather than a core principle, as often assumed. In this paper, it is argued that there are fundamental computational and empirical challenges that have not yet been addressed by distributed connectionist theories that are readily accommodated within localist approaches. This is highlighted in the context of modeling word and nonword naming, the domain in which some of the strongest claims have been made. It is shown that current PDP models provide a poor account of naming monosyllable items, and that distributed representations make it difficult for these models to scale up to more complex language phenomena. At the same time, models that learn localist representations are shown to hold promise in supporting many of the core reading and language functions on which PDP models fail. It is concluded that the common rejection of localist coding schemes within connectionist architectures is premature.

Cognitive Science↗
PubMed2003

The cognitive and neural architecture of sequence representation.

The authors theorize that 2 neurocognitive sequence-learning systems can be distinguished in serial reaction time experiments, one dorsal (parietal and supplementary motor cortex) and the other ventral (temporal and lateral prefrontal cortex). Dorsal system learning is implicit and associates noncategorized stimuli within dimensional modules. Ventral system learning can be implicit or explicit It also allows associating events across dimensions and therefore is the basis of cross-task integration or interference, depending on degree of cross-task correlation of signals. Accordingly, lack of correlation rather than limited capacity is responsible for dual-task effects on learning. The theory is relevant to issues of attentional effects on learning; the representational basis of complex, sequential skills; hippocampal-versus basal ganglia-based learning; procedural versus declarative memory; and implicit versus explicit memory.

Cognition↗
PubMed2003

Neuronal populations and single cells representing learned auditory objects.

The neural representations associated with learned auditory behaviours, such as recognizing individuals based on their vocalizations, are not well described. Higher vertebrates learn to recognize complex conspecific vocalizations that comprise sequences of easily identified, naturally occurring auditory objects, which should facilitate the analysis of higher auditory pathways. Here we describe the first example of neurons selective for learned conspecific vocalizations in adult animals--in starlings that have been trained operantly to recognize conspecific songs. The neuronal population is found in a non-primary forebrain auditory region, exhibits increased responses to the set of learned songs compared with novel songs, and shows differential responses to categories of learned songs based on recognition training contingencies. Within the population, many cells respond highly selectively to a subset of specific motifs (acoustic objects) present only in the learned songs. Such neuronal selectivity may contribute to song-recognition behaviour, which in starlings is sensitive to motif identity. In this system, both top-down and bottom-up processes may modify the tuning properties of neurons during recognition learning, giving rise to plastic representations of behaviourally meaningful auditory objects.

Acoustic Stimulation↗
PubMed2004

Learning modulates the ensemble representations for odors in primary olfactory networks.

Recent evidence suggests that odor-driven responses in the insect antennal lobe (AL) can be modified by associative and nonassociative processes, as has been shown in the vertebrate olfactory bulb. However, the specific network changes that occur in response to olfactory learning remain unknown. To characterize changes in AL network activity during learning, we developed an in vivo protocol in Manduca sexta that allows continuous monitoring of neural ensembles and feeding behavior over the course of olfactory conditioning. Here, we show that Pavlovian conditioning produced a net recruitment of responsive neural units across the AL that persisted after conditioning. Recruitment only occurred when odor reliably predicted food. Conversely, when odor did not predict food, a net loss of responsive units occurred. Simultaneous measures of feeding responses indicated that the treatment-specific patterns of neural recruitment were positively correlated with changes in the insect's behavioral response to odor. In addition to recruitment, conditioning also produced consistent and profound shifts in the temporal responses of 16% of recorded units. These results show that odor representations in the AL are dynamic and related to olfactory memory consolidation. We furthermore provide evidence that the basis of the learning-dependent changes in the AL is not simply an increase in activity in the neural network representing an odorant. Rather, learning produces a restructuring of spatial and temporal components of network responses to odor in the AL.

Animals↗
PubMedDate not supplied

Age and experience-dependent representational reorganization during spatial learning.

Previously, we found that aged rats showed a significant enhancement of hippocampal CA1 place cell spatial specificity, as well as a reduction of hilar place cell spatial specificity, during asymptote performance of a spatial memory task. Because such an age effect was not observed when animals performed a nonspatial task, the present study tested the hypothesis that the different patterns of spatial selectivity observed in memory and nonmemory tests reflected a redistribution of spatial representations that occurred in response to changing task demands. In the present experiment, after animals became familiar with the test environment and motor demands of performance on a radial maze, CA1 and hilar place cells were recorded as they learned a spatial memory task. CA1 place cells recorded from unimpaired old, but not impaired old or young, animals became more spatially selective as animals learned the task. Hilar spatial selectivity for both age groups was not significantly related to choice accuracy. These data support the hypothesis that at least a subpopulation of aged rats may benefit from reorganization of spatial representations in such a way that the normal age-related spatial learning deficit is attenuated.

Aging↗
PubMed2000

Evidence for effector independent and dependent representations and their differential time course of acquisition during motor sequence learning.

To investigate the representation of motor sequence, we tested transfer effects in a motor sequence learning paradigm. We hypothesize that there are two sequence representations, effector independent and dependent. Further, we postulate that the effector independent representation is in visual/spatial coordinates, that the effector dependent representation is in motor coordinates, and that their time courses of acquisition during learning are different. Twelve subjects were tested in a modified 2x10 task. Subjects learned to press two keys (called a set) successively on a keypad in response to two lighted squares on a 3x3 display. The complete sequence to be learned was composed of ten such sets, called a hyperset. Training was given in the normal condition and sequence recall was assessed in the early, intermediate, and late stages in three conditions, normal, visual, and motor. In the visual condition, finger-keypad mapping was rotated 90 degrees while the keypad-display mapping was kept identical to normal. In the motor condition, the keypad-display mapping was also rotated 90 degrees, resulting in an identical finger-display mapping as in normal. Subjects formed two groups with each group using a different normal condition. One group learned the sequence in a standard keypad-hand setting and subsequently recalled the sequence using a rotated keypad-hand setting in the test conditions. The second group learned the sequence with a rotated keypad-hand setting and subsequently recalled the sequence with a standard keypad-hand setting in the test conditions. Response time (RT) and sequencing errors during recall were recorded. Although subjects committed more sequencing errors in both testing conditions, visual and motor, as compared to the normal condition, the errors were below chance level. Sequencing errors did not differ significantly between visual and motor conditions. Further, the sequence recall accuracy was over 70% even by the early stage when the subjects performed the sequence for the first time with the altered conditions, visual and motor. There were parallel improvements thereafter in all the conditions. These results of positive transfer of sequence knowledge across conditions that use dissimilar finger movements point to an effector independent sequence representation, possibly in visual/spatial coordinates. Initially the RTs were similar in the visual and the motor conditions, but with training RTs in the motor condition became significantly shorter than in the visual condition, as revealed by significant interaction for the testing stage and condition term in the repeated measures ANOVA. Moreover, using RTs for single key pressing in the three conditions as baseline indices, it was again observed that RTs in the visual and motor conditions were not significantly different in the early stage, but motor RTs became significantly shorter by the late testing stage. These results support the hypothesis that the motor condition benefits more than the visual because it uses identical effector movements to the normal condition. Further, these results argue for the existence of effector dependent sequence representation, in motor coordinates, which is acquired relatively slowly. The difference in the time course of learning of these two representations may account for the differential involvement of brain areas in early and late learning phases found in lesion and imaging studies.

Adult↗
PubMed2004

The emerging lexicon of children with phonological delays: phonotactic constraints and probability in acquisition.

The effects of phonotactic constraints (i.e., the status of a sound as correctly or incorrectly articulated) and phonotactic probability (i.e., the likelihood of a sound sequence) on lexical acquisition have been investigated independently. This study investigated the interactive influence of phonotactic constraints and phonotactic probability on lexical acquisition in 3 groups of children: children with functional phonological delays (PD), phonology-matched, younger, typically developing children (PM), and age-/vocabulary-matched typically developing peers (AVM). Sixty-eight children participated in a multitrial word-learning task involving nonwords varying in phonotactic constraints (IN vs. OUT) and phonotactic probability (common vs. rare). Correct and error responses were analyzed. Results indicated that OUT sound sequences were learned more rapidly than IN sound sequences. This suggests that OUT sounds may be salient because they represent only a small subset of the child's sound system. The effect of phonotactic probability varied across groups: Children with PD showed a common sound sequence disadvantage, younger PM children showed a common sound sequence advantage, and AVM children showed no effect. Moreover, error analyses indicated that children with PD had particular difficulty creating lexical representations and associations between lexical and semantic representations when learning common sound sequences. Children with PD may rely more heavily on lexical representations to learn new words or may have difficulty learning common sound sequences because of the high degree of similarity between these sequences and other known words. Finally, the effect of phonotactic probability was consistent across IN and OUT sound sequences, suggesting that the lexical representation of both correctly articulated and misarticulated words is based on the adult-target pronunciation.

Articulation Disorders↗