Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “representation learning”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18Linked to original sources
PubMed2003

A taxonomy for spatiotemporal connectionist networks revisited: the unsupervised case.

Spatiotemporal connectionist networks (STCNs) comprise an important class of neural models that can deal with patterns distributed in both time and space. In this article, we widen the application domain of the taxonomy for supervised STCNs recently proposed by Kremer (2001) to the unsupervised case. This is possible through a reinterpretation of the state vector as a vector of latent (hidden) variables, as proposed by Meinicke (2000). The goal of this generalized taxonomy is then to provide a nonlinear generative framework for describing unsupervised spatiotemporal networks, making it easier to compare and contrast their representational and operational characteristics. Computational properties, representational issues, and learning are also discussed, and a number of references to the relevant source publications are provided. It is argued that the proposed approach is simple and more powerful than the previous attempts from a descriptive and predictive viewpoint. We also discuss the relation of this taxonomy with automata theory and state-space modeling and suggest directions for further work.

Classification↗
PubMed2004

Canonical correlation analysis: an overview with application to learning methods.

We present a general method using kernel canonical correlation analysis to learn a semantic representation to web images and their associated text. The semantic space provides a common representation and enables a comparison between the text and images. In the experiments, we look at two approaches of retrieving images based on only their content from a text query. We compare orthogonalization approaches against a standard cross-representation retrieval technique known as the generalized vector space model.

Journal Article↗
PubMed2002

Integrated perspective of evolving intrapsychic and person-environment functions: implications for deaf and hard of hearing individuals.

The authors review and integrate certain diverse theories to explain and suggest appropriate interventions for difficulties in socioemotional functioning experienced by many deaf persons. These diverse perspectives include a hierarchical circular systems approach, psychosocial stage theory, social learning theory, and representational models, or evolving expectancies of others. These perspectives, which can facilitate understanding of social behaviors and development and lead to improved interventions, provide background for a 3-level model proposed in the article. The model focuses on the relationship between the deaf person and the proximal social environment. The model's first level takes into account intrapsychic processes such as self-concept; the second highlights reciprocal interactions between the person and the social environment. The third describes the resulting memories and expectancies that develop and evolve and that influence the person's previous intrapsychic thoughts, feelings, and perceptions. These, in turn, affect social interactions, in a recurrent, spiraling fashion. This hierarchical model can be used as a framework for concurrent or sequential interventions with Deaf people.

Affect↗
PubMed2001

Chromatic structure of natural scenes.

We applied independent component analysis (ICA) to hyperspectral images in order to learn an efficient representation of color in natural scenes. In the spectra of single pixels, the algorithm found basis functions that had broadband spectra and basis functions that were similar to natural reflectance spectra. When applied to small image patches, the algorithm found some basis functions that were achromatic and others with overall chromatic variation along lines in color space, indicating color opponency. The directions of opponency were not strictly orthogonal. Comparison with principal-component analysis on the basis of statistical measures such as average mutual information, kurtosis, and entropy, shows that the ICA transformation results in much sparser coefficients and gives higher coding efficiency. Our findings suggest that nonorthogonal opponent encoding of photoreceptor signals leads to higher coding efficiency and that ICA may be used to reveal the underlying statistical properties of color information in natural scenes.

Color↗
PubMed1995

Why Johnny can't reengineer health care processes with information technology.

Many educational institutions are developing curricula that integrate computer and business knowledge and skills concerning a specific industry, such as banking or health care. We have developed a curriculum that emphasizes, equally, medical, computer, and business management concepts. Along the way we confronted a formidable obstacle, namely the domain specificity of the reference disciplines. Knowledge within each domain is sufficiently different from other domains that it reduces the leverage of building on preexisting knowledge and skills. We review this problem from the point of view of cognitive science (in particular, knowledge representation and machine learning) to suggest strategies for coping with incommensurate domain ontologies. These strategies include reflective judgment, implicit learning, abstraction, generalization, analogy, multiple inheritance, project-orientation, selectivity, goal- and failure-driven learning, and case- and story-based learning.

Commerce↗
PubMed1988

Hippocampal system dysfunction and odor discrimination learning in rats: impairment or facilitation depending on representational demands.

The performance of normal rats and that of rats with hippocampal system damage were compared on acquisition of different versions of the same two-odor discrimination task that placed different encoding and representational demands on memory. Rats with fornix lesions were impaired when explicit comparisons among multiple odor cues and differential response choices were encouraged. However, when odor-cue comparison was hindered and explicit cues for response choice were eliminated, rats with fornix lesions out performed normal animals. The results support an hypothesis that the hippocampal system is critical to a memory representation based on encoding relations among multiple percepts, and other brain systems support performance adaptations based on encodings of stimuli individually.

Animals↗
PubMed2002

Maps in birds: representational mechanisms and neural bases.

The often extraordinary navigational behavior of birds is based in part on their ability to learn map-like representations of the heterogeneous distribution of environmental stimuli in space. Whether navigating small-scale laboratory environments or large-scale field environments, birds appear to be reliant on a directional framework, for example that provided by the sun, to learn how stimuli are distributed in space and to represent them as a map. The avian hippocampus plays a critical role in some aspects of map learning. Recent results from electrophysiological studies hint at the possibility that different aspects of space may be represented in the activity of different neuronal types in the avian hippocampus.

Animals↗
PubMedDate not supplied

Representation in natural and artificial agents: an embodied cognitive science perspective.

The goal of the present paper is to provide an embodied cognitive science view on representation. Using the fundamental task of category learning, we will demonstrate that this perspective enables us to shed new light on many pertinent issues and opens up new prospects for investigation. The main focus of this paper is on the prerequisites to acquire representations of objects in the real world. We suggest that the main prerequisite is embodiment which allows an agent--human, animal or robot--to manipulate its sensory input such that invariances are generated. These invariances, in turn, are the basis of representation formation. In other words, the paper does not focus on representations per se, but rather discusses the various processes involved in order to make learning and representation acquisition possible. The argument structure is as follows. First we introduce two new perspectives on representation, namely frame-of-reference, and complete agent. Then we elaborate the complete agent perspective and focus in particular on embodiment and situatedness. We argue that embodiment has two main aspects, a dynamic and an information theoretic one. Focusing on the latter, there are a number of implications: Representation can only be understood if the embedding of the neural substrate in the physical agent is known, which includes morphology (shape), positioning and nature of sensors. Because an autonomous mobile agent in the real world is exposed to a continuously changing high-dimensional stream of sensory stimulation, if it is to learn category distinctions, it first needs a focus of attention mechanism, and then it must have a way to reduce the dimensionality of this high-dimensional sensory stream. Learning is very hard because the invariances are typically not found in the sensory data directly--the classical problem of object constancy: it is a so-called type 2 problem. Rather than trying to improve the learning algorithms--which is the standard approach--the embodied cognitive science view suggests a different approach which focuses on the nature of the data: the agent is not passively exposed to a given data distribution, but, by exploiting its body and through the interaction with the environment, it can actually generate the data. More specifically, it can generate correlated data that has the property that it can be easily learned. This learnability is due to redundancies resulting from the appropriate interactions with the environment. Through such interactions, the former type 2 problem is transformed into a type 1 problem, thus reducing the complexity of the learning task by orders of magnitude. By observing the frame-of-reference problem we will discuss to what extent these invariances are reflected--represented--in the "neural substrate", i.e. the internal mechanisms of the agent. It is concluded, that representation is not a concept that can be studied in the abstract, but should be elaborated in the context of concrete agent-environment interactions. These ideas are all illustrated with examples of natural agents and artificial agents. In particular, we will present a suite of experiments on simulated and real-world artificial agents instantiating the main arguments.

Algorithms↗
PubMed2001

Properties of incremental projection learning.

We proposed a method of incremental projection learning which provides exactly the same generalization capability as that obtained by batch projection learning in the previous paper. However, properties of the method have not yet been investigated. In this paper, we analyze its properties from the following aspects: First, it is shown that some of the training data which is regarded as redundant in most incremental learning methods have potential effectiveness, i.e. they will contribute to better generalization capability in the future learning process. Based on this fact, an improved criterion for redundancy of additional training data is derived. Second, the relationship between prior and posterior learning results is investigated where effective training data is classified into two categories from the viewpoint of improving generalization capability. Finally, a simpler representation of incremental projection learning under certain conditions is given. The size of memory required for storing prior results in the representation is fixed and independent of the total number of training data.

Learning↗
PubMed2003

Stimulus representation in SOP: I. Theoretical rationalization and some implications.

THE SOP MODEL [INFORMATION PROCESSING IN ANIMALS: Memory Mechanisms, Erlbaum, Hillsdale, NJ, 1981, p. 5] is described in terms of its assumed stimulus representation, network characteristics, and rules for learning and performance. It is shown how several Pavlovian conditioning phenomena can be accounted on the basis of the model's presumed stimulus representation. Challenges to the SOP model prompted the adoption of a componential stimulus representation in: AESOP [Contemporary Learning Theories: Pavlovian Conditioning and the Status of Traditional Learning Theory, Erlbaum, Hillsdale, NJ, 1989, p. 149], this was a dual representation of the unconditioned stimulus (US), and C-SOP [Contemporary Learning: Theory and Application, Erlbaum, Mahwah, NJ, 2001, p. 23], this was a multi-component representation of the conditioned stimulus (CS). The assumption of a componential CS representation, where large numbers of elements can be separately learned about, necessitated a modification of the learning rule. The modified, "constrained" rule was found useful to explain timing characteristics of Pavlovian conditioned responses, as well as data offered by Rescorla [J. Exp. Psychol. Anim. Behav. Process. 26 (2000) 428; Q. J. Exp. Psychol. 54B (2001) 53; J. Exp. Psychol. Anim. Behav. Process. 28 (2002) 163] showing that stimuli trained in compound do not share the same quantitative fate.

Journal Article↗
PubMed2005

Simulations of a modified SOP model applied to retrospective revaluation of human causal learning.

Dickinson and Burke (1996) proposed a modified version of Wagner's (1981) SOP associative theory to explain retrospective revaluation of human causal judgments. In this modified SOP (MSOP), excitatory learning occurs when cue and outcome representations are either both directly activated or both associatively activated. By contrast, inhibitory learning occurs when one representation is directly activated while the other is associatively activated. Finite node simulations of MSOP yielded simple acquisition, overshadowing, blocking, and inhibitory learning under forward contingencies. Importantly, retrospective revaluation was predicted in the form of unovershadowing and backward inhibitory learning. However, MSOP did not yield backward blocking. These predictions are evaluated against the relevant empirical evidence and contrasted with the predictions of other associative theories that have been applied to retrospective revaluation of human causal and predictive learning.

Forecasting↗
PubMedDate not supplied

Applications of the self-organising map to reinforcement learning.

This article is concerned with the representation and generalisation of continuous action spaces in reinforcement learning (RL) problems. A model is proposed based on the self-organising map (SOM) of Kohonen [Self Organisation and Associative Memory, 1987] which allows either the one-to-one, many-to-one or one-to-many structure of the desired state-action mapping to be captured. Although presented here for tasks involving immediate reward, the approach is easily extended to delayed reward. We conclude that the SOM is a useful tool for providing real-time, on-line generalisation in RL problems in which the latent dimensionalities of the state and action spaces are small. Scalability issues are also discussed.

Neural Networks, Computer↗
PubMed2026

Dissecting spatial patterning and signaling with directional diffusion in spatial multi-omics.

Spatial multi-omics sequencing enables the simultaneous profiling of transcriptomics, proteomics, and epigenomics at a spatial resolution, offering insights into complex tissue organization and molecular regulation. However, the effective integration of multiple omics modalities in a spatial context remains a major challenge. Here, we present SpaDDM, a spatial multi-omics integration framework based on directional diffusion models (DDMs), which supports spatial pattern identification, cross-omics alignment, and inter-and intracellular signaling flow analysis. SpaDDM employs DDM-based graph networks to learn omics-specific representations by jointly incorporating spatial coordinates and molecular measurements within each modality, followed by an attention mechanism to align features across modalities. We benchmarked SpaDDM on diverse spatial multi-omics datasets, including transcriptomics-epigenomics and transcriptomics-proteomics combinations across multiple tissues and species. SpaDDM consistently outperformed existing methods by more accurately deciphering spatial tissue patterns and effectively reducing the boundary noise between spatial regions. Moreover, the learned low-dimensional coembedded representations of individual cells serve as integral mediators for inferring the signaling flows that underlie spatial patterning. Finally, we demonstrated that SpaDDM alignment of complementary information across multi-omics layers facilitates cross-omics translation and significantly improves the prediction of cell state alignments.

Multiomics↗
PubMed2002

Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework.

The complementary learning systems framework provides a simple set of principles, derived from converging biological, psychological and computational constraints, for understanding the differential contributions of the neocortex and hippocampus to learning and memory. The central principles are that the neocortex has a low learning rate and uses overlapping distributed representations to extract the general statistical structure of the environment, whereas the hippocampus learns rapidly using separated representations to encode the details of specific events while minimizing interference. In recent years, we have instantiated these principles in working computational models, and have used these models to address human and animal learning and memory findings, across a wide range of domains and paradigms. Here, we review a few representative applications of our models, focusing on two domains: recognition memory and animal learning in the fear-conditioning paradigm. In both domains, the models have generated novel predictions that have been tested and confirmed.

Journal Article↗
PubMed1999

Homing in pigeons: the role of the hippocampal formation in the representation of landmarks used for navigation.

When given repeated training from a location, homing pigeons acquire the ability to use familiar landmarks to navigate home. Both control and hippocampal-lesioned pigeons succeed in learning to use familiar landmarks for homing. However, the landmark representations that guide navigation are strikingly different. Control and hippocampal-lesioned pigeons were initially given repeated training flights from two locations. On subsequent test days from the two training locations, all pigeons were rendered anosmic to eliminate use of their navigational map and were phase- or clock-shifted to examine the extent to which their learned landmark representations were dependent on the use of the sun as a compass. We show that control pigeons acquire a landmark representation that allows them to directly use landmarks without reference to the sun to guide their flight home, called "pilotage". Hippocampal-lesioned birds only learn to use familiar landmarks at the training location to recall the compass direction home, based on the sun, flown during training, called "site-specific compass orientation." The results demonstrate that for navigation of 20 km or more in a natural field setting, the hippocampal formation is necessary if homing pigeons are to learn a spatial representation based on numerous independent landmark elements that can be used to directly guide their return home.

Animals↗
PubMed2004

Analyzing irregular working hours: lessons learned in the development of RAS 1.0--The Representation and Analysis Software.

Actual working hours of employees vary widely, especially in the transportation industry. We developed a tool, the RAS (Representation and Analysis Software), to ease the assessment of such irregular hours and the transfer of existing knowledge of proper schedule design to the problem of irregular hours. This article discusses several critical design questions that were addressed during software development in order for it to assess irregular work patterns, including the (1) importance, in spite of a lack of established definitions, of basic concepts like, e.g., night shift, (2) difficulty of modeling and adapting existing knowledge on proper design, and (3) large number of analytical methods and additional data beyond company schedule that are necessary to meet the needs of various research groups. This article describes how the RAS addresses these three issues by illustrating its application to the work schedule of a train driver involved in the Hinton train disaster.

Employment↗
PubMed1996

How dependencies between successive examples affect on-line learning.

We study the dynamics of on-line learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. The representation error measures how well the environment is represented by the network after learning. The prediction error is the average error that a continually learning network makes on the next example. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation, the higher the representation error, i.e., the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau. Plateaus are flat spots on the error surface, which can severely slow down the learning process. In particular, they are notorious in applications with multilayer perceptrons. Our results, which are confirmed by simulations of a multilayer perceptron learning a chaotic time series using backpropagation, explain how dependencies between examples can help the learning process to escape from a plateau.

Learning↗
PubMed1993

Neural representations for sensory-motor control, I: Head-centered 3-D target positions from opponent eye commands.

This article describes how corollary discharges from outflow eye movement commands can be transformed by two stages of opponent neural processing into a head-centered representation of 3-D target position. This representation implicitly defines a cyclopean coordinate system whose variables approximate the binocular vergence and spherical horizontal and vertical angles with respect to the observer's head. Various psychophysical data concerning binocular distance perception and reaching behavior are clarified by this representation. The representation provides a foundation for learning head-centered and body-centered invariant representations of both foveated and non-foveated 3-D target positions. It also enables a solution to be developed of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space.

Brain↗