Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “sparse learning”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3Linked to original sources
PubMed1999

Language acquisition from sparse input without error feedback.

A connectionist-inspired, parallel processing network is presented which learns, on the basis of (relevantly) sparse input, to assign meaning interpretations to novel test sentences in both active and passive voice. Training and test sentences are generated from a simple recursive grammar, but once trained, the network successfully processes thousands of sentences containing deeply embedded clauses. All training is unsupervised with regard to error feedback - only Hebbian and self-organizing forms of training are employed. In addition, the active-passive distinction is acquired without any supervised provision of cues or flags (in the output layer) that indicate whether the input sentence is in active or passive sentence. In more detail: (1) The model learns on the basis of a corpus of about 1000 sentences while the set of potential test sentences contains over 100 million sentences. (2) The model generalizes its capacity to interpret active and passive sentences to substantially deeper levels of clausal embedding. (3) After training, the model satisfies criteria for strong syntactic and strong semantic systematicity that humans also satisfy. (4) Symbolic message passing occurs within the model's output layer. This symbolic aspect reflects certain prior language acquistion assumptions.

Journal Article↗
PubMed2003

Learning optimized features for hierarchical models of invariant object recognition.

There is an ongoing debate over the capabilities of hierarchical neural feedforward architectures for performing real-world invariant object recognition. Although a variety of hierarchical models exists, appropriate supervised and unsupervised learning methods are still an issue of intense research. We propose a feedforward model for recognition that shares components like weight sharing, pooling stages, and competitive nonlinearities with earlier approaches but focuses on new methods for learning optimal feature-detecting cells in intermediate stages of the hierarchical network. We show that principles of sparse coding, which were previously mostly applied to the initial feature detection stages, can also be employed to obtain optimized intermediate complex features. We suggest a new approach to optimize the learning of sparse features under the constraints of a weight-sharing or convolutional architecture that uses pooling operations to achieve gradual invariance in the feature hierarchy. The approach explicitly enforces symmetry constraints like translation invariance on the feature set. This leads to a dimension reduction in the search space of optimal features and allows determining more efficiently the basis representatives, which achieve a sparse decomposition of the input. We analyze the quality of the learned feature representation by investigating the recognition performance of the resulting hierarchical network on object and face databases. We show that a hierarchy with features learned on a single object data set can also be applied to face recognition without parameter changes and is competitive with other recent machine learning recognition approaches. To investigate the effect of the interplay between sparse coding and processing nonlinearities, we also consider alternative feedforward pooling nonlinearities such as presynaptic maximum selection and sum-of-squares integration. The comparison shows that a combination of strong competitive nonlinearities with sparse coding offers the best recognition performance in the difficult scenario of segmentation-free recognition in cluttered surround. We demonstrate that for both learning and recognition, a precise segmentation of the objects is not necessary.

Learning↗
PubMed2006

A computational theory of hippocampal function, and empirical tests of the theory.

The main aim of the paper is to present an up-to-date computational theory of hippocampal function and the predictions it makes about the different subregions (dentate gyrus, CA3 and CA1), and to examine behavioral and electrophysiological data that address the functions of the hippocampus and particularly its subregions. Based on the computational proposal that the dentate gyrus produces sparse representations by competitive learning and via the mossy fiber pathway forces new representations on the CA3 during learning (encoding), it has been shown behaviorally that the dentate gyrus supports spatial pattern separation during learning. Based on the computational proposal that CA3-CA3 autoassociative networks are important for episodic memory, it has been shown behaviorally that the CA3 supports spatial rapid one-trial learning, learning of arbitrary associations where space is a component, pattern completion, spatial short-term memory, and sequence learning by associations formed between successive items. The concept that the CA1 recodes information from CA3 and sets up associatively learned backprojections to neocortex to allow subsequent retrieval of information to neocortex, is consistent with findings on consolidation. Behaviorally, the CA1 is implicated in processing temporal information as shown by investigations requiring temporal order pattern separation and associations across time; computationally this could involve temporal decay memory, and temporal sequence memory which might also require CA3. The perforant path input to DG is implicated in learning, to CA3 in retrieval from CA3, and to CA1 in retrieval after longer time intervals ("intermediate-term memory").

Animals↗
PubMed2000

Spermiogenesis in commercial poultry species: anatomy and control.

Spermatogenesis is a complicated process dependent upon several factors. Formation of a testis requires the interaction of gene-products and hormones (androgens) on pluripotent tissue. In birds, the female is the heterogametic (ZW) sex, but W chromosomal genes do not influence gonadal development in a way similar to the SRY gene on the mammalian Y chromosome. However, autosomal genes such as SRY-like HMG box gene 9 (SOX9) may influence gonadal development. Hormones affect development; male gonads subjected to estrogen form an ovotestis, whereas ovaries exposed to aromatase inhibitors form an atypical testis. Sertoli cell numbers are set early in spermiogenesis, possibly under the influence of follicle-stimulating hormone and thyroid hormone, and this may determine the number of gonial cells that can be supported. Sertoli cells make a number of substances that affect testicular development and function, particularly anti-Müllerian hormone, which inhibits female oviduct formation from the Müllerian anlage, inhibits aromatase activity to stop estrogen production, and possibly stimulates androgen production by Leydig cells. Undifferentiated primordial germ cells (PGC) migrate to the testis and are converted to spermatogonia by factors from gonadal ridge tissue and androgens. The PGC of males in the ovary form oocytes of Z genotype, whereas the female PGC in males form mostly Z sperm (with a few of W genotype). Transmission electron microscopy micrographs of turkey testis are presented, and control of spermatogenesis by hormones and cytokines is discussed. This discussion includes follicle-stimulating hormone, luteinizing hormone, inhibin, activin, follistatin, tumor necrosis factor-alpha, growth factors such as transforming growth factor-beta, interleukins, and interferon. Although information concerning paracrine and autocrine regulation of the avian testis by these substances is sparse, much can be learned from mammalian studies, in which putative roles of each of these substances have been established. How Sertoli cells cause directed apoptosis of spermatogonia using the Fas-ligand, Fas-receptor pathway is reviewed, as well as ways to circumvent this process. A possible role for ubiquitin concerning prevention of heat-induced damage to the testis is presented.

Animals↗
PubMed2025

Sparse deconvolution of cell type medleys in spatial transcriptomics.

Mapping cell distributions across spatial locations with whole-genome coverage is essential for understanding cellular responses and signaling However, current deconvolution models aim to estimate the proportions of distinct cell types in each spatial transcriptomics spot by integrating reference single-cell data. These models often assume strong overlap between the reference and spatial datasets, neglecting biology-grounded constraints such as sparsity and cell-type variations, as well as technical sparsity. As a result, these methods rely on over-permissive algorithms that ignore given constraints leading to inaccurate predictions, particularly in heterogeneous or unmatched datasets. We introduce Weight-Induced Sparse Regression (WISpR), a machine learning algorithm that integrates spot-specific hyperparameters and sparsity-driven modeling. Unlike conventional approaches that neglect biology-grounded constraints, WISpR accurately predicts cell-type distributions while preserving biological coherence, i.e., spatially and functionally consistent cell-type localization, even in unmatched datasets. Benchmarking against five alternative methods across ten datasets, WISpR consistently outperformed competitors and predicted cellular landscapes in both normal and cancerous tissues. By leveraging sparse cell-type arrangements, WISpR provides biologically informed, high-resolution cellular maps. Its ability to decode tissue organization in both healthy and diseased states highlights WISpR's practical utility for spatial transcriptomics, particularly in challenging settings involving noise, sparsity, or reference mismatches.

Humans↗
PubMed2004

Socially learned preferences for differentially rewarded tokens in the brown capuchin monkey (Cebus apella).

Social learning is assumed to underlie traditions, yet evidence indicating social learning in capuchin monkeys (Cebus apella), which exhibit traditions, is sparse. The authors tested capuchins for their ability to learn the value of novel tokens using a previously familiar token-exchange economy. Capuchins change their preferences in favor of a token worth a high-value food reward after watching a conspecific model exchange 2 differentially rewarded tokens, yet they fail to develop a similar preference after watching tokens paired with foods in the absence of a conspecific model. They also fail to learn that the value of familiar tokens has changed. Information about token value is available in all situations, but capuchins seem to pay more attention in a social situation involving novel tokens.

Animals↗
PubMedDate not supplied

Environmental Influences and What Have We Learned From Dietary Behavior Change With Children?

Sparse research exists on factors influencing use of dietary supplements or on programs to increase dietary supplement use among children. Borrowing from the literature concerning influences on children's diet, parents likely influence children's dietary supplement intake by controlling their availability and accessibility at home and through parenting and socialization strategies. Borrowing from the literature on programs to change diet, programs to promote dietary supplements will likely work when implemented with highly motivated individuals, delivered by specialists and promoting increased, rather than decreased, consumption. Research into these areas is warranted.

Journal Article↗
PubMed2002

Self-regulation mechanism of temporally asymmetric Hebbian plasticity.

Recent biological experimental findings have shown that synaptic plasticity depends on the relative timing of the pre- and postsynaptic spikes. This determines whether long-term potentiation (LTP) or long-term depression (LTD) is induced. This synaptic plasticity has been called temporally asymmetric Hebbian plasticity (TAH). Many authors have numerically demonstrated that neural networks are capable of storing spatiotemporal patterns. However, the mathematical mechanism of the storage of spatiotemporal patterns is still unknown, and the effect of LTD is particularly unknown. In this article, we employ a simple neural network model and show that interference between LTP and LTD disappears in a sparse coding scheme. On the other hand, the covariance learning rule is known to be indispensable for the storage of sparse patterns. We also show that TAH has the same qualitative effect as the covariance rule when spatiotemporal patterns are embedded in the network.

Animals↗
PubMed1999

Mental health in schools and system restructuring.

Because health is not the primary business of schools, a school's response to mental health and psychosocial concerns usually is limited to targeted problems seen as direct barriers to learning. And because resources are sparse, priority is given to problems defined in legislative mandates. As a result, school-based mental health services are available only to a small proportion of the many students who require assistance, and interventions generally are narrowly focused and short-term. To better meet the needs of those served and to serve greater numbers, emerging trends are pushing for restructuring of school-owned services and greater linkage with community resources to develop multifaceted, comprehensive, integrated approaches. This review (a) provides an overview of what schools currently do related to mental health and psychosocial concerns, (b) clarifies key emerging trends, and (c) explores implications for major systemic changes.

Adolescent↗
PubMed1992

Proposal for an administrative curriculum in pediatric emergency medicine fellowship training.

Administrative tasks make up a significant component of the practice of pediatric emergency medicine (PEM) physicians. Our survey of 10 academic pediatric emergency departments revealed that PEM physicians who are primarily clinical spent an average of 15% of their time on administrative tasks, and PEM physicians whose positions are administrative as well as clinical spent 30 to 60% of their time on administrative tasks. Of the 101 programs responding to our survey of 220 pediatric residency programs, 80% did not address hospital administrative issues, and many that did address these issues allowed these topics only one hour of presentation time per year. It is clear that there is a discrepancy between the demands placed upon PEM physicians to perform administrative tasks and the sparse or nonexistent opportunities for learning about administrative issues during residency training. It is incumbent upon pediatric emergency fellowship programs to provide an inclusive and well-structured administrative curriculum for their trainees. This article suggests a framework for such a curriculum.

Curriculum↗
PubMedDate not supplied

Trauma on eight wheels. A study of roller skating injuries in Seattle.

Thirty-five fractures secondary to roller skating accidents from March 15 to October 1, 1979 were treated in the University of Washington-affiliated hospitals. Twenty-eight involved the wrist and elbow. Only three were in skeletally immature individuals. Sixty-three percent of the patients were female with the 20- to 34-year-old age group the most commonly involved. A fall, usually backwards, onto outstretched arms and hands was the most frequent mechanism of injury. A detailed study of the popular outdoor skating area in Seattle, Green Lake, indicated that more than 75% of the injuries occurred on sloped surfaces with otherwise favorable environmental conditions. Most people were either first time skaters or had not skated since childhood. Based on our study, we recommend: (1) skating on level, familiar terrain; (2) learning to skate in a sparsely congested area; (3) skating with experienced partners who can give instructions; and (4) using protective equipment, such as knee and elbow pads and rigid wrist splints.

Adolescent↗
PubMed2001

Does learning a complex task have to be complex? A study in learning decomposition.

Many theories of skill acquisition have had considerable success in addressing the fine details of learning in relatively simple tasks, but can they scale up to complex tasks that are more typical of human learning in the real world? Some theories argue for scalability by making the implicit assumption that complex tasks consist of many smaller parts, which are learned according to basic learning principles. Surprisingly, there has been rather sparse empirical testing of this crucial assumption. In this article, we examine this assumption directly by decomposing the learning in the Kanfer-Ackerman Air-Traffic Controller Task (Ackerman, 1988) from the learning at the global level all the way down to the learning at the keystroke level. First, we reanalyze the data from Ackerman (1988) and show that the learning in this complex task does indeed reflect the learning of smaller parts at the keystroke level. Second, in a follow-up eye-tracking experiment, we show that a large portion of the learning at the keystroke level reflects the learning even at a lower, i.e., attentional level.

Attention↗
PubMed2005

Bilinear sparse coding for invariant vision.

Recent algorithms for sparse coding and independent component analysis (ICA) have demonstrated how localized features can be learned from natural images. However, these approaches do not take image transformations into account. We describe an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model. We show that from an arbitrary set of natural images, the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformations. The learned generative model can be used to translate features to different locations, thereby reducing the need to learn the same feature at multiple locations, a limitation of previous approaches to sparse coding and ICA. Our results suggest that by explicitly modeling the interaction between local image features and their transformations, the sparse bilinear approach can provide a basis for achieving transformation-invariant vision.

Algorithms↗
PubMed2005

Statistical model of natural stimuli predicts edge-like pooling of spatial frequency channels in V2.

BACKGROUND: It has been shown that the classical receptive fields of simple and complex cells in the primary visual cortex emerge from the statistical properties of natural images by forcing the cell responses to be maximally sparse or independent. We investigate how to learn features beyond the primary visual cortex from the statistical properties of modelled complex-cell outputs. In previous work, we showed that a new model, non-negative sparse coding, led to the emergence of features which code for contours of a given spatial frequency band. RESULTS: We applied ordinary independent component analysis to modelled outputs of complex cells that span different frequency bands. The analysis led to the emergence of features which pool spatially coherent across-frequency activity in the modelled primary visual cortex. Thus, the statistically optimal way of processing complex-cell outputs abandons separate frequency channels, while preserving and even enhancing orientation tuning and spatial localization. As a technical aside, we found that the non-negativity constraint is not necessary: ordinary independent component analysis produces essentially the same results as our previous work. CONCLUSION: We propose that the pooling that emerges allows the features to code for realistic low-level image features related to step edges. Further, the results prove the viability of statistical modelling of natural images as a framework that produces quantitative predictions of visual processing.

Models, Statistical↗
PubMedDate not supplied

Associative memory in networks of spiking neurons.

Here, we develop and investigate a computational model of a network of cortical neurons on the base of biophysically well constrained and tested two-compartmental neurons developed by Pinsky and Rinzel [Pinsky, P. F., & Rinzel, J. (1994). Intrinsic and network rhythmogenesis in a reduced Traub model for CA3 neurons. Journal of Computational Neuroscience, 1, 39-60]. To study associative memory, we connect a pool of cells by a structured connectivity matrix. The connection weights are shaped by simple Hebbian coincidence learning using a set of spatially sparse patterns. We study the neuronal activity processes following an external stimulation of a stored memory. In two series of simulation experiments, we explore the effect of different classes of external input, tonic and flashed stimulation. With tonic stimulation, the addressed memory is an attractor of the network dynamics. The memory is displayed rhythmically, coded by phase-locked bursts or regular spikes. The participating neurons have rhythmic activity in the gamma-frequency range (30-80 Hz). If the input is switched from one memory to another, the network activity can follow this change within one or two gamma cycles. Unlike similar models in the literature, we studied the range of high memory capacity (in the order of 0.1 bit/synapse), comparable to optimally tuned formal associative networks. We explored the robustness of efficient retrieval varying the memory load, the excitation/inhibition parameters, and background activity. A stimulation pulse applied to the identical simulation network can push away ongoing network activity and trigger a phase-locked association event within one gamma period. Unlike as under tonic stimulation, the memories are not attractors. After one association process, the network activity moves to other states. Applying in close succession pulses addressing different memories, one can switch through the space of memory patterns. The readout speed can be increased up to the point where in every gamma cycle another pattern is displayed. With pulsed stimulation. bursts become relevant for coding, their occurrence can be used to discriminate relevant processes from background activity.

Action Potentials↗
PubMed2005

A new image representation algorithm inspired by image submodality models, redundancy reduction, and learning in biological vision.

We develop a new biologically motivated algorithm for representing natural images using successive projections into complementary subspaces. An image is first projected into an edge subspace spanned using an ICA basis adapted to natural images which captures the sharp features of an image like edges and curves. The residual image obtained after extraction of the sharp image features is approximated using a mixture of probabilistic principal component analyzers (MPPCA) model. The model is consistent with cellular, functional, information theoretic, and learning paradigms in visual pathway modeling. We demonstrate the efficiency of our model for representing different attributes of natural images like color and luminance. We compare the performance of our model in terms of quality of representation against commonly used basis, like the discrete cosine transform (DCT), independent component analysis (ICA), and principal components analysis (PCA), based on their entropies. Chrominance and luminance components of images are represented using codes having lower entropy than DCT, ICA, or PCA for similar visual quality. The model attains considerable simplification for learning from images by using a sparse independent code for representing edges and explicitly evaluating probabilities in the residual subspace.

Algorithms↗
PubMed2001

On the function of dendritic spines.

Dendritic spines occupy a strategic position in the central nervous system, yet their function is still under debate. Over the past decades, many hypotheses have been put forward to explain the specific function of spines. Recently, imaging experiments have demonstrated that spines compartmentalize calcium, a role that appears necessary for input-specific forms of synaptic plasticity. In addition, it has been discovered that spine morphology is plastic over fast time scales and can be controlled by specific biochemical pathways. Also, several aspects of the spine's morphology appear to be intricately linked to its function. The authors review these recent data and incorporate them into a model for the function of dendritic spines in CNS circuits. In their proposal, spines serve to specifically connect sparse inputs and therefore minimize the wiring necessary in the CNS while maximizing connectivity. By virtue of the same design, spines isolate inputs and thus implement local learning rules. These rules appear only necessary with sparse inputs so these two functions are intimately related. Spines therefore would play a crucial circuit role, remarkably analogous to synaptic matrix elements of associative neural networks. This model highlights the economical, yet elegant, design of CNS circuits.

Animals↗
PubMed1992

From basis functions to basis fields: vector field approximation from sparse data.

Recent investigations (Poggio and Girosi 1990b) have pointed out the equivalence between a wide class of learning problems and the reconstruction of a real-valued function from a sparse set of data. However, in order to process sensory information and to generate purposeful actions living organisms must deal not only with real-valued functions but also with vector-valued mappings. Examples of such vector-valued mappings range from the optical flow fields associated with visual motion to the fields of mechanical forces produced by neuromuscular activation. In this paper, I discuss the issue of vector-field processing from a broad computational perspective. A variety of vector patterns can be efficiently represented by a combination of linearly independent vector fields that I call "basis fields". Basis fields offer in some cases a better alternative to treating each component of a vector as an independent scalar entity. In spite of its apparent simplicity, such a component-based representation is bound to change with any change of coordinates. In contrast, vector-valued primitives such as basis fields generate vector field representations that are invariant under coordinate transformations.

Animals↗