SEARCH · Search PubMed
Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
We introduce a new learning algorithm for kernel-based topographic map formation. The algorithm generates a gaussian mixture density model by individually adapting the gaussian kernels' centers and radii to the assumed gaussian local input densities.
Schizophrenia is characterized by cerebral ventricular enlargement as an early and consistent structural anomaly. While genetic factors significantly influence both schizophrenia and cerebral ventricular enlargement, the shared genetic etiology between them requires further investigation. Using summary statistics from recent large genome-wide association studies on schizophrenia and 9 ventricular subregion volumes phenotypes. Gaussian causal mixture modeling was applied to characterize the genetic architecture and overlap between schizophrenia and ventricular subregion volumes phenotypes. Local genetic correlation was investigated with Local Analysis of Variant Association. The conjunctional false discovery rate framework was used to identify the specific shared genetic loci, annotated with FUMA. Gaussian causal mixture modeling estimated schizophrenia to be more polygenic more polygenic (9574 trait-influencing variants) than ventricular subregion volumes phenotypes (157-1267 trait-influencing variants). Conjunctional false discovery rate analysis identified 42 shared genetic loci, 17 loci were identified as novel for both schizophrenia and the ventricular subregion volumes phenotypes. Local Analysis of Variant Association revealed that 11 distinct loci demonstrated significant differences, among which 4 loci were situated in the Major Histocompatibility Complex region. Annotated genes in shared loci were enriched in molecular signaling pathways involved in inflammation and the brain structure. The shared loci between them were annotated and enriched in Major Histocompatibility Complex and inflammation-related pathways, highlighting new opportunities for future investigation.
This paper presents two new methods for robust parameter estimation of mixtures in the context of magnetic resonance (MR) data segmentation. The head is constituted of different types of tissue that can be modeled by a finite mixture of multivariate Gaussian distributions. Our goal is to estimate accurately the statistics of desired tissues in presence of other ones of lesser interest. These latter can be considered as outliers and can severely bias the estimates of the former. For this purpose, we introduce a first method, which is an extension of the expectation-maximization (EM) algorithm, that estimates parameters of Gaussian mixtures but incorporates an outlier rejection scheme which allows to compute the properties of the desired tissues in presence of atypical data. The second method is based on genetic algorithms and is well suited for estimating the parameters of mixtures of different kind of distributions. We use this property by adding a uniform distribution to the Gaussian mixture for modeling the outliers. The proposed genetic algorithm can efficiently estimate the parameters of this extended mixture for various initial settings. Also, by changing the minimization criterion, estimates of the parameters can be obtained by histogram fitting which considerably reduces the computational cost. Experiments on synthetic and real MR data show that accurate estimates of the gray and white matters parameters are computed.
Factor analysis, principal component analysis, mixtures of gaussian clusters, vector quantization, Kalman filter models, and hidden Markov models can all be unified as variations of unsupervised learning under a single basic generative model. This is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity. Through the use of other nonlinearities, we show how independent component analysis is also a variation of the same basic generative model. We show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term. We introduce a new model for static data, known as sensible principal component analysis, as well as a novel concept of spatially adaptive observation noise. We also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models.
The Generative Topographic Mapping (GTM) was originally conceived as a probabilistic alternative to the well-known, neural network-inspired, Self-Organizing Maps. The GTM can also be interpreted as a constrained mixture of distribution models. In recent years, much attention has been directed towards Student t-distributions as an alternative to Gaussians in mixture models due to their robustness towards outliers. In this paper, the GTM is redefined as a constrained mixture of t-distributions: the t-GTM, and the Expectation-Maximization algorithm that is used to fit the model to the data is modified to carry out missing data imputation. Several experiments show that the t-GTM successfully detects outliers, while minimizing their impact on the estimation of the model parameters. It is also shown that the t-GTM provides an overall more accurate imputation of missing values than the standard Gaussian GTM.
After introducing the fundamentals of BYY system and harmony learning, which has been developed in past several years as a unified statistical framework for parameter learning, regularization and model selection, we systematically discuss this BYY harmony learning on systems with discrete inner-representations. First, we shown that one special case leads to unsupervised learning on Gaussian mixture. We show how harmony learning not only leads us to the EM algorithm for maximum likelihood (ML) learning and the corresponding extended KMEAN algorithms for Mahalanobis clustering with criteria for selecting the number of Gaussians or clusters, but also provides us two new regularization techniques and a unified scheme that includes the previous rival penalized competitive learning (RPCL) as well as its various variants and extensions that performs model selection automatically during parameter learning. Moreover, as a by-product, we also get a new approach for determining a set of 'supporting vectors' for Parzen window density estimation. Second, we shown that other special cases lead to three typical supervised learning models with several new results. On three layer net, we get (i) a new regularized ML learning, (ii) a new criterion for selecting the number of hidden units, and (iii) a family of EM-like algorithms that combines harmony learning with new techniques of regularization. On the original and alternative models of mixture-of-expert (ME) as well as radial basis function (RBF) nets, we get not only a new type of criteria for selecting the number of experts or basis functions but also a new type of the EM-like algorithms that combines regularization techniques and RPCL learning for parameter learning with either least complexity nature on the original ME model or automated model selection on the alternative ME model and RBF nets. Moreover, all the results for the alternative ME model are also applied to other two popular nonparametric statistical approaches, namely kernel regression and supporting vector machine. Particularly, not only we get an easily implemented approach for determining the smoothing parameter in kernel regression, but also we get an alternative approach for deciding the set of supporting vectors in supporting vector machine.
There has been growing interest in subspace data modeling over the past few years. Methods such as principal component analysis, factor analysis, and independent component analysis have gained in popularity and have found many applications in image modeling, signal processing, and data compression, to name just a few. As applications and computing power grow, more and more sophisticated analyses and meaningful representations are sought. Mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds. Meaningful representations may be lost, however, if these local features are nongaussian or discontinuous. In this article, we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model. We employ recent developments in variational Bayesian inference and structure determination to construct a novel approach for modeling nongaussian, discontinuous manifolds. We automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ICA components needed in our mixture model. We demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful-and medically useful-features.
MOTIVATION: Identifying patterns of co-expression in microarray data by cluster analysis has been a productive approach to uncovering molecular mechanisms underlying biological processes under investigation. Using experimental replicates can generally improve the precision of the cluster analysis by reducing the experimental variability of measurements. In such situations, Bayesian mixtures allow for an efficient use of information by precisely modeling between-replicates variability. RESULTS: We developed different variants of Bayesian mixture based clustering procedures for clustering gene expression data with experimental replicates. In this approach, the statistical distribution of microarray data is described by a Bayesian mixture model. Clusters of co-expressed genes are created from the posterior distribution of clusterings, which is estimated by a Gibbs sampler. We define infinite and finite Bayesian mixture models with different between-replicates variance structures and investigate their utility by analyzing synthetic and the real-world datasets. Results of our analyses demonstrate that (1) improvements in precision achieved by performing only two experimental replicates can be dramatic when the between-replicates variability is high, (2) precise modeling of intra-gene variability is important for accurate identification of co-expressed genes and (3) the infinite mixture model with the 'elliptical' between-replicates variance structure performed overall better than any other method tested. We also introduce a heuristic modification to the Gibbs sampler based on the 'reverse annealing' principle. This modification effectively overcomes the tendency of the Gibbs sampler to converge to different modes of the posterior distribution when started from different initial positions. Finally, we demonstrate that the Bayesian infinite mixture model with 'elliptical' variance structure is capable of identifying the underlying structure of the data without knowing the 'correct' number of clusters. AVAILABILITY: The MS Windows based program named Gaussian Infinite Mixture Modeling (GIMM) implementing the Gibbs sampler and corresponding C++ code are available at http://homepages.uc.edu/~medvedm/GIMM.htm SUPPLEMENTAL INFORMATION: http://expression.microslu.washington.edu/expression/kayee/medvedovic2003/medvedovic_bioinf2003.html
Neural spike sorting is an indispensable step in the analysis of multiunit extracellular neural signal recording. The applicability of spike sorting systems has been limited, mainly to the recording of sufficiently high signal-to-noise ratios, or to the cases where supervised classification can be utilized. We present a novel unsupervised method that shows satisfactory performance even under high background noise. The system consists of an efficient spike detector, a feature extractor that utilizes projection pursuit based on negentropy maximization (Huber, 1985 and Hyvarinen et al, 1999), and an unsupervised classifier based on probability density modeling using mixture of Gaussians (Jain et al., 2000). Our classifier is based on the mixture model with a roughly approximated number of Gaussians and subsequent mode-seeking. It does not require accurate estimation of the number of units present in the recording and, thus, is better suited for use in fully automated systems. The feature extraction stage leads to better performance than those utilizing principal component analysis and two nonlinear mappings for the recordings from the somatosensory cortex of rat and the abdominal ganglion of Aplysia. The classification method yielded correct classification ratio as high as 95%, for data where it was only 66% when a kappa-means-type algorithm was used for the classification stage.
This paper presents a statistical approach to aggregating speed and phase (directional) information for vascular segmentation of phase contrast magnetic resonance angiograms (PC-MRA). Rather than relying on speed information alone, as done by others and in our own work, we demonstrate that including phase information as a priori knowledge in a Markov random field (MRF) model can improve the quality of segmentation. This is particularly true in the region within an aneurysm where there is a heterogeneous intensity pattern and significant vascular signal loss. We propose to use a Maxwell-Gaussian mixture density to model the background signal distribution and combine this with a uniform distribution for modelling vascular signal to give a Maxwell-Gaussian-uniform (MGU) mixture model of image intensity. The MGU model parameters are estimated by the modified expectation-maximisation (EM) algorithm. In addition, it is shown that the Maxwell-Gaussian mixture distribution (a) models the background signal more accurately than a Maxwell distribution, (b) exhibits a better fit to clinical data and (c) gives fewer false positive voxels (misclassified vessel voxels) in segmentation. The new segmentation algorithm is tested on an aneurysm phantom data set and two clinical data sets. The experimental results show that the proposed method can provide a better quality of segmentation when both speed and phase information are utilised.
The contourlet transform is a new two-dimensional extension of the wavelet transform using multiscale and directional filter banks. The contourlet expansion is composed of basis images oriented at various directions in multiple scales, with flexible aspect ratios. Given this rich set of basis images, the contourlet transform effectively captures smooth contours that are the dominant feature in natural images. We begin with a detailed study on the statistics of the contourlet coefficients of natural images: using histograms to estimate the marginal and joint distributions and mutual information to measure the dependencies between coefficients. This study reveals the highly non-Gaussian marginal statistics and strong interlocation, interscale, and interdirection dependencies of contourlet coefficients. We also find that conditioned on the magnitudes of their generalized neighborhood coefficients, contourlet coefficients can be approximately modeled as Gaussian random variables. Based on these findings, we model contourlet coefficients using a hidden Markov tree (HMT) model with Gaussian mixtures that can capture all interscale, interdirection, and interlocation dependencies. We present experimental results using this model in image denoising and texture retrieval applications. In denoising, the contourlet HMT outperforms other wavelet methods in terms of visual quality, especially around edges. In texture retrieval, it shows improvements in performance for various oriented textures.
We present an energy-based model that uses a product of generalized Student-t distributions to capture the statistical structure in data sets. This model is inspired by and particularly applicable to "natural" data sets such as images. We begin by providing the mathematical framework, where we discuss complete and overcomplete models and provide algorithms for training these models from data. Using patches of natural scenes, we demonstrate that our approach represents a viable alternative to independent component analysis as an interpretive model of biological visual systems. Although the two approaches are similar in flavor, there are also important differences, particularly when the representations are overcomplete. By constraining the interactions within our model, we are also able to study the topographic organization of Gabor-like receptive fields that our model learns. Finally, we discuss the relation of our new approach to previous work--in particular, gaussian scale mixture models and variants of independent components analysis.
The appearance of disproportionately large amounts of high-density breast parenchyma in mammograms has been found to be a strong indicator of the risk of developing breast cancer. Hence, the breast density model is popular for risk estimation or for monitoring breast density change in prevention or intervention programs. However, the efficiency of such a stochastic model depends on the accuracy of estimation of the model's parameter set. We propose a new approach-heuristic optimization-to estimate more accurately the model parameter set as compared to the conventional and popular expectation-maximization (EM) algorithm. After initial segmentation of a given mammogram, the finite generalized Gaussian mixture (FGGM) model is constructed by computing the statistics associated with different image regions. The model parameter set thus obtained is estimated by particle swarm optimization (PSO) and evolutionary programming (EP) techniques, where the objective function to be minimized is the relative entropy between the image histogram and the estimated density distributions. When our heuristic approach was applied to different categories of mammograms from the Mini-MIAS database, it yielded lower floor of estimation error in 109 out of 112 cases (97.3 %), and 101 out of 102 cases (99.0%), for the number of image regions being five and eight, respectively, with the added advantage of faster convergence rate, when compared to the EM approach. Besides, the estimated density model preserves the number of regions specified by the information-theoretic criteria in all the test cases, and the assessment of the segmentation results by radiologists is promising.
We present a switching Kalman filter model for the real-time inference of hand kinematics from a population of motor cortical neurons. Firing rates are modeled as a Gaussian mixture where the mean of each Gaussian component is a linear function of hand kinematics. A "hidden state" models the probability of each mixture component and evolves over time in a Markov chain. The model generalizes previous encoding and decoding methods, addresses the non-Gaussian nature of firing rates, and can cope with crudely sorted neural data common in on-line prosthetic applications.
Given the limited feasibility of population-wide bone mineral density screening and the infrequency of long-term monitoring in healthy individuals, identifying the window for early intervention and the populations to be prioritized for screening is critical. This study aimed to identify intervention windows for osteoporosis and to determine potential high-risk subtypes within the healthy population. Based on proteomic data from 41,408 healthy adults, we conducted the DE-SWAN method to identify change peaks in plasma protein during the pre-diagnostic osteoporosis phase, and employed finite Gaussian mixture model-based clustering to delineate high-risk subtypes of osteoporosis. We identified 122 protein biomarkers significantly associated with osteoporosis risk throughout the follow-up period. Importantly, we identified two critical peaks occurring approximately 10 and 6 years before diagnosis, with the former enriched in immune-related pathways and the latter prominently involving responses to retinoic acid and glucocorticoids. Furthermore, one high-risk subtype for osteoporosis was identified in both males and females, termed the Frailty and Obesity Subtype. This subtype is characterized by a high degree of frailty and obesity, accompanied by a significantly elevated risk of both osteoporosis and fractures. Finally, we developed a predictive model comprising 10 proteins for identifying high-risk subtypes of osteoporosis, which demonstrated better performance than the traditional risk factor model (AUC: 0.743 vs. 0.680). Our findings demonstrate that proteomic profiling can reveal early molecular changes and identify high-risk subtypes years before clinical onset, providing a foundation for screening and precision prevention of osteoporosis.
Dental biometrics utilizes dental radiographs for human identification. The dental radiographs provide information about teeth, including tooth contours, elative positions of neighboring teeth, and shapes of the dental work (e.g., crowns, fillings, and bridges). The proposed system has two main stages: feature extraction and matching. The feature extraction stage uses anisotropic diffusion to enhance the images and a Mixture of Gaussians model to segment the dental work. The matching stage has three sequential steps: tooth-level matching, computation of image distances, and subject identification. In the tooth-level matching step, tooth contours are matched using a shape registration method, and the dental work is matched on overlapping areas. The distance between the tooth contours and the distance between the dental work are then combined using posterior probabilities. In the second step, the tooth correspondences between the given query (postmortem) radiograph and the database (antemortem) radiograph are established. A distance based on the corresponding teeth is then used to measure the similarity between the two radiographs. Finally, all the distances between the given postmortem radiographs and the antemortem radiographs that provide candidate identities are combined to establish the identity of the subject associated with the postmortem radiographs.
This paper considers fitting a mixture of Gaussians model to high-dimensional data in scenarios where there are fewer data samples than feature dimensions. Issues that arise when using principal component analysis (PCA) to represent Gaussian distributions inside Expectation-Maximization (EM) are addressed, and a practical algorithm results. Unlike other algorithms that have been proposed, this algorithm does not try to compress the data to fit low-dimensional models. Instead, it models Gaussian distributions in the (N - 1)-dimensional space spanned by the N data samples. We are able to show that this algorithm converges on data sets where low-dimensional techniques do not.
A new learning algorithm for kernel-based topographic map formation is introduced. The kernel parameters are adjusted individually so as to maximize the joint entropy of the kernel outputs. This is done by maximizing the differential entropies of the individual kernel outputs, given that the map's output redundancy, due to the kernel overlap, needs to be minimized. The latter is achieved by minimizing the mutual information between the kernel outputs. As a kernel, the (radial) incomplete gamma distribution is taken since, for a gaussian input density, the differential entropy of the kernel output will be maximal. Since the theoretically optimal joint entropy performance can be derived for the case of nonoverlapping gaussian mixture densities, a new clustering algorithm is suggested that uses this optimum as its "null" distribution. Finally, it is shown that the learning algorithm is similar to one that performs stochastic gradient descent on the Kullback-Leibler divergence for a heteroskedastic gaussian mixture density model.