Biomedical subjects

Lawrence Carin

Publications and source records attributed to Lawrence Carin.

10 recordsLinked to original sources

PubMed2007

On classification with incomplete data.

We address the incomplete-data problem in which feature vectors to be classified are missing data (features). A (supervised) logistic regression algorithm for the classification of incomplete data is developed. Single or multiple imputation for the missing data is avoided by performing analytic integration with an estimated conditional density function (conditioned on the observed data). Conditional density functions are estimated using a Gaussian mixture model (GMM), with parameter estimation performed using both Expectation-Maximization (EM) and Variational Bayesian EM (VB-EM). The proposed supervised algorithm is then extended to the semisupervised case by incorporating graph-based regularization. The semisupervised algorithm utilizes all available data-both incomplete and complete, as well as labeled and unlabeled. Experimental results of the proposed classification algorithms are shown.

Algorithms↗

PubMed2006

A modified SPIHT algorithm for image coding with a joint MSE and classification distortion measure.

The set partitioning in hierarchical trees (SPIHT) algorithm is an efficient wavelet-based progressive image-compression technique, designed to minimize the mean-squared error (MSE) between the original and decoded imagery. However, the MSE-based distortion measure is not in general well correlated with image-recognition quality, especially at low bit rates. Specifically, low-amplitude wavelet coefficients that may be important for classification are given low priority by conventional SPIHT. In this paper, we use the kernel matching pursuits (KMP) method to autonomously estimate the importance of each wavelet subband for distinguishing between different textures, with textural segmentation first performed via a hidden Markov tree. Based on subband importance determined via KMP, we scale the wavelet coefficients prior to SPIHT coding, with the goal of minimizing a Lagrangian distortion based jointly on the MSE and classification error. For comparison we consider Bayes tree-structured vector quantization (B-TSVQ), also designed to obtain a tradeoff between MSE and classification error. The performances of the original SPIHT, the modified SPIHT, and B-TSVQ are compared.

Algorithms↗

PubMed2006

Variational Bayes for continuous hidden Markov models and its application to active learning.

In this paper, we present a varitional Bayes (VB) framework for learning continuous hidden Markov models (CHMMs), and we examine the VB framework within active learning. Unlike a maximum likelihood or maximum a posteriori training procedure, which yield a point estimate of the CHMM parameters, VB-based training yields an estimate of the full posterior of the model parameters. This is particularly important for small training sets since it gives a measure of confidence in the accuracy of the learned model. This is utilized within the context of active learning, for which we acquire labels for those feature vectors for which knowledge of the associated label would be most informative for reducing model-parameter uncertainty. Three active learning algorithms are considered in this paper: 1) query by committee (QBC), with the goal of selecting data for labeling that minimize the classification variance, 2) a maximum expected information gain method that seeks to label data with the goal of reducing the entropy of the model parameters, and 3) an error-reduction-based procedure that attempts to minimize classification error over the test data. The experimental results are presented for synthetic and measured data. We demonstrate that all of these active learning methods can significantly reduce the amount of required labeling, compared to random selection of samples for labeling.

Algorithms↗

PubMed2005

Sparse multinomial logistic regression: fast algorithms and generalization bounds.

Recently developed methods for learning sparse classifiers are among the state-of-the-art in supervised learning. These methods learn classifiers that incorporate weighted sums of basis functions with sparsity-promoting priors encouraging the weight estimates to be either significantly large or exactly zero. From a learning-theoretic perspective, these methods control the capacity of the learned classifier by minimizing the number of basis functions used, resulting in better generalization. This paper presents three contributions related to learning sparse classifiers. First, we introduce a true multiclass formulation based on multinomial logistic regression. Second, by combining a bound optimization approach with a component-wise update procedure, we derive fast exact algorithms for learning sparse multiclass classifiers that scale favorably in both the number of training samples and the feature dimensionality, making them applicable even to large data sets in high-dimensional feature spaces. To the best of our knowledge, these are the first algorithms to perform exact multinomial logistic regression with a sparsity-promoting prior. Third, we show how nontrivial generalization bounds can be derived for our classifier in the binary case. Experimental results on standard benchmark data sets attest to the accuracy, sparsity, and efficiency of the proposed methods.

Algorithms↗

PubMed2005

Time-reversal imaging for classification of submerged elastic targets via Gibbs sampling and the Relevance Vector Machine.

Time-reversal imaging (TRI) is analogous to matched-field processing, although TRI is typically very wideband and is appropriate for subsequent target classification (in addition to localization). Time-reversal techniques, as applied to acoustic target classification, are highly sensitive to channel mismatch. Hence, it is crucial to estimate the channel parameters before time-reversal imaging is performed. The channel-parameter statistics are estimated here by applying a geoacoustic inversion technique based on Gibbs sampling. The maximum a posteriori (MAP) estimate of the channel parameters are then used to perform time-reversal imaging. Time-reversal implementation requires a fast forward model, implemented here by a normal-mode framework. In addition to imaging, extraction of features from the time-reversed images is explored, with these applied to subsequent target classification. The classification of time-reversed signatures is performed by the relevance vector machine (RVM). The efficacy of the technique is analyzed on simulated in-channel data generated by a free-field finite element method (FEM) code, in conjunction with a channel propagation model, wherein the final classification performance is demonstrated to be relatively insensitive to the associated channel parameters. The underlying theory of Gibbs sampling and TRI are presented along with the feature extraction and target classification via the RVM.

Journal Article↗

PubMed2004

Joint classifier and feature optimization for comprehensive cancer diagnosis using gene expression data.

Recent research has demonstrated quite convincingly that accurate cancer diagnosis can be achieved by constructing classifiers that are designed to compare the gene expression profile of a tissue of unknown cancer status to a database of stored expression profiles from tissues of known cancer status. This paper introduces the JCFO, a novel algorithm that uses a sparse Bayesian approach to jointly identify both the optimal nonlinear classifier for diagnosis and the optimal set of genes on which to base that diagnosis. We show that the diagnostic classification accuracy of the proposed algorithm is superior to a number of current state-of-the-art methods in a full leave-one-out cross-validation study of five widely used benchmark datasets. In addition to its superior classification accuracy, the algorithm is designed to automatically identify a small subset of genes (typically around twenty in our experiments) that are capable of providing complete discriminatory information for diagnosis. Focusing attention on a small subset of genes is useful not only because it produces a classifier with good generalization capacity, but also because this set of genes may provide insights into the mechanisms responsible for the disease itself. A number of the genes identified by the JCFO in our experiments are already in use as clinical markers for cancer diagnosis; some of the remaining genes may be excellent candidates for further clinical investigation. If it is possible to identify a small set of genes that is indeed capable of providing complete discrimination, inexpensive diagnostic assays might be widely deployable in clinical settings.

Algorithms↗

PubMed2004

Application of the theory of optimal experiments to adaptive electromagnetic-induction sensing of buried targets.

A mobile electromagnetic-induction (EMI) sensor is considered for detection and characterization of buried conducting and/or ferrous targets. The sensor may be placed on a robot and, here, we consider design of an optimal adaptive-search strategy. A frequency-dependent magnetic-dipole model is used to characterize the target at EMI frequencies. The goal of the search is accurate characterization of the dipole-model parameters, denoted bythe vector theta; the target position and orientation are a subset of theta. The sensor position and operating frequency are denoted by the parameter vector p and a measurement is represented by the pair (p, O), where O denotes the observed data. The parametersp are fixed for a given measurement, but, in the context of a sequence of measurements p may be changed adaptively. In a locally optimal sequence of measurements, we desire the optimal sensor parameters, P(N+1) for estimation of theta, based on the previous measurements (p(n), On)n=1,N. The search strategy is based on the theory of optimal experiments, as discussed in detail and demonstrated via several numerical examples.

Algorithms↗

PubMed2004

A Bayesian approach to joint feature selection and classifier design.

This paper adopts a Bayesian approach to simultaneously learn both an optimal nonlinear classifier and a subset of predictor variables (or features) that are most relevant to the classification task. The approach uses heavy-tailed priors to promote sparsity in the utilization of both basis functions and features; these priors act as regularizers for the likelihood function that rewards good classification on the training data. We derive an expectation-maximization (EM) algorithm to efficiently compute a maximum a posteriori (MAP) point estimate of the various parameters. The algorithm is an extension of recent state-of-the-art sparse Bayesian classifiers, which in turn can be seen as Bayesian counterparts of support vector machines. Experimental comparisons using kernel classifiers demonstrate both parsimonious feature selection and excellent classification accuracy on a range of synthetic and benchmark data sets.

Algorithms↗

PubMed2004

Wideband time-reversal imaging of an elastic target in an acoustic waveguide.

Time-reversal is addressed for imaging elastic targets situated in an acoustic waveguide. It is assumed that the target-sensor range is large relative to the channel depth. We investigate the theory of wideband time-reversal imaging of an extended elastic target, for which the target dimensions are large relative to the principal wavelengths. When performing time-reversal imaging one requires a forward model for propagation through the channel, and the quality of the resulting image may be used as a measure of the match between the modeled and actual (measured) channel parameters. It is demonstrated that the channel parameters associated with a given measurement may be determined via a genetic-algorithm (GA) search in parameter space, employing a cost function based on the time-reversal image quality. Example GA channel-parameter-inversion results and imagery are presented for measured at-sea data.

Journal Article↗

PubMed2003

Parallel implementation of the biorthogonal multiresolution time-domain method.

The three-dimensional biorthogonal multiresolution time-domain (Bi-MRTD) method is presented for both free-space and half-space scattering problems. The perfectly matched layer (PML) is used as an absorbing boundary condition. It has been shown that improved numerical-dispersion properties can be obtained with the use of smooth, compactly supported wavelet functions as the basis, whereas we employ the Cohen-Daubechies-Fouveau (CDF) biorthogonal wavelets. When a CDF-wavelet expansion is used, the spatial-sampling rate can be reduced considerably compared with that of the conventional finite-difference time-domain (FDTD) method, implying that larger targets can be simulated without sacrificing accuracy. We implement the Bi-MRTD on a cluster of allocated-memory machines, using the message-passing interface (MPI), such that very large targets can be modeled. Numerical results are compared with analytical ones and with those obtained by use of the traditional FDTD method.

Journal Article↗