Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “mutual information”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources
PubMed2004

Estimating mutual information.

We present two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y). In contrast to conventional estimators based on binnings, they are based on entropy estimates from k -nearest neighbor distances. This means that they are data efficient (with k=1 we resolve structures down to the smallest possible scales), adaptive (the resolution is higher where data are more numerous), and have minimal bias. Indeed, the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale, giving typically systematic errors which scale as functions of k/N for N points. Numerically, we find that both families become exact for independent distributions, i.e. the estimator M(X,Y) vanishes (up to statistical fluctuations) if mu(x,y)=mu(x)mu(y). This holds for all tested marginal distributions and for all dimensions of x and y. In addition, we give estimators for redundancies between more than two random variables. We compare our algorithms in detail with existing algorithms. Finally, we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis (ICA), for improving ICA, and for estimating the reliability of blind source separation.

Journal Article↗
PubMed2000

Species independence of mutual information in coding and noncoding DNA.

We explore if there exist universal statistical patterns that are different in coding and noncoding DNA and can be found in all living organisms, regardless of their phylogenetic origin. We find that (i) the mutual information function [symbol: see text] has a significantly different functional form in coding and noncoding DNA. We further find that (ii) the probability distributions of the average mutual information [symbol: see text] are significantly different in coding and noncoding DNA, while (iii) they are almost the same for organisms of all taxonomic classes. Surprisingly, we find that [symbol: see text] is capable of predicting coding regions as accurately as organism-specific coding measures.

DNA↗
PubMed1996

Radial Basis Function Network Configuration Using Mutual Information and the Orthogonal Least Squares Algorithm.

Input nodes of neural networks are usually predetermined by using a priori knowledge or selected by trial and error. For example, in pattern recognition applications the input nodes are usually the given pattern features and in system identification applications the past input and output data are often used as inputs to the network. Some of the input variables may be irrelevant to the task in hand and therefore may cause a deterioration in network performance. Some may be redundant and may increase the complexity of the network and consume expensive computation time. In the present study, the mutual information between the input variables and the output of the network is used to select a suboptimal set of input variables for the network. The variables are selected according to the information content relevant to the output. Variables which have a higher mutual information with the output and lower dependence on other selected variables are used as network inputs. The algorithms are derived based on heuristics and performance is assessed by using radial basis function (RBF) networks trained with the orthogonal least squares algorithm (OLS), which selects the hidden layer nodes of the network according to the error reduction ratios on the network output. Both real and simulated data sets are used to demonstrate the effectiveness of the new algorithms. Copyright 1996 Elsevier Science Ltd.

Journal Article↗
PubMed2005

Predicting RNA structure using mutual information.

BACKGROUND: With the ever-increasing number of sequenced RNAs and the establishment of new RNA databases, such as the Comparative RNA Web Site and Rfam, there is a growing need for accurately and automatically predicting RNA structures from multiple alignments. Since RNA secondary structure is often conserved in evolution, the well known, but underused, mutual information measure for identifying covarying sites in an alignment can be useful for identifying structural elements. This article presents MIfold, a MATLAB toolbox that employs mutual information, or a related covariation measure, to display and predict conserved RNA secondary structure (including pseudoknots) from an alignment. RESULTS: We show that MIfold can be used to predict simple pseudoknots, and that the performance can be adjusted to make it either more sensitive or more selective. We also demonstrate that the overall performance of MIfold improves with the number of aligned sequences for certain types of RNA sequences. In addition, we show that, for these sequences, MIfold is more sensitive but less selective than the related RNAalifold structure prediction program and is comparable with the COVE structure prediction package. CONCLUSION: MIfold provides a useful supplementary tool to programs such as RNA Structure Logo, RNAalifold and COVE, and should be useful for automatically generating structural predictions for databases such as Rfam.

Algorithms↗
PubMed2001

Mutual information analysis as a tool to assess the role of aneuploidy in the generation of cancer-associated differential gene expression patterns.

Most human tumors are characterized by: (1) an aberrant set of chromosomes, a state termed aneuploidy; (2) an aberrant gene expression pattern; and (3) an aberrant phenotype of uncontrolled growth. One of the goals of cancer research is to establish causative relationships between these three important characteristics. In this paper we were searching for evidence that aneuploidy is a major cause of differential gene expression. We describe how mutual information analysis of cancer-associated gene expression patterns could be exploited to answer this question. In addition to providing general guidelines, we have applied the proposed analysis to a recently published breast cancer-associated gene expression matrix. The results derived from this particular data set provided preliminary evidence that mutual information analysis may become a useful tool to investigate the link between differential gene expression and aneuploidy.

Aneuploidy↗
PubMed2002

PET-MR image fusion in soft tissue sarcoma: accuracy, reliability and practicality of interactive point-based and automated mutual information techniques.

The fusion of functional positron emission tomography (PET) data with anatomical magnetic resonance (MR) or computed tomography images, using a variety of interactive and automated techniques, is becoming commonplace, with the technique of choice dependent on the specific application. The case of PET-MR image fusion in soft tissue is complicated by a lack of conspicuous anatomical features and deviation from the rigid-body model. Here we compare a point-based external marker technique with an automated mutual information algorithm and discuss the practicality, reliability and accuracy of each when applied to the study of soft tissue sarcoma. Ten subjects with suspected sarcoma in the knee, thigh, groin, flank or back underwent MR and PET scanning after the attachment of nine external fiducial markers. In the assessment of the point-based technique, three error measures were considered: fiducial localisation error (FLE), fiducial registration error (FRE) and target registration error (TRE). FLE, which represents the accuracy with which the fiducial points can be located, is related to the FRE minimised by the registration algorithm. The registration accuracy is best characterised by the TRE, which is the distance between corresponding points in each image space after registration. In the absence of salient features within the target volume, the TRE can be measured at fiducials excluded from the registration process. To assess the mutual information technique, PET data, acquired after physically removing the markers, were reconstructed in a variety of ways and registered with MR. Having applied the transform suggested by the algorithm to the PET scan acquired before the markers were removed, the residual distance between PET and MR marker-pairs could be measured. The manual point-based technique yielded the best results (RMS TRE =8.3 mm, max =22.4 mm, min =1.7 mm), performing better than the automated algorithm (RMS TRE =20.0 mm, max =30.5 mm, min =7.7 mm) when registering filtered back-projection PET images to MR. Image reconstruction with an iterative algorithm or registration of a composite emission-transmission image did not improve the overall accuracy of the registration process. We have demonstrated that, in this application, point-based PET-MR registration using external markers is practical, reliable and accurate to within approximately 5 mm towards the fiducial centroid. The automated algorithm did not perform as reliably or as accurately.

Fluorodeoxyglucose F18↗
PubMed2003

Mutual information and the homeopathic effect.

We explore the feasibility of using mutual information to characterize the homeopathic effect. This quantity measures the information gained about a signal at time (t + tau), from its value at an earlier time t; it quantifies the predictability of data. We illustrate our method with an analysis of the homeopathic effect of Strophanthus hispidus on the cardiac rhythm of healthy human subjects, using data from a previous experiment. Our results allow an intuitively clear rendering and agree with the similitude principle applied to this case. They also show that the solvent has a significant effect on the signal; hence, it does not act as an ideal placebo and we discuss some therapeutic corollaries to this observation.

Feasibility Studies↗
PubMedDate not supplied

A methodology for registration of a histological slide and in vivo MRI volume based on optimizing mutual information.

We present a method for registering histology and in vivo imaging that requires minimal microtoming and is automatic following the user's initialization. In this demonstration, we register a single hematoxylin-and-eosin-stained histological slide of a coronal section of a rat brain harboring a 9L gliosarcoma with an in vivo 7T MR image volume of the same brain. Because the spatial resolution of the in vivo MRI is limited, we add the step of obtaining a high spatial resolution, ex vivo MRI in situ for intermediate registration. The approach taken was to maximize mutual information in order to optimize the registration between all pairings of image data whether the sources are MRI, tissue block photograph, or stained sample photograph. The warping interpolant used was thin plate splines with the appropriate basis function for either 2-D or 3-D applications. All registrations were implemented by user initialization of the approximate pose between the two data sets, followed by automatic optimization based on maximizing mutual information. Only the higher quality anatomical images were used in the registration process; however, the spatial transformation was directly applied to a quantitative diffusion image. Quantitative diffusion maps from the registered location appeared highly correlated with the H&E slide. Overall, this approach provides a robust method for coregistration of in vivo images with histological sections and will have broad applications in the field of functional and molecular imaging.

Animals↗
PubMed2002

A method for spike sorting and detection based on wavelet packets and Shannon's mutual information.

Studying the dynamics of neural activity via electrical recording, relies on the ability to detect and sort neural spikes recorded from a number of neurons by the same electrode. We suggest the wavelet packets decomposition (WPD) as a tool to analyze neural spikes and extract their main features. The unique quality of the wavelet packets-adaptive coverage of both time and frequency domains using a set of localized packets, facilitate the task. The best basis algorithm utilizing the Shannon's information cost function and local discriminant basis (LDB) using mutual information are employed to select a few packets that are sufficient for both detection and sorting of spikes. The efficiency of the method is demonstrated on data recorded from in vitro 2D neural networks, placed on electrodes that read data from as many as five neurons. Comparison between our method and the widely used principal components method and a sorting technique based on the ordinary wavelet transform (WT) shows that our method is more efficient both in separating spikes from noise and in resolving overlapping spikes.

Action Potentials↗
PubMed1998

Mutual information discloses relationship between hemodynamic variables in artificial heart-implanted dogs.

A mutual information (MI) method for assessment of the relationship between hemodynamic variables was proposed and applied to the analysis of heart rate (HR), arterial blood pressure (BP), and renal sympathetic nerve activity (RSNA) in artificial heart-implanted dogs to quantify correlation between these parameters. MI measures the nonlinear as well as linear dependence of two variables. Simulation studies revealed that this MI technique furnishes mathematical features well suited to the investigation of nonlinear dynamics such as the cardiovascular system and can quantify a relationship between two parameters. To constitute a model free of the natural heart, two pneumatically actuated ventricular assist devices were implanted as biventricular bypasses in acute canine experiments. RSNA was detected with the use of bipolar electrodes attached to the renal sympathetic nerve. Analysis of data during control revealed that correlation between HR and RSNA was higher than that between HR and BP and that between RSNA and BP (P < 0.05). Although RSNA seemed to fluctuate noncorrelatedly with BP in higher pacing rates, the MI values between them disclosed their strong correlation. Surprisingly, correlation between RSNA and BP was stronger during a pacing rate of 60 beats/min than during higher pacing rates and control (P < 0. 05). It is suggested that the baroreflex system may be susceptible to pacing rates during the total artificial heart state. We calculated the time delay between HR and RSNA, between RSNA and BP, and between HR and BP by regarding a time delay at which the maximum MI value between each pair of parameters was given as a physiological delay. Our results indicate that RSNA leads BP, BP leads HR, and RSNA leads HR during control (P < 0.05). We conclude that this method could provide a powerful means for measuring correlation of physiological variables.

Algorithms↗
PubMed1997

Supervised mutual-information based feature selection for motor unit action potential classification.

A new supervised mutual information-based feature selection method is presented. Using real motor unit action potential (MUAP) data from 10 EMG signals, the performances of 32 time-sample feature sets, feature subsets selected using first- and second-order mutual information and features obtained using linear discriminant analysis (LDA) and principal component analysis (PCA) were evaluated using a minimum Euclidean distance (MED) classifier. The evaluation showed that by using only 20 first-order features or only 15 second-order features mean error rates and error rate variations equivalent to using all 32 samples or LDA or PCA could be obtained. The computational cost of first-order feature selection was considerably less than LDA, PCA and second-order feature selection. The performance of first-order features was further evaluated using a more robust classifier. Unlike the MED classifier, the robust classifier only assigned a candidate MUAP if the assignment was sufficiently certain. For the robust classifier the average error rates using 20 features were similar to using the full feature set, yet higher assignment rates were obtained. Results from both evaluations suggest that the sets of first-order features were an efficient representation of lower dimension, which provided high accuracy classification with reduced computational requirements.

Action Potentials↗
PubMed2004

The mutual information theory for the certification of rice coding sequences.

We report here the use of the mutual information theory for the certification of annotated rice coding sequences of both GenBank and TIGR databases. Considering coding sequences larger than 600 bp, we successfully screened out genes with aberrant compositional features. We found that they represent about 10% of both datasets after cleaning for gene redundancy. Most of the rejected accessions showed a different trend in GC3% vs GC2% plot compared to the set of accessions that have been published in international journals. This suggests the existence of a bias in the pattern recognition algorithms used by gene prediction programs.

Information Theory↗
PubMed1998

Mutual information, Fisher information, and population coding.

In the context of parameter estimation and model selection, it is only quite recently that a direct link between the Fisher information and information-theoretic quantities has been exhibited. We give an interpretation of this link within the standard framework of information theory. We show that in the context of population coding, the mutual information between the activity of a large array of neurons and a stimulus to which the neurons are tuned is naturally related to the Fisher information. In the light of this result, we consider the optimization of the tuning curves parameters in the case of neurons responding to a stimulus represented by an angular variable.

Action Potentials↗
PubMed2005

Multilocus LD measure and tagging SNP selection with generalized mutual information.

Linkage disequilibrium (LD) plays a central role in fine mapping of disease genes and, more recently, in characterizing haplotype blocks. Classical LD measures, such as D' and r(2), are frequently used to quantify relationship between two loci. A pairwise "distance" matrix among a set of loci can be constructed using such a measure, and based upon which a number of haplotype block detection and tagging single nucleotide polymorphism (SNP) selection algorithms have been devised. Although successful in many applications, the pairwise nature of these measures does not provide a direct characterization of joint linkage disequilibrium among multiple loci. Consequently, applications based on them may lead to loss of important information. In this report, we propose a multilocus LD measure based on generalized mutual information, which is also known as relative entropy or Kullback-Leibler distance. In essence, this measure seeks to quantify the distance between the observed haplotype distribution and the expected distribution assuming linkage equilibrium. We can show that this measure is approximately equal to r(2) in the special case with two loci. Based on this multilocus LD measure and an entropy measure that characterizes haplotype diversity, we propose a class of stepwise tagging SNP selection algorithms. This represents a unified approach for SNP selection in that it takes into account both the haplotype diversity and linkage disequilibrium objectives. Applications to both simulated and real data demonstrate the utility of the proposed methods for handling a large number of SNPs. The results indicate that multilocus LD patterns can be captured well, and informative and nonredundant SNPs can be selected effectively from a large set of loci.

Algorithms↗
PubMed2003

Prediction of response to incision using the mutual information of electroencephalograms during anaesthesia.

This paper presents a new approach to predict response during isoflurane anaesthesia by using mutual information (MI) time series of electroencephalograms (EEGs) and their complexity analysis. The MI between four lead electrodes was first computed using the EEG time series. The Lempel-Ziv complexity measures, C(n)s, were extracted from the MI time series. Prediction was made by means of artificial neural network (ANN). From 98 consenting patient experiments, 98 distinct EEG recordings were collected prior to incision during isoflurane anaesthesia of different levels. During and after skin incision, each patient was observed carefully for 2 min to detect subsequent responses (purposeful movement, changes in hemodynamic parameters and respiratory pattern) and then the EEG was labelled as 0.0 for responder or as 1.0 for non-responder. Training and testing the ANN used the 'drop-one-patient' method. The prediction was tested by monitoring the response to incision and the result given by the ANN. The system was able to correctly classify purposeful response in average accuracy of 91.84% of the cases. The results showed that the method has a better performance than other methods, such as spectral edge frequency, median frequency, and bispectral analysis. This method is computationally fast and acceptable real-time clinical performance was obtained.

Adult↗
PubMed2005

Adaptive subdivision for hierarchical non-rigid registration of multi-modal images using mutual information.

In this paper we present an enhanced method for non-rigid registration of volumetric multi-modal images using Mutual Information (MI). Based on a hierarchical subdivision scheme, the non-rigid matching problem is decomposed into numerous rigid registrations of sub-images of decreasing size. A thorough investigation revealed limitations of this approach, caused by a peculiar behavior of MI when applied to regions covering only a limited number of image pixels. We examine and explain the loss of MI's statistical consistency along the hierarchical subdivision. We also propose to use information theoretical measures to identify the problematic regions in order to overcome the MI drawbacks. This does not only improve the accuracy and robustness of the registration, but also can be used as a very efficient stopping criterion for the further subdivision of nodes in the hierarchy, which drastically reduces the computational costs of the entire registration procedure.

Algorithms↗
PubMedDate not supplied

Global optimization of mutual information: application to three-dimensional retrospective registration of magnetic resonance images.

A global optimization technique for image registration, based on mutual information, that can be used in conjunction with a multi-resolution paradigm is described. This technique combines genetic algorithm in continuous space, which is a stochastic method and is very efficient in large search space, with dividing rectangle, which is a deterministic method that theoretically guarantees global optimization and is efficient in small search space. Calculations were performed for determining the optimum parameters for implementing this method. This technique was applied to register magnetic resonance images of brain. For comparison, the registration results using AIR, a commonly employed software package, are presented.

Algorithms↗
PubMed2005

Normalized mutual information based registration using k-means clustering and shading correction.

In this paper the influence of intensity clustering and shading correction on mutual information based image registration is studied. Instead of the generally used equidistant re-binning, we use k-means clustering in order to achieve a more natural binning of the intensity distribution. Secondly, image inhomogeneities occurring notably in MR images can have adverse effects on the registration. We use a shading correction method in order to reduce these effects. The method is validated on clinical MR, CT and PET images, as well as synthetic MR images. It is shown that by employing clustering with inhomogeneity correction the number of misregistrations is reduced without loss of accuracy thus increasing robustness as compared to the standard non-inhomogeneity corrected and equidistant binning based registration.

Algorithms↗