Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “mutual information”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources
PubMed2006

Increased information transmission during scientific hypothesis generation: mutual information analysis of multichannel EEG.

Hypothesis generation has been regarded as one of the core reasoning processes in creative thinking and scientific discovery. To investigate changes in the amount of information transmission during scientific hypothesis generation, the averaged cross-mutual-information (A-CMI) of EEGs was estimated. Twenty-five 5th grade students were sampled in this study. EEG signals from 16 electrodes on each subject's scalp were recorded using a 32-channel EEG system. In order to generate hypotheses, the students were asked to observe 20 quail eggs that gave rise to questions such as: Why do different sizes and shapes of patterns appear on the surface of the eggs? After the observation, they were asked to generate a scientific hypothesis-a tentative causal explanation for the evoked question. The results of experimentation indicated several distinct brain activities during hypothesis generation interacting between different local brain regions. In addition, it was observed that the amount of information transmission during hypothesis generation increased in a large part of the brain region encompassing the temporal, parietal, and occipital cortexes, which implies the use of declarative and procedural memory systems. Furthermore, this study suggested the possibility that neuropsychological approaches may be potential tools to investigate the neuronal activity of EEGs during hypothesis generation.

Algorithms↗
PubMed2006

Mutual information and electron correlation in momentum space.

Mutual information and information entropies in momentum space are proposed as measures of the nonlocal aspects of information. Singlet and triplet state members of the helium isoelectronic series are employed to examine Coulomb and Fermi correlations, and their manifestations, in both the position and momentum space mutual information measures. The triplet state measures exemplify that the magnitude of the spatial correlations relative to the momentum correlations depends on and may be controlled by the strength of the electronic correlation. The examination of one- and two-electron Shannon entropies in the triplet state series yields a crossover point, which is characterized by a localized momentum density. The mutual information density in momentum space illustrates that this localization is accompanied by strong correlation at small values of p.

Journal Article↗
PubMed2005

Statistical validation of mutual information calculations: comparison of alternative numerical algorithms.

Given two time series X and Y , their mutual information, I (X,Y) = I (Y,X) , is the average number of bits of X that can be predicted by measuring Y and vice versa. In the analysis of observational data, calculation of mutual information occurs in three contexts: identification of nonlinear correlation, determination of an optimal sampling interval, particularly when embedding data, and in the investigation of causal relationships with directed mutual information. In this contribution a minimum description length argument is used to determine the optimal number of elements to use when characterizing the distributions of X and Y . However, even when using partitions of the X and Y axis indicated by minimum description length, mutual information calculations performed with a uniform partition of the XY plane can give misleading results. This motivated the construction of an algorithm for calculating mutual information that uses an adaptive partition. This algorithm also incorporates an explicit test of the statistical independence of X and Y in a calculation that returns an assessment of the corresponding null hypothesis. The previously published Fraser-Swinney algorithm for calculating mutual information includes a sophisticated procedure for local adaptive control of the partitioning process. When the Fraser and Swinney algorithm and the algorithm constructed here are compared, they give very similar numerical results (less than 4% difference in a typical application). Detailed comparisons are possible when X and Y are correlated jointly Gaussian distributed because an analytic expression for I (X,Y) can be derived for that case. Based on these tests, three conclusions can be drawn. First, the algorithm constructed here has an advantage over the Fraser-Swinney algorithm in providing an explicit calculation of the probability of the null hypothesis that X and Y are independent. Second, the Fraser-Swinney algorithm is marginally the more accurate of the two algorithms when large data sets are used. With smaller data sets, however, the Fraser-Swinney algorithm reports structures that disappear when more data are available. Third, the algorithm constructed here requires about 0.5% of the computation time required by the Fraser-Swinney algorithm.

Journal Article↗
PubMed2003

Automated generation of a four-dimensional model of the liver using warping and mutual information.

The use of mutual information (MI) based alignment to map changes in liver shape and position from exhale to inhale was investigated. Inhale and exhale CT scans were obtained with intravenous contrast for six patients. MI based alignment using thin-plate spine (TPS) warping was performed between each inhale and exhale image set. An expert radiation oncologist identified corresponding vessel bifurcations on the exhale and inhale CT image and the transformation for identified points was determined. This transformation was then used to determine the accuracy of the MI based alignment. The reproducibility of the vessel bifurcation identification was measured through repeat blinded vessel bifurcation identification. Reproducibility [standard deviation (SD)] in the L/R, A/P, and I/S directions was 0.11, 0.09, and 0.14 cm, respectively. The average absolute difference between the transformation obtained using MI based alignment and the vessel bifurcation in the L/R, A/P, and I/S directions was 0.13 cm (SD=0.10 cm), 0.15 cm (SD=0.12 cm), and 0.15 cm (SD-0.14 cm), respectively. These values are comparable to the reproducibility of bifurcation identification, indicating that MI based alignment using TPS warping is accurate to within measurement error and is a reliable tool to aid in describing deformation that the liver undergoes from the exhale to inhale state.

Algorithms↗
PubMedDate not supplied

Mutual information of sparsely coded associative memory with self-control and ternary neurons.

The influence of a macroscopic time-dependent threshold on the retrieval dynamics of attractor associative memory models with ternary neurons ¿-1, 0, +1¿ is examined. If the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the memorized patterns in the model, adapting itself in the course of the time evolution, it guarantees an autonomous functioning of the model. Especially in the limit of sparse coding, it is found that this self-control mechanism considerably improves the quality of the fixed-point retrieval dynamics, in particular the storage capacity, the basins of attraction and the information content. The mutual information is shown to be the relevant parameter to study the retrieval quality of such sparsely coded models. Numerical results confirm these observations.

Association Learning↗
PubMed2003

Mutual-information-based registration of medical images: a survey.

An overview is presented of the medical image processing literature on mutual-information-based registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutual-information-based registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges.

Algorithms↗
PubMed2005

Mutual information and correlation measures in atomic systems.

Mutual information is introduced as an electron correlation measure and examined for isoelectronic series and neutral atoms. We show that it possesses the required characteristics of a correlation measure and is superior to the behavior of the radial correlation coefficient in the neon series. A local mutual information, and related local quantities, are used to examine the local contributions to Fermi correlation, and to demonstrate and to interpret the intimate relationship between correlation and localization.

Journal Article↗
PubMed2001

Mutual information analysis of the EEG in patients with Alzheimer's disease.

OBJECTIVE: Mutual information provides a measure of both the linear and nonlinear statistical dependencies between two time series. Cross-mutual information (CMI) is used to quantify the information transmitted from one time series to another, while auto mutual information (AMI) in a time series estimates how much on average the value of the time series can be predicted from values of the time series at preceding points. The aim of this study is to assess information transmission between different cortical areas in Alzheimer's disease (AD) patients by estimating the average CMI between EEG electrodes. METHODS: We recorded the EEG from 16 scale electrodes in 15 AD patients and 15 age-matched normal controls, and estimated the local, distant, and interhemispheric CMIs of the EEG in both groups. The rate of decrease (with increasing delay) of the AMI of the EEG was also measured to evaluate the complexity of the EEG in AD patients. RESULTS: The local CMI in AD subjects was lower than that in normal controls, especially over frontal and antero-temporal regions. A prominent decrease in information transmission between distant electrodes in the right hemisphere and between corresponding interhemispheric electrodes was detected in the AD patients. In addition, the AMIs throughout the cerebrums of the AD patients decreased significantly more slowly with delay than did the AMIs of normal controls. CONCLUSIONS: These results are consistent with previous findings that suggest the association of EEG abnormalities in AD patients with functional impairment of information transmission in long cortico-cortical connections.

Aged↗
PubMed2001

A bound on mutual information for image registration.

An upper bound is derived for the mutual information between a fixed image and a deformable template containing a fixed number of gray-levels. The bound can be calculated by maximizing the entropy of the template under the constraint that the conditional entropy of the template, given the fixed image, be zero. This bound provides useful insight into the properties of mutual information as a similarity metric for deformable image registration. Specifically, it indicates that maximizing mutual information may not necessarily produce an optimal solution when the deformable transform is too flexible.

Image Processing, Computer-Assisted↗
PubMed1999

Rigid point feature registration using mutual information.

We have developed a new mutual information-based registration method for matching unlabeled point features. In contrast to earlier mutual information-based registration methods, which estimate the mutual information using image intensity information, our approach uses the point feature location information. A novel aspect of our approach is the emergence of correspondence (between the two sets of features) as a natural by-product of joint density estimation. We have applied this algorithm to the problem of geometric alignment of primate autoradiographs. We also present preliminary results on three-dimensional robust matching of sulci derived from anatomical magnetic resonance images. Finally, we present an experimental comparison between the mutual information approach and other recent approaches which explicitly parameterize feature correspondence.

Algorithms↗
PubMed2004

Gene clustering based on clusterwide mutual information.

Cluster analysis of gene-wide expression data from DNA microarray hybridization studies has proved to be a useful tool for identifying biologically relevant groupings of genes and constructing gene regulatory networks. The motivation for considering mutual information is its capacity to measure a general dependence among gene random variables. We propose a novel clustering strategy based on minimizing mutual information among gene clusters. Simulated annealing is employed to solve the optimization problem. Bootstrap techniques are employed to get more accurate estimates of mutual information when the data sample size is small. Moreover, we propose to combine the mutual information criterion and traditional distance criteria such as the Euclidean distance and the fuzzy membership metric in designing the clustering algorithm. The performances of the new clustering methods are compared with those of some existing methods, using both synthesized data and experimental data. It is seen that the clustering algorithm based on a combined metric of mutual information and fuzzy membership achieves the best performance. The supplemental material is available at www.gspsnap.tamu.edu/gspweb/zxb/glioma_zxb.

Algorithms↗
PubMed2004

Least-dependent-component analysis based on mutual information.

We propose to use precise estimators of mutual information (MI) to find the least dependent components in a linearly mixed signal. On the one hand, this seems to lead to better blind source separation than with any other presently available algorithm. On the other hand, it has the advantage, compared to other implementations of "independent" component analysis (ICA), some of which are based on crude approximations for MI, that the numerical values of the MI can be used for (i) estimating residual dependencies between the output components; (ii) estimating the reliability of the output by comparing the pairwise MIs with those of remixed components; and (iii) clustering the output according to the residual interdependencies. For the MI estimator, we use a recently proposed k -nearest-neighbor-based algorithm. For time sequences, we combine this with delay embedding, in order to take into account nontrivial time correlations. After several tests with artificial data, we apply the resulting MILCA (mutual-information-based least dependent component analysis) algorithm to a real-world dataset, the ECG of a pregnant woman.

Journal Article↗
PubMed2000

Three-state neural network: from mutual information to the Hamiltonian.

The mutual information, I, of the three-state neural network can be obtained exactly for the mean-field architecture, as a function of three macroscopic parameters: the overlap, the neural activity and the activity-overlap, i.e., the overlap restricted to the active neurons. We perform an expansion of I on the overlap and the activity-overlap, around their values for neurons almost independent of the patterns. From this expansion we obtain an expression for a Hamiltonian which optimizes the retrieval properties of this system. This Hamiltonian has the form of a disordered Blume-Emery-Griffiths model. The dynamics corresponding to this Hamiltonian is found. As a special characteristic of such a network, we see that information can survive even if no overlap is present. Hence the basin of attraction of the patterns and the retrieval capacity is much larger than for the Hopfield network. The extreme diluted version is analyzed, the curves of information are plotted and the phase diagrams are built.

Models, Neurological↗
PubMed2000

Physical nature of higher-order mutual information: intrinsic correlations and frustration

This paper studies some properties and implications of higher-order mutual information functions, which should serve for the analysis of general complex systems. We note that the higher-order mutual information can either be positive or negative depending on the correlation among ensembles. Two opposite types of correlations are discussed in connection with the concept of frustration. Simple examples are presented to demonstrate that our concepts are especially helpful in understanding the nature of correlations in frustrated systems. The higher-order mutual information provides an appropriate measure of the frustration effect.

Journal Article↗
PubMed1997

Nonlinear pattern analysis of ventricular premature beats by mutual information.

The frequency of ventricular premature beats (VPBs) has been related to the risk of mortality. However, little is known about the temporal pattern of occurrence of VPBs and its relationship to autonomic activity. Hence, we applied a general correlation measure, mutual information, to quantify how VPBs are generated over time. We also used mutual information to determine the correlation between VPB production and heart rate in order to evaluate effects of autonomic activity on VPB production. We examined twenty subjects with more than 3000 VPBs/day and simulated random time series of VPB occurrence. We found that mutual information values could be used to characterize quantitatively the temporal patterns of VPB generation. Our data suggest that VPB production is not random and VPBs generated with a higher value of mutual information may be more greatly affected by autonomic activity.

Adult↗
PubMed2003

Comparison of an image registration technique based on normalized mutual information with a standard method utilizing implanted markers in the staged radiosurgical treatment of large arteriovenous malformations.

PURPOSE: To compare a noninvasive technique based on normalized mutual information for registering image sets associated with staged radiosurgical treatments of large arteriovenous malformations (AVMs), with a gold-standard method using radiographically evident markers implanted in the skull. METHODS: Nine patients receiving multistage treatment of large AVMs at the University of California at San Francisco (UCSF) gamma knife facility were included in this study. For each patient, the transformations of shot coordinates between a reference treatment stage and subsequent treatment stages were determined at UCSF, based on radiographically defined coordinates of implanted markers in each stereotactic space. A magnetic resonance (MR) image set was acquired for each treatment stage, and used for treatment planning. The two MR image sets for each treatment pair were sent to Yale for an unbiased, independent analysis of shot transformations. An image registration technique based on normalized mutual information was used to produce a single fused image study for each treatment pair. External copper sulfate fiducial markers for both image sets were evident on the fused images, allowing coordinates in both stereotactic systems to be defined. Coordinate transformation between the two systems was determined, based on digitized coordinates of seven common fiducial marker images. RESULTS: The average measured overall root-mean-square discrepancy between the Yale and UCSF transformed shot coordinates is 1.1 +/- 0.3 mm. The corresponding error in Yale transformed coordinates is 1.0 +/- 0.3 mm, assuming an inherent 0.5 mm error in the UCSF method. CONCLUSIONS: The normalized mutual information method can be used to obtain good image registration between successive sessions in staged treatments. Further improvements in the reported methodology are outlined. Because the mutual information method is less invasive than the implanted marker method, it may be preferable in many cases.

Algorithms↗
PubMed2006

A study on medical image registration by mutual information with pyramid data structure.

The registration method based on mutual information is currently a popular technique for medical image registration, but the computation of the mutual information is complex and the registration speed is slow. In this work, a new slice accumulation pyramid (SAP) data structure was proposed to expedite the registration process. A numerical comparative study between the new data structure and the existing wavelet pyramid (WP) data structure was given, and the results confirmed that the new pyramid data structure was superior to the WP in both the calculation efficiency and the optimizing performance. Finally, SAP was applied to remove the artifacts between CT and MRI data sets, and the results showed the validation of SAP to registration of mulmodality images.

Artifacts↗
PubMed2005

Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information.

A novel feature selection method using the concept of mutual information (MI) is proposed in this paper. In all MI based feature selection methods, effective and efficient estimation of high-dimensional MI is crucial. In this paper, a pruned Parzen window estimator and the quadratic mutual information (QMI) are combined to address this problem. The results show that the proposed approach can estimate the MI in an effective and efficient way. With this contribution, a novel feature selection method is developed to identify the salient features one by one. Also, the appropriate feature subsets for classification can be reliably estimated. The proposed methodology is thoroughly tested in four different classification applications in which the number of features ranged from less than 10 to over 15,000. The presented results are very promising and corroborate the contribution of the proposed feature selection methodology.

Algorithms↗