Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “mutual information”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3Linked to original sources

Learning curves for mutual information maximization.

An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed [S. Becker and G. Hinton, Nature (London) 355, 161 (1992)]. For a generic data model, I show that in the large sample limit the structure in the data is recognized by mutual information maximization. For a more restricted model, where the networks are similar to perceptrons, I calculate the learning curves for zero-temperature Gibbs learning. These show that convergence can be rather slow, and a way of regularizing the procedure is considered.

Journal Article↗

Mutual information and phase dependencies: measures of reduced nonlinear cardiorespiratory interactions after myocardial infarction.

The heart rate variability (HRV) is related to several mechanisms of the complex autonomic functioning such as respiratory heart rate modulation and phase dependencies between heart beat cycles and breathing cycles. The underlying processes are basically nonlinear. In order to understand and quantitatively assess those physiological interactions an adequate coupling analysis is necessary. We hypothesized that nonlinear measures of HRV and cardiorespiratory interdependencies are superior to the standard HRV measures in classifying patients after acute myocardial infarction. We introduced mutual information measures which provide access to nonlinear interdependencies as counterpart to the classically linear correlation analysis. The nonlinear statistical autodependencies of HRV were quantified by auto mutual information, the respiratory heart rate modulation by cardiorespiratory cross mutual information, respectively. The phase interdependencies between heart beat cycles and breathing cycles were assessed basing on the histograms of the frequency ratios of the instantaneous heart beat and respiratory cycles. Furthermore, the relative duration of phase synchronized intervals was acquired. We investigated 39 patients after acute myocardial infarction versus 24 controls. The discrimination of these groups was improved by cardiorespiratory cross mutual information measures and phase interdependencies measures in comparison to the linear standard HRV measures. This result was statistically confirmed by means of logistic regression models of particular variable subsets and their receiver operating characteristics.

Aged↗

Mutual information for automated unwarping of rat brain autoradiographs.

An automated multimodal warping based on mutual information metric (MI) as a mapping cost function is demonstrated. Mutual information (I) is calculated from a two-dimensional (2D) gray scale histogram of an image pair, and MI (= -I) provides a matching cost function which can be effective in registration of two- or three-dimensional data sets independent of modality. Most histological image data, though information rich and high resolution, present nonlinear deformations due to the specimen sectioning and need reconstitution into deformation-corrected volumes prior to geometric mapping to an anatomical volume for spatial analyses. Section alignment via automatic 2D registrations employing MI as a global cost function and thin-plate-spline (TPS) warping is applied to deoxy-D-[14C]glucose autoradiographic image slices of a rat brain with video reference images of the uncut block face to reconstitute a cerebral glucose metabolic volume data. Unlike the traditional feature-based TPS warping algorithms, initial control points are defined independently from feature landmarks. Registration quality using automated multimodal image warping is validated by comparing MIs of the image pair registered by automated affine registration and manual warping method. The MI proves to be a robust objective matching cost function effective for automatic multimodality warping for 2D data sets and can be readily applied to volume registrations.

Algorithms↗

Distillation protocols: output entanglement and local mutual information.

A complementary behavior between local mutual information and average output entanglement is derived for arbitrary bipartite ensembles. This leads to bounds on the yield of entanglement in distillation protocols that involve disinguishing. This bound is saturated in the hashing protocol for distillation, for Bell-diagonal states.

Journal Article↗

Mutual information relevance networks: functional genomic clustering using pairwise entropy measurements.

Increasing numbers of methodologies are available to find functional genomic clusters in RNA expression data. We describe a technique that computes comprehensive pair-wise mutual information for all genes in such a data set. An association with a high mutual information means that one gene is non-randomly associated with another; we hypothesize this means the two are related biologically. By picking a threshold mutual information and using only associations at or above the threshold, we show how this technique was used on a public data set of 79 RNA expression measurements of 2,467 genes to construct 22 clusters, or Relevance Networks. The biological significance of each Relevance Network is explained.

Computer Simulation↗

Mutual information of population codes and distance measures in probability space.

We studied the mutual information between a stimulus and a system consisting of stochastic, statistically independent elements that respond to a stimulus. Using statistical mechanical methods the properties of the mutual information (MI) in the limit of a large system size N are calculated. For continuous valued stimuli, the MI increases logarithmically with N and is related to the log of the Fisher information of the system. For discrete stimuli the MI saturates exponentially with N. We find that the exponent of saturation of the MI is the Chernoff distance between response probabilities that are induced by different stimuli.

Brain↗

The use of mutual information in registration of CT and MRI datasets post permanent implant.

PURPOSE: To determine the feasibility of registration of MRI and CT datasets post permanent prostate implant by the use of mutual information. METHODS AND MATERIALS: Five patients who underwent permanent (125)I implant for prostate carcinoma were studied. Two weeks postimplant an axial CT, T2-weighted-axial, sagittal and coronal MRI, and T1-fat-saturation MRI scans were obtained. Registrations of MRI to CT and MRI to MRI datasets were performed by mutual information, an automated process of data registration matching all information in specified dataset regions of interest. Registration quality was evaluated by visual inspection, agreement with seed- to-seed registration, and histogram analysis. RESULTS: Rapid registration (<30 minutes) of CT and MRI datasets can be accomplished through the use of mutual information. All methods of registration evaluation confirmed excellent registration quality. Although D90 and V100 for the prostate were comparable between MRI- and CT-based dosimetry, dose to critical structures/microenvironments (anterior base, posterior base, bladder outlet, lower sphincter, bulbar urethra) defined on MRI varied widely. CONCLUSIONS: Efficient and accurate registration of MRI and CT datasets following prostate implant is possible, and improves the accuracy of postimplant dosimetry by superior definition of the prostate. Definition of critical microenvironments and adjacent structures will improve dose and toxicity correlation and ultimately improve planning strategies.

Brachytherapy↗

A viscous fluid model for multimodal non-rigid image registration using mutual information.

We propose a multimodal free-form registration algorithm based on maximization of mutual information. The warped image is modeled as a viscous fluid that deforms under the influence of forces derived from the gradient of the mutual information registration criterion. Parzen windowing is used to estimate the joint intensity probability of the images to be matched. The method is evaluated for non-rigid inter-subject registration of MR brain images. The accuracy of the method is verified using simulated multi-modal MR images with known ground truth deformation. The results show that the root mean square difference between the recovered and the ground truth deformation is smaller than 1 voxel. We illustrate the application of the method for atlas-based brain tissue segmentation in MR images in case of gross morphological differences between atlas and patient images.

Algorithms↗

Automated surface matching using mutual information applied to Riemann surface structures.

Many medical imaging applications require the computation of dense correspondence vector fields that match one surface with another. To avoid the need for a large set of manually-defined landmarks to constrain these surface correspondences, we developed an algorithm to automate the matching of surface features. It extends the mutual information method to automatically match general 3D surfaces (including surfaces with a branching topology). First, we use holomorphic 1-forms to induce consistent conformal grids on both surfaces. High genus surfaces are mapped to a set of rectangles in the Euclidean plane, and closed genus-zero surfaces are mapped to the sphere. Mutual information is used as a cost functional to drive a fluid flow in the parameter domain that optimally aligns stable geometric features (mean curvature and the conformal factor) in the 2D parameter domains. A diffeomorphic surface-to-surface mapping is then recovered that matches anatomy in 3D. We also present a spectral method that ensures that the grids induced on the target surface remain conformal when pulled through the correspondence field. Using the chain rule, we express the gradient of the mutual information between surfaces in the conformal basis of the source surface. This finite-dimensional linear space generates all conformal reparameterizations of the surface. We apply the method to hippocampal surface registration, a key step in subcortical shape analysis in Alzheimer's disease and schizophrenia.

Algorithms↗

The quantitative evaluation of functional neuroimaging experiments: mutual information learning curves.

Learning curves are presented as an unbiased means for evaluating the performance of models for neuroimaging data analysis. The learning curve measures the predictive performance in terms of the generalization or prediction error as a function of the number of independent examples (e.g., subjects) used to determine the parameters in the model. Cross-validation resampling is used to obtain unbiased estimates of a generic multivariate Gaussian classifier, for training set sizes from 2 to 16 subjects. We apply the framework to four different activation experiments, in this case [(15)O]water data sets, although the framework is equally valid for multisubject fMRI studies. We demonstrate how the prediction error can be expressed as the mutual information between the scan and the scan label, measured in units of bits. The mutual information learning curve can be used to evaluate the impact of different methodological choices, e.g., classification label schemes, preprocessing choices. Another application for the learning curve is to examine the model performance using bias/variance considerations enabling the researcher to determine if the model performance is limited by statistical bias or variance. We furthermore present the sensitivity map as a general method for extracting activation maps from statistical models within the probabilistic framework and illustrate relationships between mutual information and pattern reproducibility as derived in the NPAIRS framework described in a companion paper.

Adult↗

Evaluation of mutual information and genetic programming for feature selection in QSAR.

Feature selection is a key step in Quantitative Structure Activity Relationship (QSAR) analysis. Chance correlations and multicollinearity are two major problems often encountered when attempting to find generalized QSAR models for use in drug design. Optimal QSAR models require an objective variable relevance analysis step for producing robust classifiers with low complexity and good predictive accuracy. Genetic algorithms coupled with information theoretic approaches such as mutual information have been used to find near-optimal solutions to such multicriteria optimization problems. In this paper, we describe a novel approach for analyzing QSAR data based on these methods. Our experiments with the Thrombin dataset, previously studied as part of the KDD (Knowledge Discovery and Data Mining) Cup 2001 demonstrate the feasibility of this approach. It has been found that it is important to take into account the data distribution, the rule "interestingness", and the need to look at more invariant and monotonic measures of feature selection.

Algorithms↗

Average mutual information of coding and noncoding DNA.

One basic problem in the analysis of DNA sequences is the recognition of protein-coding genes. Computer algorithms to facilitate gene identification have become important as genome sequencing projects have turned from mapping to large-scale sequencing, resulting in an exponentially growing number of sequenced nucleotides that await their annotation. Many statistical patterns have been discovered that are different in coding and noncoding DNA, but most of them vary from species to species, and hence require prior training on organism-specific data sets. Here, we investigate if there exist species-independent statistical patterns that are different in coding and noncoding DNA. We introduce an information-theoretic quantity, the average mutual information (AMI), and we find that the probability distribution functions of the AMI are significantly different in coding and noncoding DNA, while they are almost identical for different species. This finding suggests that the AMI might be useful for the recognition of protein-coding regions in genomes for which training sets do not exist.

Algorithms↗

EEG in schizophrenic patients: mutual information analysis.

OBJECTIVE: The aim of the present study is to assess information transmission between different cortical areas in schizophrenics by estimating the average cross mutual information (A-CMI) and to characterize the dynamical property of the cortical areas of schizophrenic patients from multi-channel EEG by establishing the auto mutual information (AMI). METHODS: We recorded the EEG from 16 electrodes in 10 schizophrenic patients and 10 age-matched normal controls. We estimated the slope of the AMI to evaluate the complexity of the EEG signal from one electrode and the A-CMI values of all 16x16 pairs of electrodes were calculated to investigate the information transmission of different cortical areas in schizophrenic patients. RESULTS: In T5 and C3 electrodes, the schizophrenic patients had lower complexity than normal controls. The schizophrenic patients had significantly higher interhemispheric and intrahemispheric A-CMI values than the normal controls. CONCLUSIONS: These results are consistent with previous findings that suggest left hemispheric hypotemporality and inter- and/or intra-hemispheric overconnectivity in schizophrenics. Our results of the left hemispheric hypotemporality and the increased interhemispheric information transmission in temporal lobe may support the hypothesis that the abnormal laterlization in temporal lobe are due to left temporal lobe deficit in schizophrenic patients.

Adult↗

Use and uncertainties of mutual information for computed tomography/ magnetic resonance (CT/MR) registration post permanent implant of the prostate.

Post-implant dosimetric analysis for permanent implant of the prostate benefits from the use of a computed tomography (CT) dataset for optimal identification of the radioactive source (seed) positions and a magnetic resonance (MR) dataset for optimal description of the target and normal tissue volumes. The CT/MR registration process should be fast and sufficiently accurate to yield a reliable dosimetric analysis. Since critical normal tissues typically reside in dose gradient regions, small shifts in the dose distribution could impact the prediction of complication or complication severity. Standard procedures include the use of the seed distribution as fiducial markers (seed match), a time consuming process that relies on the proper identification of signals due to the same seed on both datasets. Mutual information (MI) is more efficient because it uses image data requiring minimal preparation effort. A comparison of MI registration and seed-match registration was performed for twelve patients. MI was applied to a volume limited to the prostate and surrounding structures, excluding most of the pelvic bone structures (margins around the prostate gland were approximately 2 cm right-left, approximately 1 cm anterior-posterior, and approximately 2 cm superior-inferior). Seeds were identified on a 2 mm slice CT dataset using an automatic seed identification procedure on reconstructed three-dimensional data. Seed positions on the 3 mm slice thickness T2 MR data set were identified using a point-and-click method on each image. Seed images were identified on more than one MR slice, and the results used to determine average seed coordinates for MR images and matched seed pairs between CT and MR images. On average, 42% (19%-64%) of the seeds (19-54 seeds) were identified and matched to their CT counterparts. A least-squares method applied to the CT and MR seed coordinates was used to produce the optimum seed-match registration. MI registration and seed match registration angle differences averaged 0.5 degrees, which was not significantly different from zero. Translation differences averaged 0.6 (1.2 standard deviation) mm right-left, -0.5(1.5) mm posterior-anterior, and -1.2(2.0) mm inferior-superior. Registration error estimates were approximately 2 mm for both the MI and seed-match methods. The observed standard deviations in the offset values were consistent with propagation of error. Registration methods as applied here using mutual information and seed matching are consistent, except for a small systematic difference in the inferior-superior axis for a minority of cases (approximately 15%). Cases registered with mutual information and with bony anatomy misregistration of greater than approximately 5 mm should be evaluated for rescan or seed-match registration. The improvement in efficiency of use for the MI registration method is substantial, approximately 30 min compared to several hours using seed match registration.

Algorithms↗

Independent neurons representing a finite set of stimuli: dependence of the mutual information on the number of units sampled.

We study the capacity with which a system of independent neuron-like units represents a given set of stimuli. We assume that each neuron provides a fixed amount of information and that the information provided by different neurons has a random overlap. We derive analytically the dependence of the mutual information between the set of stimuli and the neural responses on the number of units sampled. For a large set of stimuli, the mutual information rises linearly with the number of neurons and later saturates exponentially at its maximum value.

Animals↗

Analysis of mutual information content for EEG responses to odor stimulation for subjects classified by occupation.

To investigate the changes of cortico-cortical connectivity during odor stimulation of subjects classified by occupation, the mutual information content of EEGs was examined for general workers, perfume salespersons and professional perfume researchers. Analysis of the averaged-cross mutual information content (A-CMI) from the EEGs revealed that among the professional perfume researchers changes in the A-CMI values during odor stimulation were more apparent in the frontal region of the brain, while for the general workers and perfume salespersons such changes were more conspicuous in the overall posterior temporal, parietal and frontal regions. These results indicate that the brains of professional perfume researchers respond to odors mainly in the frontal region, reflecting the function of the orbitofrontal cortex (OFC) due to the occupational requirement of these subjects to discriminate or identify odors. During odor stimulation, the perfume salespersons, although relatively more exposed to odors than the general workers, showed similar changes to the general workers. The A-CMI value is in inverse proportion to psychological preferences of the professional perfume researchers and perfume salespersons, though this is not the case with the general workers. This result suggests that functional coupling for people who are occupationally exposed to odors may be related to psychological preference.

Adult↗

Mutual information function assesses autonomic information flow of heart rate dynamics at different time scales.

The autonomic information flow (AIF) represents the complex communication within the Autonomic Nervous System (ANS). It can be assessed by the mutual information function (MIF) of heart rate fluctuations (HRF). The complexity of HRF is based on several interacting physiological mechanisms operating at different time scales. Therefore one prominent time scale for HRF complexity analysis is not given a priori. The MIF reflects the information flow at different time scales. This approach is defined and evaluated in the present paper. In order to aggregate relevant physiological time scales, the MIF of HRF obtained from eight adult Lewis rats during the awake state, under general anesthesia, with additional vagotomy, and additional betal-adrenergic blockade are investigated. Physiologically relevant measures of the MIF were assessed with regard to the discrimination of these states. A simulation study of a periodically excited pendulum is performed to clarify the influence of the time scale of MIF in comparison to the Kolmogorov Sinai entropy (KSE) of that well defined system. The general relevance of the presented AIF approach was confirmed by comparing mutual information, approximate entropy, and sample entropy at their respective time scales.

Algorithms↗

Global dynamics of a network of stochastic neurons maximizes local mutual information.

We define a stochastic neuron as an element that increases its internal state with probability p until a threshold value is reached; after that its internal state is set back to the initial value. We study the local information of a stochastic neuron between the message arriving from the input neurons and the response of the neuron. We study the dependence of the local information on the firing probability alpha of the synaptic inputs in a network of such stochastic neurons. The values of alpha obtained in the simulations are the same as those obtained theoretically by maximization of local mutual information. We conclude that the global dynamics maximizes the local mutual information of single units, which means that the self-selected parameter value of the population dynamics is such that each neuron behaves as an optimal encoder.

Animals↗