Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “mutual information”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9Linked to original sources
PubMed2004

F-information measures in medical image registration.

A measure for registration of medical images that currently draws much attention is mutual information. The measure originates from information theory, but has been shown to be successful for image registration as well. Information theory, however, offers many more measures that may be suitable for image registration. These all measure the divergence of the joint distribution of the images' grey values from the joint distribution that would have been found had the images been completely independent. This paper compares the performance of mutual information as a registration measure with that of other F-information measures. The measures are applied to rigid registration of positron emission tomography (PET)/magnetic resonance (MR) and MR/computed tomography (CT) images, for 35 and 41 image pairs, respectively. An accurate gold standard transformation is available for the images, based on implanted markers. The registration performance, robustness and accuracy of the measures are studied. Some of the measures are shown to perform poorly on all aspects. The majority of measures produces results similar to those of mutual information. An important finding, however, is that several measures, although slightly more difficult to optimize, can potentially yield significantly more accurate results than mutual information.

Algorithms↗
PubMed1996

Information theoretic analysis of action potential trains. II. Analysis of correlation among n neurons to deduce connection structure.

We propose a cross-correlational method based on information theory, which produces a network connection structure to account for observed patterns of action potential activity in multi-unit recordings. Firing probabilities and conditional probabilities are estimated from the action potential trains of n neurons. Two-point mutual information (2pMI) and joint conditional mutual information (JCMI) are calculated by using the estimated probabilities, and then the n-point mutual information (npMI) is calculated. A significant peak of npMI indicates that each neuron is connected to all other neurons at specified time differences, either directly or indirectly. To distinguish between direct and indirect connection, the two-point m-joint conditional mutual information (2pJCMI) is calculated over the peak region for each pair of neurons. A minimum effective connection structure among the n neurons can be deduced in this manner. The procedure for deducing the connection structure for three- and n-neuron networks is described. We apply this method to action potential trains produced by simulated neural networks. Some limitations of the method are also discussed.

Action Potentials↗
PubMedDate not supplied

Quantifying information and performance for flash detection in the blowfly photoreceptor.

Performance on specific tasks in an organism's everyday activities is essential to survival. In this paper, we extend information-theoretic investigation of neural systems to task specific information using a detailed biophysical model of the blowfly photoreceptor. We determine the optimal detection performance using ideal observer analysis and find that detection threshold increases with background light according to a power function. We show how Fisher information is related to the detection performance and compare Fisher information and mutual information in this task-specific context. Our detailed model of the blowfly photoreceptor enables us to detangle the components of phototransduction and analyze the sensitivity of detection performance with respect to biophysical parameters. The biophysical model of the blowfly photoreceptor provides a rich framework for investigation of neural systems.

Algorithms↗
PubMed2005

Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain.

Several norms for how people should assess a question's usefulness have been proposed, notably Bayesian diagnosticity, information gain (mutual information), Kullback-Liebler distance, probability gain (error minimization), and impact (absolute change). Several probabilistic models of previous experiments on categorization, covariation assessment, medical diagnosis, and the selection task are shown to not discriminate among these norms as descriptive models of human intuitions and behavior. Computational optimization found situations in which information gain, probability gain, and impact strongly contradict Bayesian diagnosticity. In these situations, diagnosticity's claims are normatively inferior. Results of a new experiment strongly contradict the predictions of Bayesian diagnosticity. Normative theoretical concerns also argue against use of diagnosticity. It is concluded that Bayesian diagnosticity is normatively flawed and empirically unjustified.

Bayes Theorem↗
PubMed2000

Stochastic resonance in ion channels characterized by information theory.

We identify a unifying measure for stochastic resonance (SR) in voltage dependent ion channels which comprises periodic (conventional), aperiodic, and nonstationary SR. Within a simplest setting, the gating dynamics is governed by two-state conductance fluctuations, which switch at random time points between two values. The corresponding continuous time point process is analyzed by virtue of information theory. In pursuing this goal we evaluate for our dynamics the tau information, the mutual information, and the rate of information gain. As a main result we find an analytical formula for the rate of information gain that solely involves the probability of the two channel states and their noise averaged rates. For small voltage signals it simplifies to a handy expression. Our findings are applied to study SR in a potassium channel. We find that SR occurs only when the closed state is predominantly dwelled upon. Upon increasing the probability for the open channel state the application of an extra dose of noise monotonically deteriorates the rate of information gain, i.e., no SR behavior occurs.

Biophysical Phenomena↗
PubMed2003

Cortical surface registration for image-guided neurosurgery using laser-range scanning.

In this paper, a method of acquiring intraoperative data using a laser range scanner (LRS) is presented within the context of model-updated image-guided surgery. Registering textured point clouds generated by the LRS to tomographic data is explored using established point-based and surface techniques as well as a novel method that incorporates geometry and intensity information via mutual information (SurfaceMI). Phantom registration studies were performed to examine accuracy and robustness for each framework. In addition, an in vivo registration is performed to demonstrate feasibility of the data acquisition system in the operating room. Results indicate that SurfaceMI performed better in many cases than point-based (PBR) and iterative closest point (ICP) methods for registration of textured point clouds. Mean target registration error (TRE) for simulated deep tissue targets in a phantom were 1.0 +/- 0.2, 2.0 +/- 0.3, and 1.2 +/- 0.3 mm for PBR, ICP, and SurfaceMI, respectively. With regard to in vivo registration, the mean TRE of vessel contour points for each framework was 1.9 +/- 1.0, 0.9 +/- 0.6, and 1.3 +/- 0.5 for PBR, ICP, and SurfaceMI, respectively. The methods discussed in this paper in conjunction with the quantitative data provide impetus for using LRS technology within the model-updated image-guided surgery framework.

Adult↗
PubMed2002

Receptive field organization determines pyramidal cell stimulus-encoding capability and spatial stimulus selectivity.

Sensory systems must operate over a wide range of spatial scales, and single neuron receptive field (RF) organization may contribute to the ability of a neuron to encode information about stimuli having different spatial characteristics. Here we relate the RF organization of sensory neurons to their ability to encode time-varying stimuli, using linear stimulus estimation, measures of information transfer, and more conventional analysis techniques. The electrosensory systems of weakly electric fish are recognized as very tractable model systems for studies of sensory processing because behaviorally relevant stimuli are generated easily and related to known behaviors and because a detailed anatomical database is available to guide the design and interpretation of experiments. Receptive fields of neurons within the first central electrosensory-processing region have an antagonistic center-surround organization; the RF area varies with cell type, with dendritic morphology, and with the spontaneous activity patterns of the cell. Functional consequences of variations in center-surround organization were assessed by comparing responses to two spatial stimulus patterns that mimic naturalistic stimuli and that provide input to the center alone or to the center plus surround. Measures of the quality of stimulus estimation (coding fraction) and information transmission (mutual information) as well as traditional measures of responsiveness consistently demonstrate that, for cells having large surrounds, the activation of both receptive field components degrades the ability to encode time-varying stimuli. The loss of coding efficiency with center-surround stimulation probably results from cancellation of balanced excitatory and inhibitory inputs. However, cells with small surrounds relative to centers perform well under all spatial stimulus regimes.

Action Potentials↗
PubMedDate not supplied

Clinical and pharmacogenomic data mining: 1. Generalized theory of expected information and application to the development of tools.

New scientific problems, arising from the human genome project, are challenging the classical means of using statistics. Yet quantified knowledge in the form of rules and rule strengths based on real relationships in data, as opposed to expert opinion, is urgently required for researcher and physician decision support. The problem is that with many parameters, the space to be analyzed is highly dimensional. That is, the combinations of data to examine are subject to a combinatorial explosion as the number of possible events (entries, items, sub-records) (a),(b),(c),... per record (a,b,c,..) increases, and hence much of the space is sparsely populated. These combinatorial considerations are particularly problematic for identifying those associations called "Unicorn Events" which occur significantly less than expected to the extent that they are never seen to be counted. To cope with the combinatorial explosion, a novel numerical "book keeping" approach is taken to generate information terms relating to the combinatorial subsets of events (a,b,c,..), and, most importantly, the zeta (Zeta) function is employed. The incomplete Zeta function zeta(s,n) with s = 1, in which frequencies of occurrence such as n = n(a,b,c,...) determine the range of summation n, is argued to be the natural choice of information function. It emerges from Bayesian integration, taken over the distribution of possible values of information measures for sparse and ample data alike. Expected mutual information l(a;b;c) in nats (i.e., natural units analogous to bits but based on the natural logarithm), such as is available to the observer, is measured as e.g., the difference zeta(s,o(a,b,c..)) - zeta(s,e(a,b,c..)) where o(a,b,c,..) and e(a,b,c,..) are, or relate to, the observed and expected frequencies of occurrence, respectively. For real values of s > 1 the qualitative impact of strongly (positively or negatively) ranked data is preserved despite several numerical approximations. As real s increases, and the output of the information functions converge into three values +1, 0, and -1 nats representing a trinary logic system. For quantitative data, a useful ad hoc method, to report sigma-normalized covariations in an analogous manner to mutual information for significance comparison purposes, is demonstrated. Finally, the potential ability to make use of mutual information in a complex biomedical study, and to include Bayesian prior information derived from statistical, tabular, anecdotal, and expert opinion is briefly illustrated.

Clinical Trials as Topic↗
PubMed2003

Information theoretic similarity measures in non-rigid registration.

Mutual Information (MI) and Normalised Mutual Information (NMI) have enjoyed success as image similarity measures in medical image registration. More recently, they have been used for non-rigid registration, most often evaluated empirically as functions of changing registration parameter. In this paper we present expressions derived from intensity histogram representations of these measures, for their change in response to a local perturbation of a deformation field linking two images. These expressions give some insight into the operation of NMI in registration and are implemented as driving forces within a fluid registration framework. The performance of the measures is tested on publicly available simulated multi-spectral MR brain images.

Algorithms↗
PubMed2004

Dynamic analyses of information encoding in neural ensembles.

Neural spike train decoding algorithms and techniques to compute Shannon mutual information are important methods for analyzing how neural systems represent biological signals. Decoding algorithms are also one of several strategies being used to design controls for brain-machine interfaces. Developing optimal strategies to design decoding algorithms and compute mutual information are therefore important problems in computational neuroscience. We present a general recursive filter decoding algorithm based on a point process model of individual neuron spiking activity and a linear stochastic state-space model of the biological signal. We derive from the algorithm new instantaneous estimates of the entropy, entropy rate, and the mutual information between the signal and the ensemble spiking activity. We assess the accuracy of the algorithm by computing, along with the decoding error, the true coverage probability of the approximate 0.95 confidence regions for the individual signal estimates. We illustrate the new algorithm by reanalyzing the position and ensemble neural spiking activity of CA1 hippocampal neurons from two rats foraging in an open circular environment. We compare the performance of this algorithm with a linear filter constructed by the widely used reverse correlation method. The median decoding error for Animal 1 (2) during 10 minutes of open foraging was 5.9 (5.5) cm, the median entropy was 6.9 (7.0) bits, the median information was 9.4 (9.4) bits, and the true coverage probability for 0.95 confidence regions was 0.67 (0.75) using 34 (32) neurons. These findings improve significantly on our previous results and suggest an integrated approach to dynamically reading neural codes, measuring their properties, and quantifying the accuracy with which encoded information is extracted.

Action Potentials↗
PubMed2002

Information-theoretic dissection of pairwise contact potentials.

Pairwise contact potentials have a long, successful history in protein structure prediction. They provide an easily-estimated representation of many attributes of protein structures, such as the hydrophobic effect. In order to improve on existing potentials, one should develop a clear understanding of precisely what information they convey. Here, using mutual information, we quantified the information in amino acid potentials, and the importance of hydropathy, charge, disulfide bonding, and burial. Sampling error in mutual information was controlled for by estimating how much information cannot be attributed to sampling bias. We found the information in amino acid contacts to be modest: 0.04 bits per contact. Of that, only 0.01 bits of information could not be attributed to hydropathy, charge, disulfide bonding, or burial.

Amino Acids↗
PubMed2006

A theory of associating food types with their postingestive consequences.

Animals often face complex and changing food environments. While such environments are challenging, an animal should make an association between a food type and its properties (such as the presence of a nutrient or toxin). We use information theory concepts, such as mutual information, to establish a theory for the development of these associations. In this theory, associations are assumed to maximize the mutual information between foods and their consequences. We show that associations are invariably imperfect. An association's accuracy increases with the length of a feeding session and the relative frequency of a food type but decreases as time delay between consumption and postingestive consequence increases. Surprisingly, the accuracy of an association is independent of the number of additional food types in the environment. The rate of information transfer between novel foods and a forager depends on the forager's diet. In light of this theory, an animal's diet may have two competing goals: first, the provision of an appropriate balance of nutrients, and second, the ability to quickly and accurately learn the properties of novel foods. We discuss the ecological and behavioral implications of making associational errors and contrast the timescale and mechanisms of our theory with those of existing theory.

Animal Nutritional Physiological Phenomena↗
PubMed2006

Comparative analysis of base correlations in 5' untranslated regions of various species.

Translational initiation signals, such as Shine-Dalgarno (SD) sequences in bacteria and Kozak consensus sequences in vertebrates, direct ribosomes to initiate protein synthesis from mRNAs. Investigating sequence characteristics of these signals is important, particularly to infer translational initiation mechanisms. Although various statistical analyses of translational initiation signals have been done, few have focused on base correlations that assess base dependencies in the signal sequences. We used relative entropy and mutual information to analyze base conservation and correlation, respectively, in the 5' UTRs of various species. In eukaryotes, we found peaks of relative entropy at -3 from the translational start site but no peak of mutual information at that position, indicating that the base at that position (known as the core base of the Kozak sequence) is well conserved but not correlated with neighboring bases and thus functions as a single base. We observed unexpected peaks of mutual information between positions -2 and -1 in most eukaryotes. Surprisingly these base correlation also occurred in some bacteria and archaea, although there were no base preferences at neither position. Various dinucleotide patterns existed at these positions, and the correlation between bases at -2 and -1 may be relevant to the context of translational initiation. Because dinucleotide patterns of correlated pairs of nucleotides at -2 and -1 were not unique within respective organisms, the correlation could not be found when analyzing single-nucleotide conservation. Therefore, mutual information allowed us to discover signals that were not found by simply analyzing base conservation.

5' Untranslated Regions↗
PubMed2004

The integration of multiple stimulus features by V1 neurons.

We investigated how V1 neurons integrate two stimulus features by presenting stimuli from a stimulus set made up of all combinations of eight different directions of motion and nine binocular disparities. We investigated the occurrence and shape of the resulting joint tuning function. Among V1 neurons, approximately 80% were jointly tuned for disparity with orientation or direction. The joint tuning function of all jointly tuned neurons was separable into distinct tuning for disparity on the one hand, and orientation or direction tuning on the other. The degree of separability and the mutual information between the stimulus and the firing rates were strongly correlated. The mutual information of jointly tuned neurons when both features were decoded together was highly correlated with the mutual information when the two features were decoded separately, and the information was then summed. Jointly tuned neurons were just as good at representing information about single features as neurons tuned for only a single feature. The tuning properties of most jointly tuned neurons did not dynamically evolve over time, nor did jointly tuned neurons respond earlier than neurons tuned for only a single feature. The response selectivity of V1 neurons is low and decreases the information that a neuron represents about a stimulus. Together these results suggest that distinct stimulus features are integrated very early in visual processing. Furthermore, V1 generates a distributed representation through low response selectivity that avoids the curse of dimensionality by using separable joint tuning functions.

Animals↗
PubMed1979

[The correlation between autoregressive power spectrum of EEG and computerized tomography in tuberous sclerosis (author's transl)].

The autoregressive power spectrum and their component analyses of 36 electroencephalograms of 9 patients with tuberous sclerosis and 21 healthy children were studied. The data were recorded on the analog tapes, and the 20 second artifact free segment of records was digitized at 50 samples/sec. The autoregressive power spectrum and their component were calculated by the methods of Sato (1976) with the minicomputer PDP 11/40. The component consisted of the first and the second order elementary processes. The former showed a transient nonscillatory delta wave, whereas the latter showed damped oscillatory waves of delta, theta, alpha and beta rhythms in the EEG. The characteristics of these component rhythms in the EEG were given by the frequency, the time constant of the nonoscillatory delta, the damping time of the oscillatory component waves (time constant of the envelope of the damped oscillation), the mutual information amounts, etc. Thus, the correlation between these characteristics of component and CT-scan in tuberous sclerosis were examined. The results were as follows: 1. Compared with the characteristics of EEG in normal children, the mutual information amount, the damping time and/3r time constant showed significantly lower value in alpha rhythms of frontal-, central-regions and in theta, delta rhythms of occipital region in the patient with tuberous sclerosis. 2. Multiple subependymal high density areas were found in all of these patients on CT. EMI-number of these high density areas were less than that of calcification in younger children below the age of 3 years, but in older children were equivalent to that of calcification. 3. The correlation between the number of subependymal nodule on CT and the mutual information amount in the EEG showed significantly the negative coefficient in theta, delta rhythms of O1 region in older age group of patients. 4. The correlation between EMI-number of these subependymal high density areas and the mutual information amount in the EEG showed significantly thenegative coefficient in theta & delta rhythms of O1 region in all cases of patients. 5. It is considered that the multiple subependymal nodules may influence the background activities of the central and the occipital regions in the EEG of tuberous sclerosis.

Child↗
PubMed2003

Exceptional pairs of amino acid neighbors in alpha-helices.

Amino acids seem to have specific preferences for various locations in alpha-helices. These specific preferences, called singlet local propensity (SLP), have been determined by calculating the preference of occurrence of each amino acid in different positions of the alpha-helix. We have studied the occurrence of amino acids, single or pairs, in different positions, singlet or doublet, of alpha-helices in a database of 343 non-homologous proteins representing a unique superfamily from the SCOP database with a resolution better than 2.5 A from the Protein Data Bank. The preference of single amino acids for various locations of the helix was shown by the relative entropy of each amino acid with respect to the background. Based on the total relative entropy of all amino acids occurring in a single position, the N(cap) position was found to be the most selective position in the alpha-helix. A rigorous statistical analysis of amino acid pair occurrences showed that there are exceptional pairs for which, the observed frequency of occurrence in various doublet positions of the alpha-helix is significantly different from the expected frequency of occurrence in that position. The doublet local propensity (DLP) was defined as the preference of occurrences of amino acid pairs in different doublet positions of the alpha-helix. For most amino acid pairs, the observed DLP (DLP(O)) was nearly equal to the expected DLP (DLP(E)), which is the product of the related SLPs. However, for exceptional pairs of amino acids identified above, the DLP(O) and DLP(E) values were significantly different. Based on the relative values of DLP(O) and DLP(E), exceptional amino acid pairs were divided into two categories. Those, for which the DLP(O) values are higher than DLP(E), should have a strong tendency to pair together in the specified position. For those pairs which the DLP(O) values are less than DLP(E), there exists a hindrance in neighboring of the two amino acids in that specific position of the alpha-helix. These cases have been identified and listed in various tables in this paper. The amount of mutual information carried by the exceptional pairs of amino acids was significantly higher than the average mutual information carried by other amino acid pairs. The average mutual information conveyed by amino acid pairs in each doublet position was found to be very small but non-zero.

Amino Acid Sequence↗
PubMed2004

Motion tracking in MR-guided liver therapy by using navigator echoes and projection profile matching.

RATIONALE AND OBJECTIVES: Image registration in magnetic resonance (MR) image-guided liver therapy enhances surgical guidance by fusing preoperative multimodality images with intraoperative images, or by fusing intramodality images to correlate serial intraoperative images to monitor the effect of therapy. The objective of this paper is to describe the application of navigator echo and projection profile matching to fast two-dimensional image registration for MR-guided liver therapy. MATERIALS AND METHODS: We obtain navigator echoes along the read-out and phase-encoding directions by using modified gradient echo imaging. This registration is made possible by masking out the liver profile from the image and performing profile matching with cross-correlation or mutual information as similarity measures. The set of experiments include a phantom study with a 2.0-T experimental MR scanner, and a volunteer and a clinical study with a 0.5-T open-configuration MR scanner, and these evaluate the accuracy and effectiveness of this method for liver therapy. RESULTS: Both the phantom and volunteer study indicate that this method can perform registration in 34 ms with root-mean-square error of 1.6 mm when the given misalignment of a liver is 30 mm. The clinical studies demonstrate that the method can track liver motion of up to approximately 40 mm. Matching profiles with cross-correlation information perform better than with mutual information in terms of robustness and speed. CONCLUSION: The proposed image registration method has potential clinical impact on and advantages for MR-guided liver therapy.

Echo-Planar Imaging↗
PubMed2004

Modeling genetic networks from clonal analysis.

In this report a systematic approach is used to determine the approximate genetic network and robust dependencies underlying differentiation. The data considered is in the form of a binary matrix and represent the expression of the nine genes across the 99 colonies. The report is divided into two parts: the first part identifies significant pair-wise dependencies from the given binary matrix using linear correlation and mutual information. A new method is proposed to determine statistically significant dependencies estimated using the mutual information measure. In the second, a Bayesian approach is used to obtain an approximate description (equivalence class) of network structures. The robustness of linear correlation, mutual information and the equivalence class of networks is investigated with perturbation and decreasing colony number. Perturbation of the data was achieved by generating bootstrap realizations. The results are refined with biological knowledge. It was found that certain dependencies in the network are immune to perturbation and decreasing colony number and may represent robust features, inherent in the differentiation program of osteoblast progenitor cells. The methods to be discussed are generic in nature and not restricted to the experimental paradigm addressed in this study.

Animals↗