Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “multimodal integration”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9Linked to original sources
PubMed2003

Evaluation of neuronavigation in lesional and non-lesional epilepsy surgery.

OBJECTIVE: For many patients, surgery for intractable epilepsy provides not only freedom or substantial relief from seizures, but also functional improvement and increased quality of life. Precise intraoperative localization of the underlying structural and functional processes is crucial in this regard. The aim of this study was to clinically evaluate whether neuronavigation leads to an improvement in the precision and safety of epilepsy surgery. In this paper, we also attempt to assess the navigation workstation as a platform for the integration of multimodal information (multimodal information guidance). PATIENTS: Out of a series of 223 epilepsy surgery procedures, 140 were performed with the aid of neuronavigation. Patient and surgical data were prospectively collected. METHODS: We used the neuronavigation device as a common platform to merge complementary information modalities. Correlation of anatomic and structural details with functional information contributed to the surgical script in non-lesional and localization-related epilepsy surgery. At least two different information modalities contributed to planning and surgical guidance in every patient. Immediately following the operative procedure, the surgeon answered a set of questions on the reasons for the application of neuronavigation, and the efficiency and safety of navigation. Detailed analysis of the location of the operative procedure, histopathological findings and outcome was performed. RESULTS: The main benefits of neuronavigation in epilepsy surgery were precision of targeting even in small and deep-seated targets, safe manipulation in critical brain areas, accurate placement of electrodes, and correlation of electro-clinical information modalities with underlying structures. Furthermore, navigation provided individual tailoring of craniotomy and corticotomy. It was less reliable for verification of resection boundaries in the case of underlying glioma. Neuronavigational localization and its combination with image fusion and functional investigations greatly improved discussion within the epilepsy surgery team. CONCLUSION: The neuronavigation concept proved its value in epilepsy surgery by linking anatomic, pathologic and functional data of the individual patient. Enhanced by the integration of multimodal information, neuronavigation significantly improved the available treatment options.

Adolescent↗
PubMed2003

Multimodal visual-somatosensory integration in saccade generation.

Neurophysiological studies have demonstrated multisensory interaction effects in the neural structures involved in saccade generation when visual, auditory or somatosensory stimuli are presented bimodally. Visual-auditory interaction effects have been demonstrated in numerous behavioural studies of saccades but little is known about interaction effects involving somatosensory stimuli. The present study examined visual-somatosensory interaction effects on saccade generation using a multisensory paradigm, whereby task-irrelevant distractors appeared spatially-coincident with, or remote from the designated saccade target. Somatosensory distractors reduced the latency of saccades when presented before the visual target and the greatest facilitation effect was observed with spatially-coincident stimuli. Visual distractors spatially-coincident with a somatosensory target reduced latency (and increased peak velocity) when presented before and after the target. Visual distractors contralateral to somatosensory targets increased saccade latency and produced high error rates of saccades made to the distractor. The high error rates and latency modulation with visual distractors is consistent with a bias for visual stimuli in the saccadic system. In the visual target condition, saccade latency was modulated by a somatosensory distractor that was entirely task-irrelevant and this effect was always greatest with spatially-coincident distractors. The multisensory distractor effects are discussed in terms of saccades being programmed to the non-target modality, the early triggering of a non-spatial saccade 'when' signal, and multisensory neuronal enhancement effects.

Acoustic Stimulation↗
PubMed1997

Integration of surgery in multimodality therapy for esophageal cancer.

BACKGROUND: While adding chemotherapy to radiation for the treatment of esophageal cancers has been shown to be beneficial, surgery usually follows treatment or is omitted. In either case, regional control remains problematic. The purpose of this study was to test the feasibility of using chemotherapy and radiation following surgery in the treatment of of esophageal cancer and to assess the impact of this approach on regional control and survival. PATIENTS AND METHODS: Twenty-five patients with esophageal cancer were treated in a phase I pilot protocol consisting of initial esophagectomy with gastroesophagostomy and subsequent combined chemotherapy and radiation. Chemotherapy consisted of cisplatin given on day 1 and 5-fluorouracil (FU) on days 1-5 by continuous infusion. Radiation therapy was administered in varying fractionation schedules of once or twice daily concomitantly with the chemotherapy. Treatment was repeated every other week for two to four cycles. Median follow-up was 42 months. RESULTS: Acute toxicities (mucositis and cytopenias) were common but not worse than grade 3. Higher doses of 50 Gy with 2 Gy b.i.d. hyperfractionation caused late complications in four of 10 patients, (two lethal). Control of local disease for all patients was excellent with only two known and two possible local recurrences (16%) but distant metastases were common (46%). Disease-free survival was 58 and 30% at 1 and 2 years, respectively. Survival was 58 and 32% at 1 and 2 years, respectively (median survival, 19 months). CONCLUSION: The local control rate and survival were better than those in our historical experience with cisplatin and 5-FU chemotherapy and radiation given prior to surgery. A dose-fractionation schedule of < 2 Gy up to a total of 50 Gy b.i.d. is recommended to avoid late adverse effects. The role of surgery will be defined by randomized studies. Better systemic therapy is needed to impact on systemic failure.

Adenocarcinoma↗
PubMed2001

Multimodal cranial neuronavigation: direct integration of functional magnetic resonance imaging and positron emission tomography data: technical note.

OBJECTIVE: This is the first report of the direct integration of functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) data into cranial neuronavigation. METHODS: In a patient with a left precentral oligodendroglioma (World Health Organization Grade III), the Zeiss MKM system (Carl Zeiss Co., Oberkochen, Germany) was used for navigation based on thin-slice, T1-weighted, contrast-enhanced magnetic resonance imaging (MRI) scans. fMRI and methionine PET data were integrated by landmark matching, with reference to skin fiducials. RESULTS: The inaccuracy of the image fusion between fMRI and T1-weighted MRI data was 1.7 mm, that between PET and T1-weighted MRI data was 4.3 mm, and that for the subsequent registration of the navigation was 1.2 mm. The correct fMRI localization of the precentral gyrus was intraoperatively verified by cortical somatosensory evoked potential (phase-reversal) monitoring. Although the tumor was not clearly defined in the MRI scans, [11C]methionine PET demonstrated a clear tumor border, enabling us to achieve gross total tumor removal without postoperative functional deficits. CONCLUSION: Functional neuronavigation permits observation and preservation of relevant brain areas. Other functional areas (such as short-term memory areas) that can be detected only by fMRI might also warrant future monitoring. The simultaneous integration of fMRI and PET data adds a new dimension to cranial neuronavigation, enabling the observation of tumors in relation to functional cortical areas (in our case, the motor strip).

Brain↗
PubMed2003

Mode-order converter in a multimode waveguide.

A novel device that converts the order of modes in an integrated-optic multimode waveguide was proposed and fabricated. The device consists of two mode splitters and tapered waveguides. Its operation at a wavelength of 1.55 microm was confirmed by observation of the output mode profiles. The cross talk measured by an indirect method was >10.2 dB. The excess loss with reference to a straight multimode waveguide was 1.5 dB, compared with the propagation loss of a straight multimode waveguide.

Journal Article↗
PubMed2026

Artificial Intelligence for Colorectal Surgeons-Part II: Research Applications, Challenges in Adoption, and Practical Resources.

BACKGROUND: This is part II of a 2-part series examining artificial intelligence in colorectal surgery. Part I established foundational concepts and clinical applications. Implementation, however, requires understanding research methodologies, available resources, and the specific challenges currently limiting widespread adoption. These topics are the focus of part II. OBJECTIVE: To examine artificial intelligence's transformation of surgical research, provide practical implementation resources, address adoption challenges, and explore future directions in colorectal surgery. METHODS: Comprehensive literature review focusing on artificial intelligence research methodology, implementation barriers, educational resources, and emerging technologies relevant to colorectal surgeons. RESULTS: Artificial intelligence streamlines clinical trial design through predictive modeling and natural language processing, reducing enrollment challenges that contribute to failed or inadequate trial accrual. Machine learning enables heterogeneity analysis within clinical trials, identifying treatment-responsive subgroups. Foundation models unlock analysis of unstructured electronic health record data at scale. Professional societies and universities offer specialized artificial intelligence education programs, with open-access data sets facilitating research participation. However, implementation faces multifaceted challenges: technical infrastructure demands, with real-time processing requiring dedicated graphics processing unit clusters; regulatory frameworks struggling with continuously evolving algorithms; undefined liability distribution for artificial intelligence-assisted decisions; algorithmic bias risking health care disparities; and the "black box" problem limiting clinical trust. Economic barriers include substantial initial costs without clear reimbursement pathways. Future directions include multimodal artificial intelligence integrating imaging, genomics, and histopathology; cognitive robotic systems with real-time decision support; digital twin technology for patient-specific surgical simulation; and global surgical artificial intelligence networks enabling distributed learning across institutions. CONCLUSIONS: Although artificial intelligence offers transformative potential for colorectal surgery research and practice, successful implementation requires addressing technical, regulatory, ethical, and economic challenges. The surgeon's evolving role demands both traditional expertise and computational fluency. Future advances in multimodal integration, autonomous systems, and global collaboration will fundamentally reshape surgical practice but will require thoughtful implementation prioritizing patient benefit and clinical value.

Humans↗
PubMed2026

Longitudinal Clinical, Physiological, and Molecular Profiling of Female Patients With Metastatic Cancer: Protocol and Feasibility of a Multicenter High-Definition Oncology Study.

PURPOSE: A substantial proportion of patients receiving genomically matched therapies do not achieve clinical benefit, underscoring the influence of nongenetic factors on cancer outcomes. High-Definition Oncology (HDO) proposes integrating longitudinal, multimodal patient data-spanning clinical, molecular, physiological, and behavioral domains-to enable truly individualized cancer care. This manuscript describes the HDO study design, framework, and feasibility results in women with metastatic cancer. METHODS: We initiated a prospective, multicenter observational study (HDO study; ClinicalTrials.gov identifier: NCT06590506) enrolling 300 female patients with newly diagnosed metastatic breast, lung, or colorectal cancer. Here, we report the study design, standardized workflows, prespecified feasibility criteria, and early internal pilot results. Eleven data modalities are collected longitudinally, including tumor and germline genomics, germline epigenomics, gut microbiome, blood and stool metabolomics and proteomics, exposome characterization, wearable-derived physiological monitoring, digital footprint assessment, medical imaging, and patient-reported outcomes. Standardized workflows govern clinical procedures, data acquisition, biospecimen processing, and quality control across all participating sites. RESULTS: Feasibility was evaluated in the first 30 participants (10% of planned accrual). Patients completed 100% of scheduled clinical visits, 97.4% of planned plasma collections, 80.7% of stool samples, and all tumor biopsies. Wearable devices captured activity, heart rate, sleep, and blood oxygen saturation data during 95.0%, 84.2%, 90.6%, and 70.7% of total patient-days, respectively. Biospecimens met predefined quality control metrics across all molecular modalities. Engagement with mobile applications for pain and emotion reporting exceeded 80%. CONCLUSION: The HDO study demonstrates the feasibility of comprehensive, longitudinal, multimodal data collection in women with metastatic cancer. This internal pilot establishes an integrated framework for future analyses aimed at characterizing disease trajectories, defining molecular and physiological determinants of outcomes, and developing patient-specific computational models.

Humans↗
PubMed2005

Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions.

Processing of complex visual stimuli comprising facial movements, hand actions, and body movements is known to occur in the superior temporal sulcus (STS) of humans and nonhuman primates. The STS is also thought to play a role in the integration of multimodal sensory input. We investigated whether STS neurons coding the sight of actions also integrated the sound of those actions. For 23% of neurons responsive to the sight of an action, the sound of that action significantly modulated the visual response. The sound of the action increased or decreased the visually evoked response for an equal number of neurons. In the neurons whose visual response was increased by the addition of sound (but not those neurons whose responses were decreased), the audiovisual integration was dependent upon the sound of the action matching the sight of the action. These results suggest that neurons in the STS form multisensory representations of observed actions.

Acoustic Stimulation↗
PubMed1993

Stereotactic insertion of intracerebral electrodes in the investigation of epilepsy.

Intracerebral electroencephalography is sometimes necessary in the pre-surgical evaluation of patients with drug-resistant epilepsy, in order to define the site(s) of seizure onset. Accurate depth electrode insertion would ideally require multimodal image integration, repetitive and non-invasive localization, assessment of positional accuracy and freedom in target and entry point selection. A method for the acquisition of the multimodal image stereotactic information, necessary for the precise targeting of the mesial temporal structures, and the operative technique used for the accurate placement of the intracerebral electrodes is described. The use of the Gill-Thomas (G-T) stereotactic repeat localizer offers the degree of temporal freedom during the data acquisition period of the electrode implantation procedure which leads to the advantage of an unhurried multi-image integration and targeting in any individual case combined with less discomfort for the patient. The integration of the G-T repeat localizing system with the existing components of the Cosman-Roberts-Wells (CRW-3) stereotactic system offers additional advantages. The target-centered arc-radius design allows complete freedom in the selection of the entry point, offers the possibility of multiple trajectories through the same entry point and permits a choice of either oblique or orthogonal lateral approaches, that are particularly useful for the electroencephalographic sampling of the posterior mesial temporal structures. The satisfactory localizing value of the technique and the additional advantage of easy confirmation of the positional accuracy of the electrodes during the postoperative period, using standard radiographic and magnetic resonance imaging studies, increases the potential of the technique for precise placement of depth electrodes during the pre-surgical evaluation of patients with epilepsy.

Amygdala↗
PubMed2025

A Graph Contrastive Learning Method for Enhancing Genome Recovery in Complex Microbial Communities.

Accurate genome binning is essential for resolving microbial community structure and functional potential from metagenomic data. However, existing approaches-primarily reliant on tetranucleotide frequency (TNF) and abundance profiles-often perform sub-optimally in the face of complex community compositions, low-abundance taxa, and long-read sequencing datasets. To address these limitations, we present MBGCCA, a novel metagenomic binning framework that synergistically integrates graph neural networks (GNNs), contrastive learning, and information-theoretic regularization to enhance binning accuracy, robustness, and biological coherence. MBGCCA operates in two stages: (1) multimodal information integration, where TNF and abundance profiles are fused via a deep neural network trained using a multi-view contrastive loss, and (2) self-supervised graph representation learning, which leverages assembly graph topology to refine contig embeddings. The contrastive learning objective follows the InfoMax principle by maximizing mutual information across augmented views and modalities, encouraging the model to extract globally consistent and high-information representations. By aligning perturbed graph views while preserving topological structure, MBGCCA effectively captures both global genomic characteristics and local contig relationships. Comprehensive evaluations using both synthetic and real-world datasets-including wastewater and soil microbiomes-demonstrate that MBGCCA consistently outperforms state-of-the-art binning methods, particularly in challenging scenarios marked by sparse data and high community complexity. These results highlight the value of entropy-aware, topology-preserving learning for advancing metagenomic genome reconstruction.

canonical correlation analysis↗
PubMedDate not supplied

From multimodality digital imaging to multimedia patient record.

The constant improvement in computer power and performance nowadays offers convenient and efficient means of manipulating images, graphics, and movies on off-the-shelf workstations. With this improvement the trend toward integration of multimodality clinical documents from patient records comes naturally. Images and graphs are certainly the most important part of the complementary information that must accompany the text and numerical data. It is, however, possible to include sounds and voice messages together with all the other modalities. In medicine that could certainly help conveying hart murmur or sounds, but could also offer a convenient way of including vocal messages and comments. These new possibilities will certainly change the way physicians use workstations for direct communication. The computer industry will soon offer means of interactive communication between remote users through computer workstations. That alone will open a completely new era in cooperative computing and remote consultation scenarios in medicine. More than the technology itself, a complete change in behavior and work habits can be expected in the medical community.

Computer Communication Networks↗
PubMed2004

Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition.

The ability of organisms to discriminate social signals, such as affective displays, using different sensory modalities is important for social communication. However, a major problem for understanding the evolution and integration of multimodal signals is determining how humans and animals attend to different sensory modalities, and these different modalities contribute to the perception and categorization of social signals. Using a matching-to-sample procedure, chimpanzees discriminated videos of conspecifics' facial expressions that contained only auditory or only visual cues by selecting one of two facial expression photographs that matched the expression category represented by the sample. Other videos were edited to contain incongruent sensory cues, i.e., visual features of one expression but auditory features of another. In these cases, subjects were free to select the expression that matched either the auditory or visual modality, whichever was more salient for that expression type. Results showed that chimpanzees were able to discriminate facial expressions using only auditory or visual cues, and when these modalities were mixed. However, in these latter trials, depending on the expression category, clear preferences for either the visual or auditory modality emerged. Pant-hoots and play faces were discriminated preferentially using the auditory modality, while screams were discriminated preferentially using the visual modality. Therefore, depending on the type of expressive display, the auditory and visual modalities were differentially salient in ways that appear consistent with the ethological importance of that display's social function.

Acoustic Stimulation↗
PubMed2005

Multimodal analgesia techniques and postoperative rehabilitation.

The concept of multimodal analgesia involves the use of different classes of analgesics and different sites of analgesic administration to provide superior dynamic pain relief with reduced analgesic-related side effects. Although multimodal analgesia techniques have assumed increasing importance in the management of perioperative pain, it has become increasingly apparent that postoperative outcome may not be improved. Nevertheless, the integration of multimodal analgesia techniques with a multimodal and multidisciplinary rehabilitation program may enhance recovery, reduce hospital stay, and facilitate early convalescence.

Analgesia↗
PubMed2025

Diagnosing the undiagnosed: AI-enhanced multimodal modeling for placental mesenchymal dysplasia in high-risk pregnancies.

Placental mesenchymal dysplasia (PMD) is a rare vascular placental disorder that mimics molar pregnancy but often coexists with a viable fetus, making its misdiagnosis potentially devastating. In high-risk pregnancies, artificial intelligence (AI)-enhanced multimodal modeling - incorporating imaging, genomics, proteomics, and clinical features - offers a transformative diagnostic strategy. Leveraging Bayesian hyperparameter optimization for model refinement, this approach improves diagnostic accuracy while reducing uncertainty and clinician hesitation. Recent clinical studies support its efficacy and interpretability through SHAP and LIME models, while real-time surgical enhancements using Bayesian methods highlight its broader clinical utility. Despite current challenges such as data heterogeneity and integration barriers, multimodal AI provides unprecedented resolution in placental analysis, enabling precise differentiation between PMD and similar fetopathies. Ultimately, this advancement supports timely, non-invasive diagnosis, personalized management, and emotionally informed decision-making aligned with ethical AI implementation standards.

Bayesian optimization↗
PubMed2026

CAUSAL artificial intelligence and data-driven decision intelligence in personalized medicine: a review of healthcare informatics systems.

This review examines the integration of causal artificial intelligence (AI) and data-driven decision intelligence within healthcare informatics systems to advance personalized medicine and clinical decision-making. A narrative review methodology was employed, synthesizing interdisciplinary literature from major databases, including PubMed, Scopus, Web of Science, IEEE Xplore, and ScienceDirect. Studies focusing on causal inference, decision intelligence, and healthcare informatics applications in personalized medicine were included. Data were extracted on methodological approaches, healthcare settings, analytical techniques, and clinical applications, followed by thematic synthesis. Findings indicate that causal AI enhances clinical decision support by enabling estimation of treatment effects and simulation of intervention outcomes at the individual patient level. Integration of multimodal health data such as electronic health records, genomic data, and real-time monitoring improves prediction accuracy and supports tailored treatment strategies. Additionally, causal models improve interpretability, fostering clinician trust and facilitating transparent decision-making. Robust healthcare informatics infrastructures, including interoperable systems and data warehouses, were identified as critical enablers of causal analytics. Overall, causal AI represents a transformative advancement in healthcare analytics, supporting more informed, individualized, and evidence-based clinical decisions. Its integration within healthcare informatics systems has significant potential to improve patient outcomes and guide the future of intelligent, personalized healthcare delivery.

Precision Medicine↗
PubMed2001

Activation in the ipsilateral posterior parietal cortex during tool use: a PET study.

The basis of perceptual assimilation of tool and hand has been considered to be in modification of body schemata, for which integration of multimodal sensory information about our body parts is required. Using positron emission tomography and H(2)(15)O, we aimed to identify brain regions that change their neural activity in association with changes in neural processing of visual and/or somatosensory information when humans use a simple tool. Normal subjects were instructed to manipulate a small graspable object with a pair of tongs or with the fingers of their right or left hand. The only site activated during manipulation with the tool, compared with the fingers, with the right hand was the lateral edge of the right intraparietal sulcus (IPS). During manipulation using the left hand with the tool, compared with using the fingers, an area in the middle part of the left IPS was activated. Areas in the contralateral hemisphere were activated during both the tool-use and the finger-use tasks compared to the control task, but there was no statistically significant difference between the tool-use and the finger-use tasks. Therefore, the results suggest that the ipsilateral posterior parietal cortex was recruited during the tool-use tasks to integrate visuosomatosensory information.

Adult↗
PubMed2025

JASMINE: A powerful representation learning method for enhanced analysis of incomplete multi-omics data.

Integrative analysis of multi-omics data provides a more comprehensive and nuanced view of a subject's biological state. However, high-dimensionality and ubiquitous modality missingness present significant analytical challenges. Existing methods for incomplete multi-omics data are scarce, do not fully leverage both modality-specific and shared information, and produce task-biased representations. We propose JASMINE, a self-supervised representation learning method for incomplete multi-omics data that preserves both modality-specific and joint information and enhances sample similarity structure. JASMINE produces embeddings that achieve superior performance across multiple tasks for two different incomplete multi-omics datasets while requiring only a single round of training per dataset.

missing data↗
PubMed1999

Larval and pupal development of the mushroom bodies in the honey bee, Apis mellifera.

The mushroom bodies are paired neuropils in the insect brain that act as multimodal sensory integration centers and are involved in learning and memory. Our studies, by using 5-bromo-2-deoxyuridine incorporation and the Feulgen technique, show that immediately before pupation, the brain of the developing honey bee (Apis mellifera) contains approximately 2,000 neuroblasts devoted to the production of the mushroom body intrinsic neurons (Kenyon cells). These neuroblasts are descended from four clusters of 45 or fewer neuroblasts each already present in the newly hatched larva. Subpopulations of Kenyon cells, distinct in cytoarchitecture, position, and immunohistochemical traits, are born at different, but overlapping, periods during the development of the mushroom bodies, with the final complement of these neurons in place by the mid-pupal stage. The mushroom bodies of the adult honey bee have a concentric arrangement of Kenyon cell types, with the outer layers born first and pushed to the periphery by later born neurons that remain nearer the center of proliferation. This concentricity is further reflected in morphologic and immunohistochemical traits of the adult neurons, and is demonstrated clearly by the pattern of expression of Drosophila myocyte enhancer factor 2 (DMEF2)-like immunoreactivity. This is the first comprehensive study of larval and pupal development of the honey bee mushroom bodies. Similarities to patterns of neurogenesis observed in the mushroom bodies of other insects and in the vertebrate cerebral cortex are discussed.

Animals↗