Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “multimodal integration”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8Linked to original sources
PubMed1992

Specialized extended hospital treatment for borderline patients.

The intensive psychoanalytically oriented hospital treatment described by the authors was developed specifically for borderline patients. This unique treatment, an integrated multimodal approach, is based on the theory that the disturbed behaviors and relationships of borderline patients are manifestations of intrapsychic impairment. The authors use a detailed case report to illustrate how the theory is transformed into technical precepts and applied to all aspects of the treatment.

Activities of Daily Living↗
PubMedDate not supplied

[New aspects of complex chronic tinnitus. I: Assessment of a multi-modality behavioral medicine treatment concept].

"Complex tinnitus" is a diagnostic term denoting a disturbance pattern where the patient hears highly annoying and painful noises or sounds that do not originate from a recognisable external source and can be described only by the patient himself. It seems that the suffering mainly depends upon the extent to which the tinnitus is experienced as a phenomenon that is beyond control. Part I reports on an examination of the treatment success achieved with 28 consecutive patients who had been treated according to an integrative multimodal behavioural medicine concept. This resulted--despite continual loudness--in a decrease in the degree of unpleasantness of the tinnitus, by 17% (p less than 0.01) with corresponding normalisation of decisive symptom factors in Hopkins Symptom-Check-List (SCL-90-R) and Freiburg Personality-Inventary (FPI-R). On the whole, 19 out of the total of 28 patients showed essential to marked improvement of the disturbance pattern. Part II presents a multidimensional tinnitus model and the essential psychotherapeutic focal points of a multimodal psychotherapy concept in complex chronic tinnitus, as well as the parallel phenomena in the chronic pain syndrome.

Adult↗
PubMedDate not supplied

[New aspects of complex chronic tinnitus. II: The lost silence: effects and psychotherapeutic possibilities in complex chronic tinnitus].

"Complex tinnitus" is a diagnostic term denoting a disturbance pattern where the patient hears highly annoying and painful noises or sounds that do not originate from a recognisable external source and can be described only by the patient himself. It seems that the suffering mainly depends upon the extent to which the tinnitus is experienced as a phenomenon that is beyond control. Part I reports on an examination of the treatment success achieved with 28 consecutive patients who had been treated according to an integrative multimodal behavioural medicine concept. This resulted--despite continual loudness--in a decrease in the degree of unpleasantness of the tinnitus, by 17% (p less than 0.01) with corresponding normalisation of decisive symptom factors in Hopkins-Symptom-Check-List (SCL-90-R) and Freiburg Personality-Inventary (FPI-R). On the whole, 19 out of the total of 28 patients showed essential to marked improvement of the disturbance pattern. Part II presents a multidimensional tinnitus model and the essential psychotherapeutic focal points of a multimodal psychotherapy concept in complex chronic tinnitus, as well as the parallel phenomena in the chronic pain syndrome.

Behavior Therapy↗
PubMed1983

[Combined radiotherapy and chemotherapy for the treatment of esophageal cancer].

Eight patients with esophageal cancer were treated by a new treatment schedule consisting of low dose irradiation, crescendo cisplatin and bleomycin polyacrylate pasta. As monitored endoscopically, therapeutic responses were satisfactory: seven out of 8 patients have survived for a range of 3 to 20 months and still active at work or cancer-free. However, one patient suffered from a second malignancy of adenocarcinoma of the upper esophagus different from the initial squamous cell carcinoma at the lower esophagus which had successfully been treated 3 months before. The present therapeutic design aims at treatment of lymphatic spreads in the adjacent structures as well as the original tumor in the esophagus and submucosal invasions. It is basically a consecutive, multimodal integration of selective concentration of therapeutic effects (extensive radiotherapy, topical application of bleomycin polyacrylate pasta, lymphatic chasing with colloidal bleomycin, and spatial concentration of cisplatin as the result of radiation-induced inflammation), perpetuation of the repairable DNA damage, and biological amplifications (protection against esophageal perforation with polyacrylate coating, and specific cancer cell recruitment). Application of the present therapeutic design is being expanded to the treatment of cancer of other specific sites such as the head and neck tumors and rectal cancer with undeniable prospects.

Adenocarcinoma↗
PubMed1994

Regional brain abnormalities in schizophrenia measured with magnetic resonance imaging.

OBJECTIVE: To determine general and regional indices of structural brain abnormality in schizophrenia. DESIGN: Case-control comparison study. SUBJECTS: Fifty-two patients diagnosed as having schizophrenia according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition, were compared with 90 healthy volunteers recruited from the community. MEASUREMENTS: Structural brain images were acquired using magnetic resonance; measurements were obtained using three-dimensional visualization of volume-rendered brains and an automated atlas-based dissection of specific regions. General measures included the volume of total brain tissue, total cerebrospinal fluid (CSF), and CSF within the ventricular system. Regional measures included the volume of tissue and CSF in the frontal, temporal, parietal, and occipital lobes and the cerebellum. RESULTS: Compared with the controls, the patients had a smaller average volume of total brain tissue and a greater average volume of total and ventricular CSF. A specific relative decrease in brain tissue was found only in the frontal lobes, although the volume of CSF was greater in patients than in controls in all brain regions. CONCLUSION: In addition to the generalized brain abnormalities observed in schizophrenia, a regional abnormality may be present in frontal regions. Since the frontal lobes integrate multimodality information and perform a variety of "higher" cognitive and emotional functions that are impaired in schizophrenia, the frontal abnormality noted is consistent with the clinical presentation of the illness. Impaired frontal function and a disruption in its complex circuitry (including thalamocortical projections) may explain why patients with schizophrenia often have significant deficits in formulating concepts and organizing their thinking and behavior.

Adult↗
PubMed1998

Golgi cells in the superficial granule cell domain overlying the ventral cochlear nucleus: morphology and electrophysiology in slices.

Golgi cells are poised to integrate multimodal influences by participating in circuits involving granule cells in the cochlear nuclei. To understand their physiological role, intracellular recordings were made from anatomically identified Golgi cells in slices of the cochlear nuclei from mice. Cell bodies, dendrites, and terminals for all seven labeled cells were restricted to the narrow plane of the superficial granule cell domain over the ventral cochlear nucleus. The axonal arborization was the most striking feature of all Golgi cells; a dense plexus of terminals covered an area 200-400 microm in diameter in the vicinity of the cell body and dendrites. Axonal beads often surrounded granule cell bodies, indicating that granule cells are probable targets. Cells had input resistances up to 130 M omega and fired regular, overshooting action potentials. Golgi cells probably receive auditory nerve input, because shocks to the cut end of the auditory nerve excited Golgi cells with excitatory postsynaptic potentials (EPSPs). The latency of EPSPs shortened to a minimum and the amplitude of EPSPs grew in several steps as the strength of shocks was increased. The minimum latency of EPSPs in Golgi cells was on average 1.3 milliseconds, 0.6 milliseconds longer than the minimum latencies of EPSPs in nearby octopus and T stellate cells. The long latency raises the possibility that Golgi cells receive input from slowly conducting, unmyelinated auditory nerve fibers. Golgi cells are also excited by interneurons with N-methyl-D-aspartate receptors, probably granule cells, because repetitive shocks and single shocks in the absence of extracellular Mg2+ evoked late EPSPs that were reversibly blocked by DL-2-amino-5-phosphono-valeric acid.

Animals↗
PubMed1992

Integrated presentation of multimodal brain images.

This article discusses the fusion of brain images from multiple modalities as well as the presentation of the integrated image information. The paper has three parts. First, individual brain imaging modalities are compared as regards clinical appreciation, invasiveness, dimensionality, spatial resolution, temporal resolution, and cost. Next, methods to combine multiple images are briefly surveyed and collated by characteristics as accuracy, patient-friendliness, reproducibility, labour-extensiveness, feasibility of retrospective matching, and general applicability. Finally, techniques to display multimodal image information are outlined and examples of the various options for integrated presentation are shown.

Brain↗
PubMed2026

Multimodal features and prognostic risk assessment in locally advanced gastric cancer patients following neoadjuvant therapy based on machine learning algorithms: a multicenter study.

BACKGROUND: Neoadjuvant therapy (NAT) is recommended for locally advanced gastric cancer (LAGC), but some patients respond poorly. We aimed to construct a multimodal model integrating CT images, transcriptomic sequencing, and clinicopathological data to assess prognosis in LAGC patients receiving NAT. MATERIALS AND METHODS: This multicenter study included 505 LAGC patients who underwent NAT. Radiomic features were extracted from preoperative CT images of 505 patients. RNA-seq was performed on 277 post-NAT specimens, with additional data from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases (n&#x2009;=&#x2009;804). Patients were divided into training (168 cases), internal validation (72 cases), and external validation cohorts. Machine learning algorithms identified key radiomic, molecular, and clinical features associated with NAT response, which were then integrated into a multimodal model to predict overall survival (OS) and disease-free survival (DFS). RESULTS: Six radiomic and three molecular features significantly associated with NAT response were selected. Radiomic risk (hazard ratio [HR]: 4.0, P&#x2009;<&#x2009;0.001) and molecular risk (HR: 7.1, P&#x2009;<&#x2009;0.001) were independent prognostic factors. By integrating radiomic risk, molecular risk, and clinical characteristics, a multimodal model (MuMo) was constructed.The C-index results (OS, C-index&#x2009;=&#x2009;0.855; DFS, C-index&#x2009;=&#x2009;0.786) demonstrated that MuMo outperformed the single-modality models and ypTNM staging.Mechanistic analysis suggested that the efficacy of neoadjuvant therapy was significantly enriched in immune-inflammatory pathways. CONCLUSIONS: MuMo can effectively predict postoperative survival risk in LAGC patients receiving NAT, serving as a powerful tool for optimizing prognostic assessment.

Humans↗
PubMed1998

A system for medical consultation and education using multimodal human/machine communication.

Recent developments in networking and computing have enabled collaborative biomedical engineering research by geographically separated participants. One of the most promising goals is to use these technologies to extend human intellectual capabilities in medical decision making. These emerging technologies are poised to drastically reduce healthcare cost by providing service at remote locations. This also increases diagnosis capacity since information is made available to experts at any location. In this paper, we propose a novel application of a recently developed interactive and distributed system in medical consultation and education. Our approach builds on the notion that interactive and distributive capabilities of the system are crucial for medical consultation and education. The presented application uses a multiuser, collaborative environment with multimodal human/machine communication in the dimensions of sight, sound, and touch. The experimental setup, consisting of two user stations, and the multimodal interfaces, including sight (eye-tracking), sound (automatic speech), and touch (microbeam pen), were tested and evaluated. The system uses a collaborative workspace as a common visualization space. Users communicate with the application through a fusion agent by eye-tracking, speech, and microbeam pen. The audio/video teleconferencing is also included to help the radiologists to communicate with each other simultaneously while they are working on the mammograms. The system used in this study has three software agents: a fusion agent, a conversational agent, and an analytic agent. The fusion agent interprets multimodal commands by integrating the multimodal inputs. The conversational agent answers the user's questions and detects human-related or semantic errors and notifies the user about the results of the image analysis. The analytic agent enhances the digitized images using the wavelet denoising algorithm if requested by the user. To show how well the system performs in practice, we used the system for medical consultation on mammograms. Results also show that the relevant information about the region of interest (ROI) of the mammograms chosen by the users is extracted automatically and used to enhance the mammograms.

Education, Distance↗
PubMed2006

The neurocognitive bases of human multimodal food perception: sensory integration.

This review addresses a fundamental neuroscientific question in food perception: how multimodal features of food are integrated. Much research and conceptualization has emerged related to multisensory integration in vision, audition and somatosensation, while it remains poorly understood and researched within the chemical and mouth feel senses. This review aims to bridge this gap. We discuss the main concepts in the fields of auditory, visual and somatosensory multisensory integration and relate them to oral-sensory (gustatory and somatosensory) and olfactory (orolfactory) interactions. We systematically review the psychophysical literature pertaining to intra- and intermodal interactions related to food perception, while making explicit distinctions between peripheral and central interactions. As the neural bases of crossmodal orolfaction currently are poorly understood, we introduce several plausible neuroscientific models, which provide a framework for further neuroscientific exploration in this area. We are guided by a new meta-analysis of the odor-taste neuroimaging literature, as well as by single-unit, anatomical and psychophysical studies. Finally, we propose strong involvement of recurrent neural networks in multisensory integration and make suggestions for future research.

Animals↗
PubMed1999

Integration and visualization of multimodality brain data for language mapping.

A goal of the University of Washington Brain Project is to develop software tools for processing, integrating and visualizing multimodality language data obtained at the time of neurosurgery, both for surgical planning and for the study of language organization in the brain. Data from a single patient consist of four magnetic resonance-based image volumes, showing anatomy, veins, arteries and functional activation (fMRI). The data also include the location, on the exposed cortical surface, of sites that were electrically stimulated for the presence of language. These five sources are mapped to a common MR-based neuroanatomical model, then visualized to gain a qualitative appreciation of their relationships, prior to quantitative analysis. These procedures are described and illustrated, with emphasis on the visualization of fMRI activation, which may be deep in the brain, with respect to surface-based stimulation sites.

Anatomy, Cross-Sectional↗
PubMed2025

miss-SNF: a multimodal patient similarity network integration approach to handle completely missing data sources.

MOTIVATION: Precision medicine leverages patient-specific multimodal data to improve prevention, diagnosis, prognosis, and treatment of diseases. Advancing precision medicine requires the non-trivial integration of complex, heterogeneous, and potentially high-dimensional data sources, such as multi-omics and clinical data. In the literature, several approaches have been proposed to manage missing data, but are usually limited to the recovery of subsets of features for a subset of patients. A largely overlooked problem is the integration of multiple sources of data when one or more of them are completely missing for a subset of patients, a relatively common condition in clinical practice. RESULTS: We propose miss-Similarity Network Fusion (miss-SNF), a novel general-purpose data integration approach designed to manage completely missing data in the context of patient similarity networks. miss-SNF integrates incomplete unimodal patient similarity networks by leveraging a non-linear message-passing strategy borrowed from the SNF algorithm. miss-SNF is able to recover missing patient similarities and is "task agnostic", in the sense that can integrate partial data for both unsupervised and supervised prediction tasks. Experimental analyses on nine cancer datasets from The Cancer Genome Atlas (TCGA) demonstrate that miss-SNF achieves state-of-the-art results in recovering similarities and in identifying patients subgroups enriched in clinically relevant variables and having differential survival. Moreover, amputation experiments show that miss-SNF supervised prediction of cancer clinical outcomes and Alzheimer's disease diagnosis with completely missing data achieves results comparable to those obtained when all the data are available. AVAILABILITY AND IMPLEMENTATION: miss-SNF code, implemented in R, is available at https://github.com/AnacletoLAB/missSNF.

Humans↗
PubMed2006

Implementation of discrete unitary transformations by multimode waveguide holograms.

Integration of holograms into multimode waveguides allows the implementation of arbitrary unitary mode transformations and unitary matrix-vector multiplication. Theoretical analysis is used to justify a design approach to implement specific functions in these devices. Based on this approach, a compact mode-order converter, a Hadamard transformer, and a spatial pattern generator-correlator are proposed and analyzed. Beam propagation simulations are used to verify the theoretical calculations and to address bandwidth, scalability, and fabrication criteria. Optical pattern generators were successfully fabricated using standard photolithographic techniques to demonstrate the feasibility of the devices.

Journal Article↗
PubMed2026

Integrating histology and spatial transcriptomics via multimodal transformers and contrastive representation learning for accurate gene expression prediction.

Predicting spatial gene expression from Histological images is a fundamental task in understanding tissue organization and molecular phenotypes. However, existing methods often rely on single-model representations or lack effective alignment between image and transcriptomic features. To address these limitations, we propose a unified multimodal learning framework that integrates histological imaging and spatial transcriptomics through a shared latent representation space. Specifically, histological H&E images are encoded by a ResNet50-based convolutional stem and a MobileViT Transformer backbone to extract hierarchical visual representations. Both modalities are projected into a shared latent space via linear-GELU-dropout transformation blocks, enabling cross-modal alignment through a contrastive learning objective that maximizes agreement between the corresponding image and the spot embeddings. Experimental results on the 10x Genomics Visium dataset of human liver tissue demonstrate that MViTGene achieves significantly higher prediction accuracy than existing methods across multiple gene subsets, with improvements of 20%, 33%, and 12% in predicting marker genes, highly expressed genes, and highly variable genes, respectively. The significant improvement in relevance indicates that the model can more accurately capture the true correspondence between tissue morphology and gene expression, therefore enabling more reliable biological interpretation. It provides a computational tool for high-throughput spatial gene expression prediction that balances performance and interpretability.

Humans↗
PubMed1988

Morphological and physiological characterization of individual olfactory interneurons connecting the brain and eyestalk ganglia of the crayfish.

1. In order to understand the functional organization of the crustacean olfactory system, we are using intracellular recording and staining techniques to correlate the structure and function of single, odorant-sensitive interneurons in the brain of the crayfish Procambarus clarkii. We describe here the anatomy and physiology of interneurons that connect the brain with the medullae terminales or other eyestalk ganglia. 2. All of the interneurons in our study are at least third-order olfactory neurons (second-order olfactory interneurons) because they respond to chemostimulation of the olfactory organ (the antennules) but do not branch in the olfactory lobe (the neuropil to which primary olfactory receptor cells of the antennules project). 3. Much of the central nervous system, including the three main divisions of the brain (protocerebrum, deuterocerebrum, tritocerebrum) and the medullae terminales, are involved in integrating olfactory or multimodal (including olfactory) information, since these areas contain neurites of olfactory interneurons. Previous studies have indicated that regions involved in such processing include the olfactory lobes and accessory lobes of the deuterocerebrum, and regions I, II, IV, and VII (in some species) of the medullae terminales. Our results show that also prominent among regions involved in olfactory or multimodal (including olfactory) integration are the anterior and posterior optic neuropils of the protocerebrum, the lateral and medial antennular neuropils of the deuterocerebrum, the tegumentary neuropils and the antennal neuropils of the tritocerebrum, and neuropils III, VI, XII of the medullae terminales. 4. These olfactory interneurons were sensitive to chemostimulation (unimodal), chemo- and mechanostimulation (bimodal), or chemo-, mechano-, and photostimulation (trimodal). Responses could be excitatory or inhibitory, even for a given neuron. Morphologically complex interneurons (those having bilateral branching) were more likely to have complex response characteristics (trimodal sensitivity) than were morphologically simpler interneurons (those having unilateral branching). Olfactory interneurons with a soma in the medulla terminalis showed the most complex response profiles: they were trimodal, and were exicted by odorants but were inhibited by touch and/or light. This finding suggests that these are complex, high order interneurons. 5. Our studies reveal that olfactory and other sensory information is transmitted between the brain and the medullae terminales (and possibly other eyestalk ganglia) by a coactivated, parallel array of structurally and functionally diverse neurons.

Action Potentials↗
PubMed2025

Multimodal deep learning for immunotherapy response prediction and biomarker discovery in non-small cell lung cancer.

OBJECTIVE: Immunotherapy has emerged as a promising treatment for advanced non-small cell lung cancer (NSCLC), but accurately predicting which patients will benefit from it remains a major clinical challenge. To address this, we aim to develop a novel multimodal method, DeepAFM, that integrates histopathology, genomic features, and clinical information to predict patient responses to anti-PD-(L)1 immunotherapy. MATERIALS AND METHODS: A total of 93 patients with advanced NSCLC were included in this study. Histopathological whole-slide images were processed using a self-supervised VQVAE2 for representation learning. PCA and K-means clustering were then applied for dimensionality reduction and feature grouping. Key regions of interest were visualized through permutation importance evaluation and color-coding techniques. The extracted histopathological features, along with genomic alterations and clinical variables, were integrated into the DeepAFM multimodal prediction model. RESULTS: The DeepAFM achieved a high predictive performance with an area under the curve (AUC) of 0.77 (95% confidence interval: 0.69-1.00). Attention-based heatmaps revealed that the model could identify critical pathological patterns, genomic mutations, and clinical indicators associated with patient responses to immunotherapy. DISCUSSION: The integration of multimodal data enabled the model to capture complex interactions among pathology, genomics, and clinical characteristics, enhancing the interpretability and predictive power of immunotherapy response prediction. The visualization techniques facilitated the identification of biologically meaningful features and potential biomarkers. CONCLUSION: This study demonstrates the effectiveness of the DeepAFM in predicting responses to immunotherapy in advanced NSCLC. The approach not only improves prediction accuracy but also provides valuable insights for personalized treatment strategies and biomarker discovery.

Humans↗