SEARCH · Search PubMed

Results for “multimodal data fusion”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources

PubMed2004

Hypotension-induced loss of intraoperative monitoring data during surgical correction of scheuermann kyphosis: a case report.

STUDY DESIGN: Presentation of a case report of Scheuermann kyphosis surgical correction. OBJECTIVE: To describe a scenario where both neurogenic mixed evoked potentials and somatosensory-evoked potentials were lost due solely to hypotension before any correction of a kyphotic spinal deformity was performed. SUMMARY OF BACKGROUND DATA: Multimodality intraoperative neurophysiologic monitoring of the spinal cord has become widely utilized during surgical correction of scoliotic and kyphotic deformities. Most spinal surgeries also benefit from a state of hypotension to minimize blood loss, but unchecked and persistent hypotension may lead to inadequate perfusion to the spinal cord, resulting in spinal cord dysfunction noted by diminution of neuromonitoring data. METHODS: An 18-year-old boy with a 95 degrees Scheuermann kyphosis underwent a posterior spinal fusion for correction of his deformity. Intraoperative neurophysiologic monitoring consisting of neurogenic mixed evoked potentials and somatosensory-evoked potentials were performed throughout surgery. RESULTS: After placement of segmental pedicle screw fixation points and multiple osteotomies, before any instrumented correction of the deformity, all lower extremity neuromonitoring data were acutely lost. The surgeon was immediately warned of the data loss, with the mean arterial pressure noted to be 50 mm Hg. The mean arterial pressure was raised with the use of epinephrine bolus and dopamine infusion. Subsequently, all lower extremity neuromonitoring data returned. A Stagnara wake-up test was performed, which the patient passed, and the surgical correction was performed with his pressure maintained on a dopamine infusion. He awakened without neurologic deficits and had an uneventful recovery. CONCLUSIONS: Although a state of mild hypotension may be beneficial to limit blood loss during spinal deformity corrective surgery, acute and/or prolonged hypotension may jeopardize spinal cord vascularity and should be avoided especially during surgical treatment of high-risk deformities such as kyphosis. Early warning by multimodality physiologic neuromonitoring appears to be a useful method to alert surgeons of the potentially devastating problem of hypotension-induced spinal cord dysfunction and allows immediate corrective actions.

Adolescent↗

PubMedDate not supplied

New avenues in 3-D computerised (stereopathological) imaging of breast cancer (review).

Multimodal methods of three-dimensional (3-D) imaging of breast cancer are described. These involve scanning confocal microscopy, using 50 MHz acoustic or near-infrared images, four-view (tetrahedral) radiography and x-ray projection microscopy. Computerised volume data from these techniques can be used to produce three-dimensional images of tissue ranging from 500 microns to approximately 4 mm in thickness. Preliminary findings indicate that stereoscopic images or 3-D computerised reconstructions are capable of advancing the understanding of the structure of ductal carcinoma in situ, lesions simulating microinvasive breast carcinoma, surgical clearance of high-grade calcifying ductal carcinoma in situ, and the 3-D growth patterns of invasive forms of breast carcinoma. In the future computerised image fusion techniques seem likely to be able to take advantage of multimodal imaging of breast cancers, thus correcting primary imaging artefacts, improving robustness, and combining complementary information. In addition, the use of computerised tetrahedral radiography may change the intraoperative assessment of breast cancers, which mostly depend at present upon subsequent laboratory procedures that take days to perform.

Breast Neoplasms↗

PubMed2001

Neuro-fuzzy systems for computer-aided myocardial viability assessment.

This paper describes a multimodality framework for computer-aided myocardial viability assessment based on neuro-fuzzy techniques. The proposed approach distinguishes two main levels: the modality-independent inference level and the modality-dependent application level. This two-level distinction releases the hard constraint of multimodality image registration. An abstract description template is used to describe the different myocardial functions (contractile function, perfusion, metabolism). Parameters extracted from different image modalities are combined to derive a diagnostic image. The neuro-fuzzy techniques make our system transparent, adaptive and easily extendable. Its effectiveness and robustness are demonstrated in a positron emission tomography/magnetic resonance imaging data fusion application.

Artificial Intelligence↗

PubMed1997

[Image fusion of different tomographic methods (PET/CT/MRI) effectively contribute to therapy planning].

Image fusion (and image registration) is a fundamental topic of three-dimensional (3D) medical image processing. The need to register different imaging modalities arises for several reasons: some anatomical details, especially soft tissues, are more easily seen in MRI than CT images, but the bony structures are better visualised by CT. The visualisation of any morphological volume with a functional data set which may come from SPECT or PET is a very important method both for research purposes and for routine diagnostics. Our developed programs allowing us to solve image fusion tasks also contained some free available softwares. These packages helped us to start additional software development for more advanced applications having elaborated this way the informatical background of multimodality image processing. This paper presents three oncological applications of the image fusion: PET-CT-MRI registration for 3D radiotherapy planning, PET-MRI registration in planning surgery and metastasis localization. All these registration processes were solved by the landmark-matching registration method.

Decision Making↗

PubMed2004

PET/CT image navigation and communication.

UNLABELLED: The advent of multimodality imaging scanners combining PET and CT has led to a new paradigm in image display and presentation that raises new challenges in workstation interpretation software, image navigation, and communication. The essence of multimodality imaging is the ability to overlay imaging information from different modalities in a visually compelling fashion. This is accomplished by combining functional and anatomic data into multidimensional views using color-encoding techniques that provide visual clues on the spatial distribution of image data. DISCUSSION: Combined PET/CT scanners provide spatially registered images from the two modalities acquired simultaneously in a single imaging session. Special reconstruction software and image display programs are required to rescale the native images from different spatial resolution into orthogonal or oblique reformatted planes in which data from PET images are color coded and superimposed on corresponding anatomic CT images. The color overlay technique allows the user to visually identify areas of high tracer activity and determine the underlying anatomic structure. Because of the multidimensional nature of the data, visualization requires interactive multidimensional navigation techniques that allow the viewer to move the visualization planes through three spatial directions and two additional dimensions. The fourth dimension is the continuum blend from PET to CT fusion, and the fifth is the dynamic range of the CT images that can be adjusted to display different tissue characteristics, such as bones, soft tissue, and lungs. Software tools currently available are often relatively complex, requiring the user to perform cumbersome maneuvers and time-consuming image manipulation to navigate through all dimensions and obtain adequate image settings and plane positioning for diagnostic interpretation of the image data. Moreover, the ability to convey these images to referring physicians is usually limited because of the lack of adequate viewing software. Distribution of results is usually performed instead through static "snapshots" of the fused images generated by the interpreting radiologist. The ability of the referring physician to navigate through the set of multimodality image data is thus limited. CONCLUSION: The wider adoption of multimodality PET/CT imaging techniques in routine clinical use will depend heavily on the development of more adequate image display and navigation tools that allow interpreting physicians to navigate easily and efficiently through multiple dimensions of data. Distribution of results to referring physicians and care providers also requires new tools for interactively reviewing the multimodality data, and current static images obtained from fused image data remain inadequate for proper visualization of the true content of images.

Algorithms↗

PubMedDate not supplied

Retrospective fusion of radiographic and MR data for localization of subdural electrodes.

Prior to epilepsy surgery, subdural electrodes are often implanted and monitored for a few days to identify the focus of abnormal electrical activity. During the implantation and subsequent brain resection, there may be uncertainty about the exact location of the electrodes with respect to features of brain anatomy such as specific gyral convolutions or lesions. In experiments with a phantom and patients, implanted electrodes were imaged with multiplanar skull radiographs (or CT scans). After retrospective registration with preimplantation MR data, the electrodes were mapped from these studies onto an MR-derived three-dimensional brain model. The resulting multimodality displays showed the relationship of the electrodes to brain anatomy. In one patient the position of each electrode with respect to a metabolic lesion was also displayed by mapping preimplantation PET data onto the same brain model. This new display of electrode positions may strengthen the interpretation of subdural electrical recordings and thereby reduce uncertainty in planning the resection of epileptic tissue.

Adult↗

PubMed2001

Multimodal cranial neuronavigation: direct integration of functional magnetic resonance imaging and positron emission tomography data: technical note.

OBJECTIVE: This is the first report of the direct integration of functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) data into cranial neuronavigation. METHODS: In a patient with a left precentral oligodendroglioma (World Health Organization Grade III), the Zeiss MKM system (Carl Zeiss Co., Oberkochen, Germany) was used for navigation based on thin-slice, T1-weighted, contrast-enhanced magnetic resonance imaging (MRI) scans. fMRI and methionine PET data were integrated by landmark matching, with reference to skin fiducials. RESULTS: The inaccuracy of the image fusion between fMRI and T1-weighted MRI data was 1.7 mm, that between PET and T1-weighted MRI data was 4.3 mm, and that for the subsequent registration of the navigation was 1.2 mm. The correct fMRI localization of the precentral gyrus was intraoperatively verified by cortical somatosensory evoked potential (phase-reversal) monitoring. Although the tumor was not clearly defined in the MRI scans, [11C]methionine PET demonstrated a clear tumor border, enabling us to achieve gross total tumor removal without postoperative functional deficits. CONCLUSION: Functional neuronavigation permits observation and preservation of relevant brain areas. Other functional areas (such as short-term memory areas) that can be detected only by fMRI might also warrant future monitoring. The simultaneous integration of fMRI and PET data adds a new dimension to cranial neuronavigation, enabling the observation of tumors in relation to functional cortical areas (in our case, the motor strip).

Brain↗

PubMed2003

Multimodality cranial image fusion using external markers applied via a vacuum mouthpiece and a case report.

PURPOSE: To present a simple and precise method of combining functional information of cranial SPECT and PET images with CT and MRI, in any combination. MATERIAL AND METHODS: Imaging is performed with a hockey mask-like reference frame with image modality-specific markers in precisely defined positions. This frame is reproducibly connected to the VBH vacuum mouthpiece, granting objectively identical repositioning of the frame with respect to the cranium. Using these markers, the desired 3-D imaging modalities can then be manually or automatically registered. This information can be used for diagnosis, treatment planning, and evaluation of follow-up, while the same vacuum mouthpiece allows precisely reproducible stereotactic head fixation during radiotherapy. RESULTS: 244 CT and MR data sets of 49 patients were registered to a root square mean error (RSME) of 0.9 mm (mean). 64 SPECT-CT fusions on 18 of these patients gave an RMSE of 1.4 mm, and 40 PET-CT data sets of eight patients were registered to 1.3 mm. An example of the method is given by means of a case report of a 52-year-old patient with bilateral optic nerve meningioma. CONCLUSION: This technique is a simple, objective and accurate registration tool to combine diagnosis, treatment planning, treatment, and follow-up, all via an individualized vacuum mouthpiece. Especially for low-resolution PET and even more so for some very diffuse SPECT data sets, activity can now be accurately correlated to anatomic structures.

Equipment Design↗

PubMed1997

Advances in brain imaging: a new ethical challenge.

Technical advances in the past 25 years permitted substantial advances in the neuroimaging field, expanding the diagnostic and research potentials and significantly reducing the use of old invasive imaging techniques for research purposes. The safer procedures now available allow acquisition of reference data, morphological assessment and functional characterisation from healthy volunteers. However, enrollment of volunteers is still a sensitive ethical issue. Ethical problems related to informed consent, for both research and diagnostic procedures, in patients with neuropsychiatric disorders represent an additional crucial issue. Furthermore, with both functional and structural neuroimaging studies, there is a theoretical risk of violation of individual privacy. Research in the neuroimaging field should tend to increase the amount of information obtained through appropriate post-processing procedures, including multimodality image fusion, and to limit stress and discomfort.

Brain↗

PubMed1997

[The robotization of neurosurgery: state of the art and future outlook].

Neurosurgery is by excellence a field of application for robots, based on multimodal image guidance. Specific motorized tools have been already developed and routinely applied in stereotaxy to position a probe holder or in conventional neurosurgery to hold a microscope oriented towards a given target. The potentialities of these approaches have triggered industrial developments currently commercially available. These systems use data bases, primarily coming from multimodal numerical images from X-ray radiology to magnetic resonance imaging. These spatially encoded data are transferred through digital networks to workstations where images can be processed and surgical procedures are preplanned, then transferred to the robotic systems to which they are connected. We have been using a stereotactic robot since 1989 and a microscope robot since 1995 in various surgical routine procedures. The future of these applications mainly rely on the technical progress in informatics, about image recognition to adapt the preplanning to the actual surgical situation, to correct brain shifts for instance, about image fusion, integrated knowledge such such as brain atlases, as well as virtual reality. The future developments, covering surgical procedure, research and teaching, will sure be far beyond our wildest expectations.

Forecasting↗

PubMed2001

Minimal residual disease in soft-tissue sarcomas.

Soft-tissue sarcomas are characterized by the development of disease recurrence in a distinct subgroup of patients. Despite intense efforts in applying multimodal treatment, the risk of local recurrence or distant metastases remains a considerable threat to patients with soft-tissue sarcoma. This review focuses on the efforts aimed at defining local and systemic tumor extent at the level of minimal residual disease (MRD) tumor cell detection. Examination of MRD in soft-tissue sarcomas has experienced a significant boost from the definition of fusion transcripts resulting from stable chromosomal translocations. The sensitivity and exclusive specificity of the reverse-transcription polymerase chain reaction (RT-PCR) protocols have given insights into tumor cell residues in nearly all body compartments. The accumulated data demonstrates that even after oncologic resections most patients will still harbor a significant tumor burden. Clinical concepts arising out of these new data are under way. One of these concepts is the targeting of the fusion transcript for therapy. However, this approach is still restricted to the experimental setting. The development of clinical applications remains a challenging task, requiring the treatment of as many patients as possible in centers specializing in all of the affected disciplines.

Adult↗

PubMed2004

A hierarchical parametric algorithm for deformable multimodal image registration.

Image fusion is of utmost importance for many applications in image analysis. Particularly in medical imaging, images of different modalities are necessary because they provide complementary information that must be merged for an optimal use. The fusion of these images, which can be achieved through a registration process, makes it possible to superimpose all available information on the same frame. In many cases, a rigid transformation is sufficient to align correctly the images. However, there are cases where a non-rigid transformation is needed: geometrical distortions present in one image, non-rigid motion, etc. The purpose of this paper is to propose a generic method to account for these deformations in case of multimodal images. We have applied the algorithm in the particular context of 3D medical images and present results on simulated and real data.

Algorithms↗

PubMed1998

[Robotics in neurosurgery: current status and future prospects].

Neurosurgery is in essence a field of application development for robots, based on multimodal image guidance. Specific motorized tools have already been developed and routinely applied in stereotaxy to position a probe holder or in conventional neurosurgery to hold a microscope oriented towards a given target. The potentialities of these approaches have triggered industrial developments which are now commercially available. These systems use databases, primarily coming from multimodal numerical images from X-ray radiology to magnetic resonance imaging. These spatially encoded data are transferred through digital networks to workstations where images can be processed and surgical procedures are pre-planned, then transferred to the robotic systems to which they are connected. We have been using a stereotaxic robot since 1989 and a microscope robot since 1995 in various surgical routine procedures. The future of these applications rely mainly on the technical progress in informatics, about image recognition to adapt the pre-planning to the actual surgical situation, to correct brain shifts (for instance), about image fusion, integrated knowledge such as brain atlases, as well as virtual reality. The future developments, covering surgical procedure, research and teaching, are sure to be far beyond our wildest expectations.

Forecasting↗

PubMed2025

miss-SNF: a multimodal patient similarity network integration approach to handle completely missing data sources.

MOTIVATION: Precision medicine leverages patient-specific multimodal data to improve prevention, diagnosis, prognosis, and treatment of diseases. Advancing precision medicine requires the non-trivial integration of complex, heterogeneous, and potentially high-dimensional data sources, such as multi-omics and clinical data. In the literature, several approaches have been proposed to manage missing data, but are usually limited to the recovery of subsets of features for a subset of patients. A largely overlooked problem is the integration of multiple sources of data when one or more of them are completely missing for a subset of patients, a relatively common condition in clinical practice. RESULTS: We propose miss-Similarity Network Fusion (miss-SNF), a novel general-purpose data integration approach designed to manage completely missing data in the context of patient similarity networks. miss-SNF integrates incomplete unimodal patient similarity networks by leveraging a non-linear message-passing strategy borrowed from the SNF algorithm. miss-SNF is able to recover missing patient similarities and is "task agnostic", in the sense that can integrate partial data for both unsupervised and supervised prediction tasks. Experimental analyses on nine cancer datasets from The Cancer Genome Atlas (TCGA) demonstrate that miss-SNF achieves state-of-the-art results in recovering similarities and in identifying patients subgroups enriched in clinically relevant variables and having differential survival. Moreover, amputation experiments show that miss-SNF supervised prediction of cancer clinical outcomes and Alzheimer's disease diagnosis with completely missing data achieves results comparable to those obtained when all the data are available. AVAILABILITY AND IMPLEMENTATION: miss-SNF code, implemented in R, is available at https://github.com/AnacletoLAB/missSNF.

Humans↗

PubMedDate not supplied

Image fusion of computed tomographic and magnetic resonance images for the development of a three-dimensional musculoskeletal model of the equine forelimb.

Biomechanical models that compute the lengths and forces of muscle-tendon units are broadly applicable to the study of factors that promote injury and the planning and effects of orthopedic surgical procedures in equine athletes. A three-dimensional (3D) generic musculoskeletal model of the equine forelimb comprised of bony segment, muscle-tendon, and ligament information, was developed based on high-resolution computed tomographic (CT) and T1-weighted magnetic resonance (MR) images from an isolated forelimb of a Thoroughbred racehorse. Image fusion was achieved through coregistration of CT and MR images with an image analysis program (Analyze) by adjustment of the relative position and orientation of fiducial markers visible in both modalities until the mutual information between the images was maximized. 3D surfaces of the bones and origin/insertion sites, centroid paths and volumes of the muscle-tendon and ligamentous structures were obtained from the multimodal (CT/MR) images using semiautomated and manual segmentation combined with sagittal and transverse color-cryosection anatomic images obtained from three other cadaveric equine forelimbs. Once bony and soft-tissue structures were reconstructed in the same coordinate system, data were imported to a software package for interactive musculoskeletal modeling (SIMM). The combination of integrated CT and MR acquisitions and anatomical images provided an accurate and efficient means of generating a 3D model of the musculoskeletal structures of an average-sized equine adult horse.

Animals↗

PubMed2025

Foundation model based multimodal transformer framework for survival analysis in HER2 stratified breast cancer.

Objective. To improve survival prediction for HER2-positive breast cancer by integrating histopathological, molecular, and clinical data using a multimodal transformer framework.Approach. We propose a multimodal transformer framework for breast cancer survival prediction using HER2 stratified (SurvMBC), a foundation model-enhanced architecture that fuses three data modalities: whole-slide images, clinical narratives, and molecular features. Tumor microenvironment features are extracted using a pathology language and image pre-training (PLIP), clinical narratives are processed with BioBERT, and miRNA expression plus DNA methylation data are embedded using Gen2Vec. These representations are integrated through a cross-modal transformer with attention mechanisms for survival prediction.Main results. The model was evaluated on 1,095 HER2-positive breast cancer patients from The Cancer Genome Atlas. SurvMBC achieved a concordance index (C-index) of 0.857 (95% CI: 0.834, 0.880), a low integrated Brier score, and a strong inverse negative binomial log-likelihood. Risk stratification based on model outputs significantly separated high- and low-risk groups (log-rankp< 0.01) and showed strong associations with tumor stage, grade, and hormone receptor status (allp< 0.05).Significance. SurvMBC demonstrates the effectiveness of multimodal fusion in addressing tumor heterogeneity and improving prognostic accuracy. The attention-based integration enables context-aware learning of survival-relevant features across modalities, supporting individualized risk stratification and risk-adaptive treatment planning for HER2 stratified breast cancer patients.

Breast Neoplasms↗

PubMed2001

[Cytokines: current status and prospects in the treatment of skin tumors].

Various therapeutic options using cytokines have been described in the treatment of melanoma, T cell lymphoma, B cell lymphoma, squamous cell carcinoma, basal cell carcinoma and Merkel cell carcinoma. The treatment regimens include cytokine substitution, cytokine induction, cytokine transfection and therapeutic cytokine constructs. In the adjuvant treatment of melanomas, IFN-alpha has become well established. Statistical evaluations of different adjuvant trials show that a significant prolongation of recurrence-free intervals can be achieved. IL-2 has a role in the therapy of advanced melanomas as well as in vaccination strategies. Further possible therapeutic immune modulations, which have been evaluated in experimental approaches and pilot studies, include treatment with IL-4, IL-7 and GM-CSF. Treatment with IL-12 promises to open new perspectives. A well established regimen in the treatment of T cell lymphoma stages Ia-IIb is the combination of PUVA and IFN-alpha. In vitro data also indicate an important (patho)physiological role for IL-12, so that this agent has been tested in phase I studies. IL-2, IFN-gamma, and the fused cytokine-toxin molecules DAB389IL-2 offer further therapeutic alternatives. B cell lymphomas are treated with antibody-IL-2 fusion proteins. Advanced or inoperable squamous cell carcinoma and basal cell carcinoma may be treated with local IFN-alpha injections. IFN-alpha or TNF-alpha may be considered for the treatment of recurrent or advanced Merkel cell carcinoma. In dermatological oncology cytokine treatment focuses on melanome an T cell lymphome. Cytokine application is mainly an integral part of multimodal regimens.

Animals↗

PubMedDate not supplied

Impact of functional restoration after anterior cervical fusion on chronic disability in work-related neck pain.

BACKGROUND CONTEXT: Spinal surgery in the workers compensation population shows evidence of less favorable outcomes than in general health cases. Although spine surgery has been alleged to be a cause of poor outcomes, such outcomes may be improved by appropriate postsurgical rehabilitation. PURPOSE: To compare objective demographic, physical and psychological measurements and socioeconomic outcomes of treatment in work-related disabling cervical pain for the combination of anterior cervical fusion (ACF) plus functional restoration, compared with rehabilitation alone. STUDY DESIGN/SETTING: A prospective study of patients undergoing ACF for degenerative disc disease before rehabilitation for work-related musculoskeletal disorders versus neck pain unoperated controls, with data collected in an outpatient tertiary interdisciplinary rehabilitation setting. PATIENT SAMPLE: A group of 52 patients completed a functional restoration treatment program after undergoing ACF (Group S) at one or two levels for degenerative cervical disc disease. During the study period, 625 patients with work-related neck pain were identified from the same study population, from which a rehabilitation (Group R) comparison group (n=150) was identified who were stratified according to the number and location of other compensable body parts. OUTCOME MEASURES: Socioeconomic outcomes relevant to chronic disabling work-related cervical spinal disorders are reported based on 1-year posttreatment interviews. Pre- to posttreatment assessment of pain intensity, disability, depression and cumulative physical capability were assessed prospectively. METHODS: All patients were totally or partially disabled before completing an intensive, medically supervised, functional restoration program combining quantitatively directed exercise progression with a multimodal disability management approach. Preprogram preparation included drug detoxification, psychotropic medication management and preparatory aerobic and mobility training. The intensive treatment phase involved strength and endurance training, with counseling geared to goals of work return and fitness maintenance. The 1-year structured clinical interview had a contact rate of 93% to 95%, and partial information acquisition on all patients. RESULTS: Although Group S had lower work return and work retention outcomes, the differences were not significant. Group S patients had significantly more health utilization from a new provider in the year after completion of functional restoration (46% vs 24%; OR=2.7 [1.3, 5.3], p<.004). Group S patients were also more likely to be depressed, both at pre- and postrehabilitation. There were no significant differences in recurrent injury, additional surgery, physical measures or pain/disability self-report between the groups. CONCLUSIONS: Workers compensation patients with chronic disabling work-related cervical spinal disorders who undergo a cervical fusion, combined with functional restoration, have socioeconomic outcomes after their surgery statistically similar to those for unoperated controls. Surgery patients had a higher rate of additional health-care-seeking behaviors from new providers and a greater likelihood of being clinically depressed before and after rehabilitation. This study suggests that cervical fusion for degenerative disc disease in workers compensation patients is not contraindicated, as long as interdisciplinary rehabilitation is available for complex cases after the surgical procedure.

Adult↗