Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “Software Validation”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,549 records · Page 86Linked to original sources
PubMed2004

A comparison of cluster analysis methods using DNA methylation data.

MOTIVATION: Aberrant DNA methylation is common in cancer. DNA methylation profiles differ between tumor types and subtypes and provide a powerful diagnostic tool for identifying clusters of samples and/or genes. DNA methylation data obtained with the quantitative, highly sensitive MethyLight technology is not normally distributed; it frequently contains an excess of zeros. Established tools to analyze this type of data do not exist. Here, we evaluate a variety of methods for cluster analysis to determine which is most reliable. RESULTS: We introduce a Bernoulli-lognormal mixture model for clustering DNA methylation data obtained using MethyLight. We model the outcomes using a two-part distribution having discrete and continuous components. It is compared with standard cluster analysis approaches for continuous data and for discrete data. In a simulation study, we find that the two-part model has the lowest classification error rate for mixture outcome data compared with other approaches. The methods are illustrated using DNA methylation data from a study of lung cancer cell lines. Compared with competing hierarchical clustering methods, the mixture model approaches have the lowest cross-validation error for detecting lung cancer subtype (non-small versus small cell). The Bernoulli-lognormal mixture assigns observations to subgroups with the lowest uncertainty. AVAILABILITY: Software is available upon request from the authors. SUPPLEMENTARY INFORMATION: http://www-rcf.usc.edu/~kims/SupplementaryInfo.html

Algorithms↗
PubMed1990

Validation of a new automatic system for biplane quantitative coronary arteriography.

In a collaboration between the University of Texas (software) and the University of Zürich (hardware) a compact, automatic system for biplane quantitative coronary arteriography was developed. The system is based on a 35 mm film projector, a slow-scan CCD-camera (image digitizing) and a computer workstation (Apollo DN 3000, image storage and processing). A new calibration procedure based on two fixed reference points in the center of the image intensifier was used (isocenter technique). Contour detection of coronary arteries was carried out in biplane projection using a geometric-densitometric edge-detection algorithm. The proximal and distal luminal areas, as well as the minimal luminal area of the stenotic vessel segment were determined. Accuracy and precision were determined from precision drilled holes in a plexiglas cube which were filled with 50%, 75% and 100% contrast medium. The diameter of the holes ranged from 0.5 to 5.0 mm. The mean difference and the standard deviation of the differences between the true and the measured diameters were 0.12 +/- 0.14 mm for plane A and 0.26 +/- 0.17 mm for plane B, respectively. After a second order correction the mean difference amounted to 0.02 +/- 0.09 mm for plane A and 0.02 +/- 0.12 mm for plane B, respectively. Intra- and interobserver variability were evaluated in 5 patients (age 60 +/- 10 years) with coronary artery disease using 16 normal and 5 stenotic vessel segments (cross-sectional area ranging from 0.8 to 8.7 mm2). Two independent observers analyzed the same vessel segment twice. Intraobserver variability expressed as the standard error of estimate in percent of the mean angiographic vessel area (SEE) amounted to 2.1% for observer 1 and 4.4% for observer 2, respectively. Interobserver variability expressed as SEE was 4.1% for measurement 1 and 3.6% for measurement 2, respectively.

Aged↗
PubMed2002

A novel measurement technique to assess the effects of coronary brachytherapy in clinical trials.

This paper presents a novel measurement technique to assess the effects of coronary brachytherapy. This new technique is based upon the conventional quantitative coronary analysis (QCA) technique, which is accepted worldwide as an accurate and reliable analysis tool for clinical trials. This paper provides the definitions and main issues important for correct brachytherapy analysis. Based on these definitions, this novel technique is implemented as an extension of conventional QCA software, as a multisegmental analysis tool. It allows to follow the influence of radiation on restenosis, and the mutual relation between intervention devices. A pilot interobserver study was performed to assess the reliability and reproducibility of the brachytherapy analysis tool, using 15 patient cases. The validation results show that the segment lengths, minimum lumen diameter, and reference diameters of the user-defined and derived (sub)segments can be assessed reproducible. However, these good results can only be obtained, when strict and extensive image acquisition and image analysis protocols are followed. From this pilot validation study presented in this paper and only based on a small number of patients, we may conclude that the software can be applied to clinical trials.

Brachytherapy↗
PubMed2004

Regression models for unbalanced longitudinal ordinal data: computer software and a simulation study.

A computer program GGOREX in the form of a SAS macro is developed for the analysis of longitudinal ordinal data. It is extended from GEECAT and GEEGOR developed by Williamson, Lipsitz and Kim in their paper in 1999. An illustrative example with some preliminary data from a study conducted by the National Institute of Child Health and Human Development and the University of Alabama at Birmingham is given. Another set of computer programs is developed to make it possible to conduct a simulation study to validate the GGOREX procedure in a finite sample. A simple computer program in FORTRAN using the IMSL software library is developed to solve for the probability distribution of the longitudinal ordinal responses according to the correlation specification.

Computer Simulation↗
PubMed2004

Validation and characterization of uninhibited enzyme kinetics performed in multiwell plates.

Several systematic errors may occur during the analysis of uninhibited enzyme kinetic data using commercially available multiwell plate reader software. A MATLAB program is developed to remove these systematic errors from the data analysis process for a single substrate-enzyme system conforming to Michaelis-Menten kinetics. Three experimental designs that may be used to validate a new enzyme preparation or assay methodology and to characterize an enzyme-substrate system, while capitalizing on the ability of multiwell plate readers to perform multiple reactions simultaneously, are also proposed. These experimental designs are used to (i) test for enzyme inactivation and the quality of data obtained from an enzyme assay using Selwyn's test, (ii) calculate the limit of detection of the enzyme assay, and (iii) calculate Km and Vm values. If replicates that reflect the overall error in performing a measurement are used, the latter two experiments may be performed with internal estimation of the error structure. The need to correct for the systematic errors discussed and the utility of the proposed experimental designs were confirmed by numerical simulation. The proposed experiments were conducted using recombinant inducible nitric oxide synthase preparations and the oxyhemoglobin assay.

Biochemistry↗
PubMed2006

Computer-assisted reader software versus expert reviewers for polyp detection on CT colonography.

OBJECTIVE: The purpose of our study was to assess the sensitivity of computer-assisted reader (CAR) software for polyp detection compared with the performance of expert reviewers. MATERIALS AND METHODS: A library of colonoscopically validated CT colonography cases were collated and separated into training and test sets according to the time of accrual. Training data sets were annotated in consensus by three expert radiologists who were aware of the colonoscopy report. A subset of 45 training cases containing 100 polyps underwent batch analysis using ColonCAR version 1.2 software to determine the optimum polyp enhancement filter settings for polyp detection. Twenty-five consecutive positive test data sets were subsequently interpreted individually by each expert, who was unaware of the endoscopy report, and before generation of the annotated reference via an unblinded consensus interpretation. ColonCAR version 1.2 software was applied to the test cases, at optimized polyp enhancement filter settings, to determine diagnostic performance. False-positive findings were classified according to importance. RESULTS: The 25 test cases contained 32 nondiminutive polyps ranging from 6 to 35 mm in diameter. The ColonCAR version 1.2 software identified 26 (81%) of 32 polyps compared with an average sensitivity of 70% for the expert reviewers. Eleven (92%) of 12 polyps > or = 10 mm were detected by ColonCAR version 1.2. All polyps missed by experts 1 (n = 4) and 2 (n = 3) and 12 (86%) of 14 polyps missed by expert 3 were detected by ColonCAR version 1.2. The median number of false-positive highlights per case was 13, of which 91% were easily dismissed. CONCLUSION: ColonCAR version 1.2 is sensitive for polyp detection, with a clinically acceptable false-positive rate. ColonCAR version 1.2 has a synergistic effect to the reviewer alone, and its standalone performance may exceed even that of experts.

Adult↗
PubMed2003

In silico study of breast cancer associated gene 3 using LION Target Engine and other tools.

Sequence analysis of individual targets is an important step in annotation and validation. As a test case, we investigated human breast cancer associated gene 3 (BCA3) with LION Target Engine and with other bioinformatics tools. LION Target Engine confirmed that the BCA3 gene is located on 11p15.4 and that the two most likely splice variants (lacking exon 3 and exons 3 and 5, respectively) exist. Based on our manual curation of sequence data, it is proposed that an additional variant (missing only exon 5) published in a public sequence repository, is a prediction artifact. A significant number of new orthologs were also identified, and these were the basis for a high-quality protein secondary structure prediction. Moreover, our research confirmed several distinct functional domains as described in earlier reports. Sequence conservation from multiple sequence alignments, splice variant identification, secondary structure predictions, and predicted phosphorylation sites suggest that the removal of interaction sites through alternative splicing might play a modulatory role in BCA3. This in silico approach shows the depth and relevance of an analysis that can be accomplished by including a variety of publicly available tools with an integrated and customizable life science informatics platform.

Adaptor Proteins, Signal Transducing↗
PubMed2001

Validity of DISHES 98, a computerised dietary history interview: energy and macronutrient intake.

OBJECTIVE: To estimate the relative validity of a computerised dietary history instrument (DISHES 98). SETTINGS: Munich and Berlin. SUBJECTS: A total of 148 persons aged 19-59 y recruited from two research centres. DESIGN: A relative validation study. Energy and macronutrient intakes obtained with DISHES 98 were compared to those assessed with 3-day weighed dietary records and with a 24 h dietary recall. RESULTS: Intakes of energy, total, saturated and monounsaturated fat, polysaccharides and alcohol were significantly higher and intake of dietary fibre was significantly lower with the 3-day records than with DISHES 98. For intakes of total, animal and vegetable protein, total carbohydrates, mono- and disaccharides and cholesterol the mean difference between DISHES 98 and the 3-day dietary records was less than 5% of the intake with DISHES 98. Pearson's correlation coefficients between DISHES 98 and 3-day records varied from 0.34 for intake of polyunsaturated fat to 0.69 for intake of disaccharides and from 0.27 for polyunsaturated fat to 0.65 for total carbohydrates between DISHES 98 and the 24 h recall. The proportion of participants classified into the same or adjacent quintile of intake varied between 66.9% for polyunsaturated fat and 90.4% for alcohol comparing DISHES 98 and 3-day records and between 60.2% for polyunsaturated fat and 78.4% for total carbohydrates comparing DISHES 98 and 24 h recalls. CONCLUSION: The observed differences between DISHES 98 and the other methods are in an acceptable range for assessing dietary intake in epidemiologic studies.

Adult↗
PubMed2004

Spectral sensitivity study of dose distributions for a commercial convolution/superposition algorithm.

The focus of this study is to validate whether the sensitivity of dose distribution following the interface of different media can be used to distinguish between small variations of photon energy spectra in the context of the convolution/superposition algorithm in the polyenergetic implementation (Philips Pinnacle3, ADAC Laboratories, Milpitas, CA). Calculations were performed in homogeneous water and heterogeneous lung/water phantoms. Spectra were generated, in which the weights of the low-, medium- and high-energy components were adjusted sequentially. The heterogeneity correction factor CFlung, the D20/D10 ratio for homogeneous water and logarithmic derivative in buildup region LDbuildup were assessed for their relative ability to discriminate between different spectra for various field sizes. In accordance with another study (Charland et al 2004), the superior discrimination ability of the CFlung and LDbuildup tests over the D20/D10 test was observed for changes in an energy component as small as 0.3% of the total weight in the energy spectrum. Furthermore, new tests utilizing transverse dose profile data for discriminating between spectra, Fringe Index (FI) and Penumbra Index (PI), were introduced. The discrimination ability of the PI and FI tests was superior when a medium containing interface effects was exploited to obtain the transverse profile data (water/lung phantom for PIhung and FIlung tests) as opposed to when a homogeneous water medium was used (PIwater and FIwater tests).

Algorithms↗
PubMed2001

Validity of lower extremity strength and power utilizing a new closed chain dynamometer.

PURPOSE: The purpose of this study was to compare selected variables measured on a traditional isokinetic dynamometer (Cybex II) with a new lower extremity, closed chain dynamometer (Omnikinetic, OmK). METHODS: Twelve subjects (6 male, 6 female, age = 28+/-5 yr, mean +/- SD) performed Cybex II knee flexion and extension at 1.05, 3.14, and 5.23 rad x s(-1). A maximal effort of 10 repetitions of lower extremity concentric extension and eccentric flexion at 36% of subject's 1-RM was performed on the OmK. Crank power and joint (ankle, knee, and hip) kinetics were recorded as a mean of 10 repetitions. RESULTS: t-Tests revealed right versus left leg differences (P < 0.05) for Cybex II peak torque flexion at 5.23 rad x s(-1), and OmK knee and hip peak power and hip root mean square power (RMS) power. Cybex peak knee torques were related (Pearson r values 0.78-0.92, P < 0.01) to OmK peak knee torques. Cybex average power was related to OmK knee power (Pearson r values 0.71-0.96, P < 0.01) and OmK crank power (r = 0.62-0.94, P < .01). Correlations tended to be stronger comparing the OmK with the fastest (5.23 rad x s(-1)) Cybex II speed. CONCLUSIONS: These results suggest that the OmK knee and crank kinetic data are comparable to Cybex It isokinetic dynamometry. The ability to evaluate lower extremity joint exercise at a subject's maximal movement speed, in addition to the use of a closed-chain, multi-joint motion, may allow for the OmK to provide a more global evaluation of lower extremity kinetics during seated concentric-extension, eccentric-flexion exercise.

Adult↗
PubMed2006

Method for counting motor units in mice and validation using a mathematical model.

Weakness and atrophy are clinical signs that accompany muscle denervation resulting from motor neuron disease, peripheral neuropathies, and injury. Advances in our understanding of the genetics and molecular biology of these disorders have led to the development of therapeutic alternatives designed to slow denervation and promote reinnervation. Preclinical in vitro research gave rise to the need of a method for measuring the effects in animal models. Our goal was to develop an efficient method to determine the number of motor neurons making functional connections to muscle in a transgenic mouse model of amyotrophic lateral sclerosis (ALS). We developed a novel protocol for motor unit number estimation (MUNE) using incremental stimulation. The method involves analysis of twitch waveforms using a new software program, ITS-MUNE, designed for interactive calculation of motor unit number. The method was validated by testing simulated twitch data from a mathematical model of the neuromuscular system. Computer simulations followed the same stimulus-response protocol and produced waveform data that were indistinguishable from experiments. We show that our MUNE protocol is valid, with high precision and small bias across a wide range of motor unit numbers. The method is especially useful for large muscle groups where MUNE could not be done using manual methods. The results are reproducible across naïve and expert analysts, making it suitable for easy implementation. The ITS-MUNE analysis method has the potential to quantitatively measure the progression of motor neuron diseases and therefore the efficacy of treatments designed to alleviate pathologic processes of muscle denervation.

Algorithms↗
PubMed2001

GREAT-ER: a new tool for management and risk assessment of chemicals in river basins. Contribution to GREAT-ER #10.

The GREAT-ER (Geo-referenced Regional Exposure Assessment Tool for European Rivers) project team has developed and validated an accurate aquatic chemical exposure prediction tool for use within environmental risk assessment schemes. The software system GREAT-ER 1.0 calculates the distribution of predicted environmental concentrations (PECs) of consumer chemicals in surface waters, for individual river stretches as well as for entire catchments. The system uses an ARC/INFO-ArcView (ESRI) based Geographical Information System (GIS) for data storage and visualization, combined with simple mathematical models for prediction of chemical fate. At present, the system contains information for four catchments in Yorkshire, one catchment in Italy, and two in Germany, while other river basins are being added. Great-ER 1.0 has been validated by comparing simulations with the results of an extensive monitoring campaign for two 'down-the-drain' chemicals, i.e. the detergent ingredients boron and Linear Alkylbenzene Sulphonate (LAS). GREAT-ER 1.0 is currently being expanded with models for the terrestrial (diffuse input), air and estaurine compartments.

Environmental Monitoring↗
PubMed2004

Regulatory motif finding by logic regression.

MOTIVATION: Multiple transcription factors coordinately control transcriptional regulation of genes in eukaryotes. Although many computational methods consider the identification of individual transcription factor binding sites (TFBSs), very few focus on the interactions between these sites. We consider finding TFBSs and their context specific interactions using microarray gene expression data. We devise a hybrid approach called LogicMotif composed of a TFBS identification method combined with the new regression methodology logic regression. LogicMotif has two steps: First, potential binding sites are identified from transcription control regions of genes of interest. Various available methods can be used in this step when the genes of interest can be divided into groups such as up-and downregulated. For this step, we also develop a simple univariate regression and extension method MFURE to extract candidate TFBSs from a large number of genes in the availability of microarray gene expression data. MFURE provides an alternative method for this step when partitioning of the genes into disjoint groups is not preferred. This first step aims to identify individual sites within gene groups of interest or sites that are correlated with the gene expression outcome. In the second step, logic regression is used to build a predictive model of outcome of interest (either gene expression or up- and down-regulation) using these potential sites. This 2-fold approach creates a rich diverse set of potential binding sites in the first step and builds regression or classification models in the second step using logic regression that is particularly good at identifying complex interactions. RESULTS: LogicMotif is applied to two publicly available datasets. A genome-wide gene expression data set of Saccharomyces cerevisiae is used for validation. The regression models obtained are interpretable and the biological implications are in agreement with the known resuts. This analysis suggests that LogicMotif provides biologically more reasonable regression models than previous analysis of this dataset with standard linear regression methods. Another dataset of S.cerevisiae illustrates the use of LogicMotif in classification questions by building a model that discriminates between up- and down-regulated genes in iron copper deficiency. LogicMotif identifies an inductive and two repressor motifs in this dataset. The inductive motif matches the binding site of the transcription factor Aft1p that has a key role in regulation of the uptake process. One of the novel repressor sites is highly present in transcription control regions of FeS genes. This site could represent a TFBS for an unknown transcription factor involved in repression of genes encoding FeS proteins in iron deficiency. We establish the robustness of the method to the type of outcome variable used by considering both continuous and binary outcome variables for this dataset. Our results indicate that logic regression used in combination with cluster/group operating binding site identification methods or with our proposed method MFURE is a powerful and flexible alternative to linear regression based motif finding methods. AVAILABILITY: Source code for logic regression is freely available as a package of the R programming language by Ruczinski et al. (2003) and can be downloaded at http://bear.fhcrc.org/~ingor/logic/download/download.html an R package for MFURE is available at http://www.stat.berkeley.edu/~sunduz/software.html

Algorithms↗
PubMed1997

Information analysis and validation of intelligent monitoring systems in intensive care units.

Validation of intelligent systems is an important task to perform. Typically the results of the validation analysis are used to verify whether or not the system satisfies the initial design requirements, and to acquire new knowledge and/or refine the knowledge already acquired. In practice, the validation of intelligent systems usually requires the application of several different techniques (e.g., retrospective, prospective, quantitative). In this work the authors present the methodology devised to validate PATRICIA: an intelligent monitoring system designed to advise clinicians on the management of patients dependent on mechanical ventilation. The application of this methodology requires that appropriate validation paradigms are selected, depending on both the application domain and the characteristics of the intelligent system. The article also presents and discusses validation results.

Expert Systems↗
PubMed2002

Analysis of expression profile using fuzzy adaptive resonance theory.

MOTIVATION: It is well understood that the successful clustering of expression profiles give beneficial ideas to understand the functions of uncharacterized genes. In order to realize such a successful clustering, we investigate a clustering method based on adaptive resonance theory (ART) in this report. RESULTS: We apply Fuzzy ART as a clustering method for analyzing the time series expression data during sporulation of Saccharomyces cerevisiae. The clustering result by Fuzzy ART was compared with those by other clustering methods such as hierarchical clustering, k-means algorithm and self-organizing maps (SOMs). In terms of the mathematical validations, Fuzzy ART achieved the most reasonable clustering. We also verified the robustness of Fuzzy ART using noised data. Furthermore, we defined the correctness ratio of clustering, which is based on genes whose temporal expressions are characterized biologically. Using this definition, it was proved that the clustering ability of Fuzzy ART was superior to other clustering methods such as hierarchical clustering, k-means algorithm and SOMs. Finally, we validate the clustering results by Fuzzy ART in terms of biological functions and evidence. AVAILABILITY: The software is available at http//www.nubio.nagoya-u.ac.jp/proc/index.html

Algorithms↗
PubMed2004

Quantitative evaluation of recall and precision of CAT Crawler, a search engine specialized on retrieval of Critically Appraised Topics.

BACKGROUND: Critically Appraised Topics (CATs) are a useful tool that helps physicians to make clinical decisions as the healthcare moves towards the practice of Evidence-Based Medicine (EBM). The fast growing World Wide Web has provided a place for physicians to share their appraised topics online, but an increasing amount of time is needed to find a particular topic within such a rich repository. METHODS: A web-based application, namely the CAT Crawler, was developed by Singapore's Bioinformatics Institute to allow physicians to adequately access available appraised topics on the Internet. A meta-search engine, as the core component of the application, finds relevant topics following keyword input. The primary objective of the work presented here is to evaluate the quantity and quality of search results obtained from the meta-search engine of the CAT Crawler by comparing them with those obtained from two individual CAT search engines. From the CAT libraries at these two sites, all possible keywords were extracted using a keyword extractor. Of those common to both libraries, ten were randomly chosen for evaluation. All ten were submitted to the two search engines individually, and through the meta-search engine of the CAT Crawler. Search results were evaluated for relevance both by medical amateurs and professionals, and the respective recall and precision were calculated. RESULTS: While achieving an identical recall, the meta-search engine showed a precision of 77.26% (+/-14.45) compared to the individual search engines' 52.65% (+/-12.0) (p < 0.001). CONCLUSION: The results demonstrate the validity of the CAT Crawler meta-search engine approach. The improved precision due to inherent filters underlines the practical usefulness of this tool for clinicians.

Data Collection↗
PubMed2003

BioWareDB: the biomedical software and database search engine.

UNLABELLED: A wealth of bioinformatics tools and databases has been created over the last decade and most are freely available to the general public. However, these valuable resources live a shadow existence compared to experimental results and methods that are widely published in journals and relatively easily found through publication databases such as PubMed. For the general scientist as well as bioinformaticists, these tools can deliver great value to the design and analysis of biological and medical experiments, but there is no inventory presenting an up-to-date and easily searchable index of all these resources. To remedy this, the BioWareDB search engine has been created. BioWareDB is an extensive and current catalog of software and databases of relevance to researchers in the fields of biology and medicine, and presently consists of 2800 validated entries. AVAILABILITY: BioWareDB is freely available over the Internet at http://www.biowaredb.org/

Abstracting and Indexing↗
PubMed2006

Assessing and documenting general competencies in otolaryngology resident training programs.

OBJECTIVES: The objectives of this study were to: 1) implement web-based instruments for assessing and documenting the general competencies of otolaryngology resident education, as outlined by the Accreditation Council of Graduate Medical Education (ACGME); and 2) examine the benefit and validity of this online system for measuring educational outcomes and for identifying insufficiencies in the training program as they occur. METHODS: We developed an online assessment system for a surgical postgraduate education program and examined its feasibility, usability, and validity. Evaluations of behaviors, skills, and attitudes of 26 residents were completed online by faculty, peers, and nonphysician professionals during a 3-year period. Analyses included calculation and evaluation of total average performance scores of each resident by different evaluators. Evaluations were also compared with American Board of Otolaryngology-administered in-service examination (ISE) scores for each resident. Convergent validity was examined statistically by comparing ratings among the different evaluator types. RESULTS: Questionnaires and software were found to be simple to use and efficient in collecting essential information. From July 2002 to June 2005, 1,336 evaluation forms were available for analysis. The average score assigned by faculty was 4.31, significantly lower than that by nonphysician professionals (4.66) and residents evaluating peers (4.63) (P < .001), whereas scores were similar between nonphysician professionals and resident peers. Average scores between faculty and nonphysician groups showed correlation in constructs of communication and relationship with patients, but not in those of professionalism and documentation. Correlation was observed in respect for patients but not in medical knowledge between faculty and resident peer groups. Resident ISE scores improved in the third year of the study and demonstrated high correlation with faculty perceptions of medical knowledge (r = 0.65, P = .007). CONCLUSIONS: Compliance for completion of forms was 97%. The system facilitated the educational management of our training program along multiple dimensions. The small perceptual differences among a highly selected group of residents have made the unambiguous validation of the system challenging. The instruments and approach warrant further study. Improvements are likely best achieved in broad consultation among other otolaryngology programs.

Adult↗