Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “ROC Curve”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources
PubMed1992

ROC curve analysis: an example showing the relationships among serum lipid and apolipoprotein concentrations in identifying patients with coronary artery disease.

Clinical accuracy, defined as the ability to discriminate between states of health, is the fundamental property of any diagnostic test or system. It is readily expressed as clinical sensitivity and specificity, and elegantly represented by the receiver operating characteristic (ROC) curve. To demonstrate the use of ROC curves, we reexamine a study of the ability of serum lipid and apolipoprotein measures to discriminate among degrees of coronary artery disease in patients undergoing coronary angiography. ROC curve analysis reveals that none of these indexes is highly accurate, but demonstrates a modest increase in the accuracy of apolipoprotein over lipid indexes.

Apolipoproteins↗
PubMed2006

Tests of equivalence and non-inferiority for diagnostic accuracy based on the paired areas under ROC curves.

Assessment of equivalence or non-inferiority in accuracy between two diagnostic procedures often involves comparisons of paired areas under the receiver operating characteristic (ROC) curves. With some pre-specified clinically meaningful limits, the current approach to evaluating equivalence is to perform the two one-sided tests (TOST) based on the difference in paired areas under ROC curves estimated by the non-parametric method. We propose to use the standardized difference for assessing equivalence or non-inferiority in diagnostic accuracy based on paired areas under ROC curves between two diagnostic procedures. The bootstrap technique is also suggested for both non-parametric method and the standardized difference approach. A simulation study was conducted empirically to investigate the size and power of the four methods for various combinations of distributions, data types, sample sizes, and different correlations. Simulation results demonstrate that the bootstrap procedure of the standardized difference approach not only can adequately control the type I error rate at the nominal level but also provides equivalent power under both symmetrical and skewed distributions. A numerical example using published data illustrates the proposed methods.

Computer Simulation↗
PubMed2004

[Fitting of ROC curves for continuous data by using correction of the mean and standard deviation].

We examined ROC curve fitting with continuous rating data by using a corrected mean value and corrected standard deviation as binormal ROC parameters. Curve fitting was performed by using corrected categorical data in the same way as the continuous data. A large residual of curve fitting was found in cases of difference between means of positive and negative signal data greater than three times the standard deviation, or with a ratio of the standard deviation of positive and negative signal data above six. We conclude that our proposed method performed well for ROC curve fitting except in some special cases.

Data Interpretation, Statistical↗
PubMedDate not supplied

A Bayesian approach to a general regression model for ROC curves.

A fully Bayesian approach to a general nonlinear ordinal regression model for ROC-curve analysis is presented. Samples from the marginal posterior distributions of the model parameters are obtained by a Markov-chain Monte Carlo (MCMC) technique--Gibbs sampling. These samples facilitate the calculation of point estimates and credible regions as well as inferences for the associated areas under the ROC curves. The analysis of an example using freely available software shows that the use of noninformative vague prior distributions for all model parameters yields posterior summary statistics very similar to the conventional maximum-likelihood estimates. Clinically important advantages of this Bayesian approach are: the possible inclusion of prior knowledge and beliefs into the ROC analysis (via the prior distributions), the possible calculation of the posterior predictive distribution of a future patient outcome, and the potential to address questions such as: "What is the probability that a certain diagnostic test is better in one setting than in another?"

Bayes Theorem↗
PubMed2006

ROC curves predicted by a model of visual search.

In imaging tasks where the observer is uncertain whether lesions are present, and where they could be present, the image is searched for lesions. In the free-response paradigm, which closely reflects this task, the observer provides data in the form of a variable number of mark-rating pairs per image. In a companion paper a statistical model of visual search has been proposed that has parameters characterizing the perceived lesion signal-to-noise ratio, the ability of the observer to avoid marking non-lesion locations, and the ability of the observer to find lesions. The aim of this work is to relate the search model parameters to receiver operating characteristic (ROC) curves that would result if the observer reported the rating of the most suspicious finding on an image as the overall rating. Also presented are the probability density functions (pdfs) of the underlying latent decision variables corresponding to the highest rating for normal and abnormal images. The search-model-predicted ROC curves are 'proper' in the sense of never crossing the chance diagonal and the slope is monotonically changing. They also have the interesting property of not allowing the observer to move the operating point continuously from the origin to (1, 1). For certain choices of parameters the operating points are predicted to be clustered near the initial steep region of the curve, as has been observed by other investigators. The pdfs are non-Gaussians, markedly so for the abnormal images and for certain choices of parameter values, and provide an explanation for the well-known observation that experimental ROC data generally imply a wider pdf for abnormal images than for normal images. Some features of search-model-predicted ROC curves and pdfs resemble those predicted by the contaminated binormal model, but there are significant differences. The search model appears to provide physical explanations for several aspects of experimental ROC curves.

Algorithms↗
PubMed2000

Using ROC curves to test models of recognition memory: the relationship between presentation duration and slope.

Many theories of memory predict that the slope of the z-transformed receiver-operating characteristic (ROC) curve should decrease with increases in the memory strength of old items. While several prior studies have failed to demonstrate this effect, the results of two experiments demonstrate that increasing presentation duration can reduce the slope of the z-transformed ROC curve. These results raise questions about the generality of prior results and of dual-process theories designed to accommodate those results. We close by emphasizing that determining the experimental circumstances that affect whether or not the slope of the z-transformed ROC curve decreases will provide important constraints on theories of recognition memory.

Humans↗
PubMed2003

ROC curve analysis for biomarkers based on pooled assessments.

Interleukin-6 is a biomarker of inflammation which has been suggested as having potential discriminatory ability for myocardial infarction. Because of its high assaying cost it is very expensive to evaluate this marker. In order to reduce this cost we propose pooling the specimens. In this paper we examine the efficiency of ROC curve analysis, specifically the estimation of the area under the ROC curve, when dealing with pooled data. We study the effect of pooling when there are only a fixed number of individuals available for testing and pooling is carried out to save on the number of assays. Alternatively we examine how many pooled assays of size g are necessary to provide essentially the same information as N individual assays. We measure loss of information by means of the change in root mean square error of the estimate of the area under the ROC curve and study the extent of this loss via a simulation study.

Biomarkers↗
PubMed2002

ROC curves in evaluation of serum fibrosis indices for hepatic fibrosis.

AIM: Use Receiver operating characteristic (ROC) curves to find out the relationship between serum level of hyaluronic acid (HA), type III procollagen (PCIII), N-terminal procollagen III peptide (PIIINP), laminin (LN), type IV collagen (C-IV) and hepatic fibrosis, as well as to determine their value in clinical practice. METHODS: 114 serum samples from chronic hepatitis patients were assayed for fibrosis indices including HA, PCIII, PIIINP, LN and IV-C with radioimmunoassay (RIA). Liver biopsy was also performed in all these patients and the biopsy material was examined histopathologically. RESULTS: ROC curves analysis showed that area under the curve (AUC) of PIIINP, HA, PCIII, C-IV and LN was 0.800, 0.728, 0.727, 0.583 and 0.463, respectively. The analysis also showed that PIIINP (r=0.452), HA (r=0.497) and PCIII (r=0.404) have greater diagnosis performances than C-IV(r=0.238) and LN (r=0.128) according to fibrosis staging. The sensitivity of HA plus PIIINP was 55.1 %, it was the most sensitive combination. Combined three or more than three indices that based on HA, the specificity was 100 %. Using combination assays can improve the specificity, but its sensitivity was not high. Serum fibrosis indices increased as the grade of inflammation aggravated. But only PIIINP and PCIII had significant difference between G1 and G2 (PIIINP: 13.16+/-8.07 vs 8.32+/-5.09; PCIII: 164.22+/-65.69 vs 138.23+/-77.63). The coefficient correlation of the results of inflammation grade and fibrosis staging to HA was 0.525 and 0.553 respectively, that to PCIII, 0.446 and 0.412, that to LN, 0.234 and 0.194, and that to IV-C, 0.363 and 0.351, respectively. CONCLUSION: Serum fibrosis indices can indicate tendency of hepatic fibrosis, but it cannot replace liver biopsy. However, as diagnostic markers, more efficient serum fibrosis indices for the diagnosis of hepatic fibrosis need to be explored.

Adolescent↗
PubMed2001

Prediction of the usefulness of combined mammography and scintimammography in suspected primary breast cancer using ROC curves.

UNLABELLED: The aim of this study was to compare the accuracy of 99mTc-methoxyisobutylisonitrile scintimammography (SMM) and conventional mammography in patients presenting with suspected primary breast cancer. Receiver-operating-characteristic (ROC) curve analysis was applied to determine if a combination of x-ray mammography (XMM) and SMM was more accurate than a single test alone. METHODS: The results of SMM, XMM, and a combination of both studies performed over a 3-y period on 374 suspicious lesions in 353 patients with no previous history of breast cancer were reviewed. Each scan report was reviewed and graded as follows: grade 1, definitely normal or benign; grade 2, probably normal or benign; grade 3, equivocal; grade 4, probably cancer; and grade 5, definitely cancer. The results were verified by pathologic examination of biopsy material obtained from each suspicious mass. ROC curves were generated from these results. RESULTS: There were 204 malignant breast tumors and 170 nonmalignant breast lesions. SMM diagnosed correctly 181 breast cancers and was true-negative in 122 benign breast lesions: sensitivity, 89%; specificity, 71%; positive predictive value (PPV), 79%; and negative predictive value (NPV), 84%. XMM diagnosed correctly 143 malignant tumors and was true-negative in 117 nonmalignant lesions. The sensitivity, specificity, PPV, and NPV for XMM were 70%, 69%, 73%, and 66%, respectively. Using a combination of the two tests, the combined sensitivity was 93%, specificity was 72%, PPV was 80%, and NPV was 90%. Using the index of the area under the ROC curve obtained by the rating method showed that the combination of XMM and SMM was significantly more accurate than either of the individual tests if performed alone (P < 0.05). CONCLUSION: This study shows that the combination of XMM and SMM produces more accurate results than either modality alone. Therefore, if there is doubt about the accuracy of XMM, SMM should be used as the second-line test in breast imaging.

Adult↗
PubMed1994

Evaluation of screening methods for Down's syndrome using bootstrap comparison of ROC curves.

This paper concerns the prediction of fetal Down's syndrome in pregnant women. Down's syndrome is the most common congenital cause of severe mental retardation. We elaborate two predictive functions of trisomy 21, combining maternal age and a maternal serum marker. We evaluated them by means of receiver operating characteristic (ROC) curves which give a representation of sensitivity and specificity of a prediction model when varying the cutoff of the predictor on the whole spectrum. Since normal statistical methods for comparison of ROC curves rely on distributional assumptions which were not verified, we used bootstrapping of ROC curves as a check for the statistical significance of differences between the areas under the curves.

Algorithms↗
PubMed1991

ROC curves and the binormal assumption.

Previous articles in this series have described how receiver operating characteristic (ROC) graphs provide comprehensive graphic representations of the diagnostic performance of non-binary tests and have explained how one constructs "trapezoidal" ROC graphs in which discrete cutoff points are plotted and connected with line segments. In this article, we describe a set of mathematical assumptions that permit the generation of a continuous, smooth ROC curve for a given diagnostic test. These assumptions permit us to characterize a test's performance using a small number of parameters and also to explore properties of diagnostic tests. In this article, we describe a set of mathematical assumptions that can be used to link receiver operating characteristic (ROC) curves to the underlying distribution of values of the diagnostic variable being measured. We will illustrate these assumptions using a diagnostic test that distinguishes alcohol abusers from normal consumers of alcohol and abstainers.

Alcoholism↗
PubMed2000

New approaches to measuring the performance of programs that generate differential diagnoses using ROC curves and other metrics.

INTRODUCTION: Evaluation of computer programs which generate multiple diagnoses can be hampered by a lack of effective, well recognized performance metrics. We have developed a method to calculate mean sensitivity and specificity for multiple diagnoses and generate ROC curves. METHODS: Data came from a clinical evaluation of the Heart Disease Program (HDP). Sensitivity, specificity, positive and negative predictive value (PPV, NPV) were calculated for each diagnosis type in the study. A weighted mean of overall sensitivity and specificity was derived and used to create an ROC curve. Alternative metrics Comprehensiveness and Relevance were calculated for each case and compared to the other measures. RESULTS: Weighted mean sensitivity closely matched Comprehensiveness and mean PPV matched Relevance. Plotting the Physician's sensitivity and specificity on the ROC curve showed that their discrimination was similar to the HDP but sensitivity was significantly lower. CONCLUSIONS: These metrics give a clear picture of a program's diagnostic performance and allow straightforward comparison between different programs and different studies.

Diagnosis, Computer-Assisted↗
PubMed1996

Use of receiver-operator characteristic (ROC) curve analysis to evaluate predictors of response to clomipramine therapy.

We used receiver-operator characteristic (ROC) curve analysis to evaluate predictors of response to clomipramine in obsessive-compulsive disorder (OCD). Previously, we identified response predictors among 230 OCD patients who received clomipramine in a placebo-controlled, multicenter clinical trail. We found that at baseline a later age of OCD onset, low scores on the Hamilton Depression scale, and high scores on items 3 and 8 of the Yale-Brown Obsessive Compulsive Scale predicted good response. Certain early side effects also predicted outcome. We fitted a logistic regression model containing baseline information and then calculated each patient's estimated response probability by substituting individuals' values in the regression equation. Next we compared the estimated response risks with each patient's known outcome. Finally, we produced a ROC curve by plotting the true positive and false positive rates for various cutoff points of the risk scores. The same steps were followed for Weeks 1 through 4, adding information about early side effects and weekly response. We found that baseline information predicted outcome better than chance, and predictive ability increased with data on side effects and early response.

Adult↗
PubMedDate not supplied

Statistical comparison of two ROC-curve estimates obtained from partially-paired datasets.

The authors propose a new generalized method for ROC-curve fitting and statistical testing that allows researchers to utilize all of the data collected in an experimental comparison of two diagnostic modalities, even if some patients have not been studied with both modalities. Their new algorithm, ROCKIT, subsumes previous algorithms as special cases. It conducts all analyses available from previous ROC software and provides 95% confidence intervals for all estimates. ROCKIT was tested on more than half a million computer-simulated datasets of various sizes and configurations representing a range of population ROC curves. The algorithm successfully converged for more than 99.8% of all datasets studied. The type I error rates of the new algorithm's statistical test for differences in Az estimates were excellent for datasets typically encountered in practice, but diverged from alpha for datasets arising from some extreme situations.

Algorithms↗
PubMedDate not supplied

Comparing the areas under more than two independent ROC curves.

For a diagnostic test, the area under the associated receiver operating characteristic (ROC) curve is considered a measure of the efficacy of the test. Statistical methodology for the comparison of the areas under more than two independent ROC curves is developed. The jackknife is used to devise an F test using the pseudovalues as data. A Studentized range (SR) test is also considered using the original area estimates. A Monte Carlo study is performed to evaluate the significance level and power of the two test statistics. Both statistics conform well to the 0.10, 0.05, and 0.01 significance levels when the sampling design is balanced between cases with and without the disease. Power is also comparable. For unbalanced designs, the SR test on the original area estimates is very conservative while the F test on pseudovalues performs well. The F test is recommended as the method of choice for comparing the areas, although for balanced designs the SR test, with its computational simplicity, may be preferred.

Diagnosis↗
PubMed1994

ROC curve regression analysis: the use of ordinal regression models for diagnostic test assessment.

Diagnostic tests commonly are characterized by their true positive (sensitivity) and true negative (specificity) classification rates, which rely on a single decision threshold to classify a test result as positive. A more complete description of test accuracy is given by the receiver operating characteristic (ROC) curve, a graph of the false positive and true positive rates obtained as the decision threshold is varied. A generalized regression methodology, which uses a class of ordinal regression models to estimate smoothed ROC curves has been described. Data from a multi-institutional study comparing the accuracy of magnetic resonance (MR) imaging with computed tomography (CT) in detecting liver metastases, which are ideally suited for ROC regression analysis, are described. The general regression model is introduced and an estimate for the area under the ROC curve and its standard error using parameters of the ordinal regression model is given. An analysis of the liver data that highlights the utility of the methodology in parsimoniously adjusting comparisons for covariates is presented.

Algorithms↗
PubMed1989

Nonparametric comparison of entire ROC curves for computerized ECG left ventricular hypertrophy algorithms using data from the Framingham Heart Study.

A computer program may be capable of several different statements for left ventricular hypertrophy (eg, possible LVH, probable LVH, consistent with LVH), but such statements resulting from discretized levels of sensitivity/specificity would represent only isolated points on a receiver-operating characteristic (ROC) curve, which is a plot of all levels of sensitivity versus specificity. Even if two algorithms use the same discrete scales, their performances may not readily be compared. The authors present a comparison methodology for ROC curves using ROC area as a nonparametric measure of the ability of the algorithm to separate the two populations; the ROC area ranges from 0.5 (no ability) to 1.0 (perfect separation) and is unbiased if the normal versus abnormal populations have no common values for the measurement. The methodology compares the performance of ECG algorithms on the same population of cases by testing for significant differences of ROC areas and incorporating correlation of the algorithms in a nonparametric way. To illustrate this methodology, they use ECG and echocardiographic data from the Framingham Heart Study.

Algorithms↗
PubMed2004

Semiparametric estimation of time-dependent ROC curves for longitudinal marker data.

One approach to evaluating the strength of association between a longitudinal marker process and a key clinical event time is through predictive regression methods such as a time-dependent covariate hazard model. For example, a Cox model with time-varying covariates specifies the instantaneous risk of the event as a function of the time-varying marker and additional covariates. In this manuscript we explore a second complementary approach which characterizes the distribution of the marker as a function of both the measurement time and the ultimate event time. Our goal is to extend the standard diagnostic accuracy concepts of sensitivity and specificity so as to recognize explicitly both the timing of the marker measurement and the timing of disease. The accuracy of a longitudinal marker can be fully characterized using time-dependent receiver operating characteristic (ROC) curves. We detail a semiparametric estimation method for time-dependent ROC curves that adopts a regression quantile approach for longitudinal data introduced by Heagerty and Pepe (1999, Applied Statistics, 48, 533-551). We extend the work of Heagerty and Pepe (1999, Applied Statistics, 48, 533-551) by developing asymptotic distribution theory for the ROC estimators where the distributional shape for the marker is allowed to depend on covariates. To illustrate our method, we analyze pulmonary function measurements among cystic fibrosis subjects and estimate ROC curves that assess how well the pulmonary function measurement can distinguish subjects that progress to death from subjects that remain alive. Comparing the results from our semiparametric analysis to a fully parametric method discussed by Etzioni et al. (1999, Medical Decision Making, 19, 242-251) suggests that the ability to relax distributional assumptions may be important in practice.

Adolescent↗