SEARCH · Search PubMed

Results for “ROC Curve”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3Linked to original sources

PubMed2000

An interpretation for the ROC curve and inference using GLM procedures.

The accuracy of a medical diagnostic test is often summarized in a receiver operating characteristic (ROC) curve. This paper puts forth an interpretation for each point on the ROC curve as being a conditional probability of a test result from a random diseased subject exceeding that from a random nondiseased subject. This interpretation gives rise to new methods for making inference about ROC curves. It is shown that inference can be achieved with binary regression techniques applied to indicator variables constructed from pairs of test results, one component of the pair being from a diseased subject and the other from a nondiseased subject. Within the generalized linear model (GLM) binary regression framework, ROC curves can be estimated, and we highlight a new semiparametric estimator. Covariate effects can also be evaluated with the GLM models. The methodology is applied to a pancreatic cancer dataset where we use the regression framework to compare two different serum biomarkers. Asymptotic distribution theory is developed to facilitate inference and to provide insight into factors influencing variability of estimated model parameters.

Humans↗

PubMedDate not supplied

ROC curves and the areas under them for dichotomized tests: empirical findings for logistically and normally distributed diagnostic test results.

Many measures, including sensitivity and specificity, predictive values, and likelihood ratios, are available for the assessment of diagnostic tests. A drawback of the use of these measures is that continuous test results are often dichotomized, with consequent loss of information. Receiver operating characteristic (ROC) curves do not depend on discrimination thresholds, and therefore the area under the ROC curve (AUC) is one of the preferred measures. Although quantitative test results are often presented dichotomized, it would be convenient still to be able to estimate the ROC curve and the AUC. The authors present equations for such estimates when only one pair of a true- and a false-positive rate is given, for inherently logistically and normally distributed data. Illustrative empirical data are provided for both distributions. In contradiction to earlier reports, the authors also show that differential disease verification may skew the ROC curve. The ROC curve is thus not invariant to selection bias.

Bayes Theorem↗

PubMed1997

Receiver operating characteristic (ROC) curve analysis of transrectal ultrasonographic prostate cancer detection.

PURPOSE: To provide an overview of receiver operating characteristics (ROC) curve theory and create an ROC curve describing transrectal ultrasound (TRUS) detection of prostate cancer (PCa). METHODS: One hundred thirty-two patients with either an abnormal digital rectal exam (DRE) or a prostate-specific antigen (PSA) above 4 ng/ml or both underwent TRUS and biopsy of the peripheral zone (PZ) of each quadrant. ROC software was used to create an ROC curve. RESULTS: One hundred seventy-nine of 528 quadrants (34%) harbored PCa. The performance of TRUS in detection of PCa as defined by the area below the ROC is 0,809. CONCLUSIONS: Future investigators are encouraged to employ ROC analysis of TRUS to permit more meaningful comparisons of performance.

Journal Article↗

PubMed1998

COMPROC and CHECKNORM: computer programs for comparing accuracies of diagnostic tests using ROC curves in the presence of verification bias.

To assess relative accuracies of two diagnostic tests, we often compare the areas under the receiver operating characteristic (ROC) curves of these two tests in a paired design. Standard methods for analyzing data from a paired design require that every patient tested has the known disease status. In practice, however, some of the patients with test results may not have verified disease status. Any analysis using only verified cases may result in verification bias. COMPROC is an easy to use program for comparing the effectiveness of two diagnostic tests based on the area under the ROC curve in the presence of verification bias. COMPROC compensates for verification bias by implementing the maximum likelihood (ML) estimation of the areas and covariance matrix of two ROC curves under the missing at random (MAR) assumption as described by Zhou (Biometrics 54 (1998) 349-366). This method assumes normality of the difference of the two ROC curve area estimators. We also describe a program CHECKNORM that does a bootstrap analysis to test this normality assumption (B. Efron, R.J. Tibshirani, An Introduction to the Bootstrap, Chapman and Hall, London, 1993). COMPROC allows for the inclusion of observed covariates that may influence the decision to verify the disease status of a patient. The program computes the estimates of the area under the ROC curve for the two diagnostic tests along with the variance of each area, the covariance between the two areas, a two-sided p-value, and a confidence interval for the difference of the areas. The programs COMPROC and CHECKNORM require the scripting language Perl and the statistical software SAS and can be run on both UNIX machines as well as PCs. The use of COMPROC and CHECKNORM is illustrated in a clinical study designed to compare relative accuracies of MRI and CT in detecting pancreatic cancer.

Area Under Curve↗

PubMed2008

On the exact interval estimation for the difference in paired areas under the ROC curves.

An important measure for comparison of accuracy between two diagnostic procedures is the difference in paired areas under the receiver operating characteristic (ROC) curves. Non-parametric and maximum likelihood methods have been proposed for interval estimation for the difference in paired areas under ROC curves. However, these two methods are asymptotic procedures and their performance in finite sample sizes has not been thoroughly investigated. We propose to use the concept of generalized pivotal quantities (GPQs) to construct an exact confidence interval for the difference in paired areas under ROC curves. A simulation study is conducted to empirically investigate the probability coverage and expected length of the three methods for various combinations of sample sizes, values of the area under the ROC curve and correlations. Simulation results demonstrate that the exact confidence interval based on the concept of GPQs provides not only sufficient probability coverage but also reasonable expected length. Numerical examples using published data sets illustrate the proposed method.

Clinical Trials as Topic↗

PubMed2002

Comparison of bandwidth selection methods for kernel smoothing of ROC curves.

In this paper we compared four non-parametric kernel smoothing methods for estimating an ROC curve based on a continuous-scale test. All four methods produced a smooth ROC curve of the test. The difference in these four methods lay with the way they chose their bandwidth parameters. To assess the relative performance of the four bandwidth selection methods, we conducted a simulation study using different underlying distributions, along with varied sample sizes. The results from our simulation study suggested that the kernel smoothing method originally proposed by Altman and Léger for estimation of the distribution function was the best choice for estimation of an ROC curve. We illustrated these methods with a real example.

Computer Simulation↗

PubMedDate not supplied

Meta-analysis of ROC curves.

The authors present a method to combine several independent studies of the same (continuous or semiquantitative) diagnostic test, where each study reports a complete ROC curve; a plot of the true-positive rate or sensitivity against the false-positive rate or one minus the specificity. The result of the analysis is a pooled ROC curve, with a confidence band, as opposed to earlier proposals that result in a pooled area under the ROC curve. The analysis is based on a two-parameter model for the ROC curve that can be estimated for each individual curve. The parameters are then pooled with a bivariate random-effects meta-analytic method, and a curve can be drawn from the pooled parameters. The authors propose to use a model that specifies a linear relation between the logistic transformations of sensitivity and one minus specificity. Specifically, they define V = In(sensitivity/(1 - sensitivity)) and U = In((1 - specificity)/specificity), and then D = V - U, S = V + U. The model is defined as D = alpha + betaS. The parameters alpha and beta are estimated using weighted linear regression with bootstrapping to get the standard errors, or using maximum likelihood. The authors show how the procedure works with continuous test data and with categorical test data.

Alcoholism↗

PubMed1992

ROC curves for the initial assessment of new diagnostic tests.

New diagnostic tests are mainly evaluated by determining the sensitivity and specificity of the test. These test characteristics were originally meant to be used in making diagnoses. For evaluative purposes their usefulness is weakened by their susceptibility to selection and their dependence on the cut-off points that are used for test positivity. The plotting of a receiver operating characteristic (ROC) curve might be a solution to these problems. Furthermore, the ROC curve yields a measure for the diagnostic power of the test expressed in one number instead of two, namely the area under the curve (AUC). Finally, the ROC curve and its AUC permit easy comparison of different tests and the performance of different interpreters of one test. The construction and use of ROC curves are described and illustrated with data of a case-referent investigation into the relationship between iron status parameters and the presence of acute myocardial infarction. The AUCs of ferritin and serum iron, 0.61 and 0.68 respectively, are too low to suggest meaningful usefulness in clinical practice.

Humans↗

PubMed1992

[Clinical decision analysis and ROC curve].

This workshop was planned to teach the importance of clinical decision analysis and the ROC curve in clinical medicine. A pre-workshop test and post-workshop test were given before and after the workshop. In the clinical decision analysis a heart disease case was shown, a decision tree was made, and the chance node and expected value were calculated. The ROC curve was prepared from the results of urine analysis of patients with renal and urinary tract infections, and the cut-off points were changed variously. There were 27 participants. Twelve of them took both the pre-workshop test and post-workshop test, and about half of them answered that they understood how to perform the analysis. There tended to be more participants who understood how to make the clinical decision analysis than the ROC curve.

Decision Support Techniques↗

PubMedDate not supplied

The area under the ROC curve and its competitors.

The area under the receiver operating characteristic (ROC) curve is a popular measure of the power of a (two-disease) diagnostic test, but it is shown here to be an inconsistent criterion: tests of indistinguishable clinical impacts may have different areas. A class of diagnosticity measures (DMs) of proven optimality is proposed instead. Once a regret(-like) measure of diagnostic uncertainty is agreed upon, the associated DM is uniquely defined and, indeed, calculable from the ROC curve configuration. Two scaled variants of the ROC are introduced and used to advantage in the analysis. They may also be helpful to students of medical decision making.

Decision Theory↗

PubMed1993

Diagnostic accuracy evaluation using ROC curve analysis.

The diagnostic accuracy of a biochemical quantity is inversely related to the overlapping zone between the values of the population suffering from a disease and the population which does not. The ROC curves are an indirect measure of the overlapping zone between both populations. Specimens (plasma and urine) taken from 928 patients with symptoms of acute abdominal pain were used and the catalytic concentration of alpha-amylase, pancreatic alpha-amylase and triacylglycerol lipase (determined by two methods) were measured. Definitive diagnosis was obtained by following the directives of expert groups on the evaluation of diagnostic tests. Diagnostic accuracy was characterized by calculating the diagnostic sensitivity and specificity, by representing the ROC curves and by quantifying the areas under the ROC curves. The catalytic concentration of pancreatic alpha-amylase in plasma was the quantity with a greater area under the ROC curve (A = 0.9740) and then the one which had greatest diagnostic accuracy. If we considered the upper limit of the reference interval to be the cut-off value, the catalytic concentration of pancreatic alpha-amylase in plasma had a diagnostic sensitivity and specificity values of 0.96 and 0.88 respectively for the acute pancreatitis.

Acute Disease↗

PubMed2006

Estimation of time-dependent area under the ROC curve for long-term risk prediction.

Sensitivity, specificity, and area under the ROC curve (AUC) are often used to measure the ability of survival models to predict future risk. Estimation of these parameters is complicated by the fact that these parameters are time-dependent and by the fact that censoring affects their estimation just as it affects estimation of survival curves or coefficients of survival regression models. The authors present several estimators that overcome these complications. One approach is a recursive calculation over the ordered times of events, analogous to the Kaplan-Meier approach to survival function estimation. Another is to first apply Bayes' theorem to write the parameters of interest in terms of conditional survival functions that are then estimated by survival analysis methods. Simulation studies demonstrate that the proposed estimators perform well in practical situations, when compared with an estimator (c-statistic, from logistic regression) that ignores time. An illustration with data from a cardiovascular follow-up study is provided.

Area Under Curve↗

PubMed2000

Regional confidence bands for ROC curves.

The performance of a diagnostic test is characterised by its specificity and sensitivity. For a quantitative diagnostic test these criteria depend on the selected cut-off point. The receiver operating characteristic (ROC) curve of a quantitative diagnostic test is generated by plotting sensitivity against specificity as the cut-off point runs through the whole range of possible test values. In practice, the ROC curve is estimated from clinical data. One important goal is to select an optimal cut-off point. For this purpose the sample variability has to be taken into account. Recently, Campbell has introduced nonparametric asymptotic simultaneous confidence bands that are valid for the whole ROC curve. In this paper a nonparametric asymptotic approach for the construction of regional confidence bands for ROC curves is proposed. It can be applied for any specificity interval of interest. Our approach is based on the asymptotic theory of empirical and quantile processes. To investigate the small sample properties of the different approaches, a Monte Carlo study was conducted using normal and log-normal data. A method for sample size calculation is presented. Finally, the approaches are applied to a tumour marker in the diagnosis of bone marrow metastases.

Bone Marrow Neoplasms↗

PubMed2003

[Transcranial color Doppler sonography (TCCD) in the diagnosis of vasospasm in the middle cerebral artery. Evaluation of TCCD diagnostic effectiveness by means of receiver operating characteristics curve (ROC) analysis].

Transcranial Doppler sonography is commonly used for diagnosis of cerebral vasospasm. However, the overall diagnostic performance of this method in detection of arterial narrowing has not been established. Blood velocity threshold, diagnostic for vasospasm, has been proposed for conventional, "blind" transcranial Doppler sonography (TCD). Nevertheless it cannot be used for transcranial color Doppler sonography (TCCD), because in this method the obtained blood velocities in the major cerebral arteries are higher than are in TCD. The aim of this study was to estimate the accuracy of transcranial color Doppler sonography in detection of middle cerebral artery (MCA) narrowing by means of receiver operating characteristic curve analysis (ROC). One hundred thirty four patients were studied with TCCD immediately before cerebral angiography. There were 75 men and 59 women, age range from 18 to 74 years, mean age 49 years. Of the 268 MCAs examined, 227 arteries were finally included in the construction of the ROC curve. Angiographic vasospasm was graded as none, mild (equal to or less than 25% of vessel caliber reduction--16 patients) and moderate-to-severe (more than 25% of vessel caliber reduction--29 patients). The overall diagnostic performance of transcranial color Doppler sonography in the detection of moderate-to-severe vasospasm of middle cerebral artery was found to be very high. The value of associated area under the ROC curve was 0.94. The value of this area for all vasospasms was 0.85, and this indicates good accuracy of a test. The best performing TCCD parameter for the detection of MCA narrowing was revealed to be peak-systolic velocity. The optimal trade-off between sensitivity and specificity was achieved with a peak systolic velocity of 182 cm/s. Therefore this velocity threshold is proposed as most universal for the diagnosis of vasospasm in the middle cerebral artery.

Adolescent↗

PubMed1994

Efficient confidence bounds for ROC curves.

The specificity and sensitivity of a quantitative diagnostic marker depends on the selected cut-off point. The ROC curve is generated by plotting sensitivity against specificity as the cut-off point runs through the whole range of possible marker values. Confidence bounds for ROC curves may be constructed by combining two separate confidence intervals for sensitivity and specificity. In the present paper, a more efficient method, which is based on a statistical test introduced by Greenhouse and Mantel, is presented. The resulting 95 per cent confidence intervals are up to 40 per cent smaller. A formula is given to calculate the sample size necessary to obtain a confidence interval of a stipulated length at a specified point of the ROC curve. As an example, the performance of a tumour marker in the diagnosis of bone marrow metastases is investigated.

Bone Marrow↗

PubMed1998

Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data.

We show that truth-state runs in rank-ordered data constitute a natural categorization of continuously-distributed test results for maximum likelihood (ML) estimation of ROC curves. On this basis, we develop two new algorithms for fitting binormal ROC curves to continuously-distributed data: a true ML algorithm (LABROC4) and a quasi-ML algorithm (LABROC5) that requires substantially less computation with large data sets. Simulation studies indicate that both algorithms produce reliable estimates of the binormal ROC curve parameters a and b, the ROC-area index Az, and the standard errors of those estimates.

Algorithms↗

PubMed1994

The ROC curve in the evaluation of fasting capillary blood glucose as a screening test for diabetes and IGT.

OBJECTIVE: To assess the performance of different cutoff points of fasting capillary glycemia (FCG) in the diagnosis of diabetes and impaired glucose tolerance (IGT) using the receiver operating characteristics (ROC) curve approach. RESEARCH DESIGN AND METHODS: This study included a sample of 4,019 subjects without a previous history of diabetes who were recruited for a confirmatory 2-h post-glucose challenge capillary glycemia from a larger sample of 21,847 individuals screened with FCG. The sensitivity and the specificity of FCG as a screening test were analyzed in the cutoff range 3.9-8.9 mmol/l, and the corresponding ROC curves were plotted to assess the performance of the test. RESULTS: The screening test performance was better for diabetes than for IGT in the full range of cutoff points studied. Sensitivities ranged between 37.9 and 97.1% for diabetes and 1.8 and 94.4% for IGT; the areas under the ROC curves were 0.91 +/- 0.01 and 0.75 +/- 0.01, respectively. The cutoffs showing the best equilibrium between sensitivity and specificity approached 5.6 and 5.0 mmol/l for diabetes and IGT, respectively. Factors such as age, color, and family history of diabetes can affect the screening test performance. CONCLUSIONS: ROC curves can provide useful information toward improving the usefulness of FCG as a screening test for abnormalities of glucose tolerance.

Adult↗

PubMed2004

Advantages to transforming the receiver operating characteristic (ROC) curve into likelihood ratio co-ordinates.

Traditionally, the receiver operating characteristic (ROC) curve for a diagnostic test plots true positives (sensitivity) against false positives (one minus specificity). However, this representation brings with it several drawbacks. A transformation to positive and negative likelihood ratio co-ordinates, scaled by base-ten logarithms, offers several advantages. First we motivate the use of positive and negative likelihood ratios, emphasizing their relationship to modification of the odds ratio. Then we highlight properties of likelihood ratios using the traditional ROC axes. Finally, we demonstrate ROC curves and their properties after conversion to likelihood ratio co-ordinates. These graphs do not waste space for tests lacking diagnostic power, and offer a simple visual assessment of a test's impact on the odds ratio.

Diagnostic Tests, Routine↗