Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “ROC Curve”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources
PubMed1992

A receiver operating characteristic (ROC) curve analysis of a model of mental health services use by Puerto Rican poor.

In this study, the contribution of four distinct domains of the Help Seeking-Decision Making model to predicting the use of mental health services is examined. Using a proposed methodology the authors assess the relevance of this model and its domains to mental services planning. The methodology combines logistic regression analysis and receiver operating characteristic (ROC) curves. Logistic regression analysis allows us to examine the individual variables of the model and generate predictions about use. ROC curves allow us to compare and interpret the relative contribution of a predisposing domain, a physical and mental health domain, an enabling-restrictive domain, and an organizational domain in correctly classifying users and nonusers of mental health services. The physical and mental health domain yielded a Somer's D-statistic of 0.7, which corresponds to an 85% correct classification of randomly selected pairs of users and nonusers. The study findings suggest that comparing ROC curves helps to describe and interpret the domains of the model that are relevant for making predictions about who will or will not use mental health services during a 1-year period.

Adolescent↗
PubMed2005

Measurement error and confidence intervals for ROC curves.

Measurement error in a continuous test variable may bias estimates of the summary properties of receiver operating characteristics (ROC) curves. Typically, unbiased measurement error will reduce the diagnostic potential of a continuous test variable. This paper explores the effects of possibly heterogenous measurement error on estimated ROC curves for binormal test variables. Corrected estimators for specific points on the curve are derived under the assumption of known or estimated measurement variances for individual test results. These estimators and associated confidence intervals do not depend on normal assumptions for the distribution of the measurement error and are shown to be approximately unbiased for moderate size samples in a simulation study. An application from a study of emerging imaging modalities in breast cancer is used to demonstrate the new techniques.

Computer Simulation↗
PubMed2005

A conditional nonparametric test for comparing two areas under the ROC curves from a paired design.

RATIONALE AND OBJECTIVES: To develop a conditional nonparametric procedure for comparing two correlated areas under receiver operating characteristic (ROC) curves (AUC). MATERIALS AND METHODS: A nonparametric conditional test to compare areas under two ROC curves was developed using the distribution of the elements of the nonparametric AUC estimators in a permutation space. The conditioning is made on the observed discordances between the relative orderings of ratings of the normal and abnormal cases for the two modalities taken over all possible pairs. The type I error of the procedure was verified using computer simulations. The power of the test was compared with an existing unconditional procedure on simulated datasets from binormal distributions as well as from a mixture of binormal distributions of ratings. RESULTS: The proposed test is conservative for low sample sizes, large AUC, and high correlation between modalities. It possesses a reasonable type I error for sample sizes as low as 20 actually positive and 20 actually negative cases. In plausible situations in which the sample in observer performance studies can not be monotonically transformed into a binormal distribution, this approach may have modest power advantages over the conventional nonparametric test. CONCLUSION: The conditional nonparametric test presented here is an alternative approach to existing unconditional procedures and may offer advantages in certain types of observer performance studies.

Area Under Curve↗
PubMed1996

Diagnostic accuracy of hyaluronan and type III procollagen amino-terminal peptide serum assays as markers of liver fibrosis in chronic viral hepatitis C evaluated by ROC curve analysis.

Diagnostic accuracy of two serum markers of liver fibrosis, hyaluronan (HA) and amino-terminal peptide of type III procollagen (P-III-P), was studied in a cohort of 326 untreated patients with chronic viral hepatitis C. Both P-III-P (RIA-gnost P-III-P, Behring Diagnostic) and HA (HA-test, Pharmacia) serum concentrations correlated with the histological grades of liver fibrosis (P < 0.001). Receiver-operating characteristic (ROC) curves showed that serum HA had greater diagnostic performance than P-III-P, both for discriminating patients with extensive liver fibrosis from those with no or mild fibrosis (area under the ROC curves: 0.864 vs 0.691, P <0.001) or for discriminating patients with cirrhosis from those without cirrhosis (area under the ROC curves: 0.924 vs 0.734, P <0.001). At cutoff values of 0.8 kU/L for serum P-III-P and 85 micrograms/L for serum HA, sensitivities were 70.0% and 64.5%, and specificities were 63.4% and 91.2%, respectively, for discriminating patients with extensive liver fibrosis from those with no or mild fibrosis. At the cutoff values of 1.0 kU/L for serum P-III-P and 110 micrograms/L for serum HA, sensitivities were 60.0% and 79.2%, and specificities were 74.0% and 89.4%, respectively, for discriminating patients with liver cirrhosis from those without cirrhosis. We conclude that, because the diagnostic accuracy of serum HA is greater than that of serum P-III-P as a marker of liver fibrosis, serum HA should be preferred when monitoring liver fibrosis in patients with chronic viral hepatitis C.

Adult↗
PubMed1982

The meaning and use of the area under a receiver operating characteristic (ROC) curve.

A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics, is presented. It is shown that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject. Moreover, this probability of a correct ranking is the same quantity that is estimated by the already well-studied nonparametric Wilcoxon statistic. These two relationships are exploited to (a) provide rapid closed-form expressions for the approximate magnitude of the sampling variability, i.e., standard error that one uses to accompany the area under a smoothed ROC curve, (b) guide in determining the size of the sample required to provide a sufficiently reliable estimate of this area, and (c) determine how large sample sizes should be to ensure that one can statistically detect differences in the accuracy of diagnostic techniques.

Evaluation Studies as Topic↗
PubMed2006

Estimation of haplotype associated with several quantitative phenotypes based on maximization of area under a receiver operating characteristic (ROC) curve.

An algorithm for estimating haplotypes associated with several quantitative phenotypes is proposed. The concept of a receiver operating characteristic (ROC) curve was introduced, and a linear combination of the quantitative phenotypic values was considered. This set of values was divided into two parts: values for subjects with and without a particular haplotype. The goodness of its partition was evaluated by the area under the ROC curve (AUC). The AUC value varied from 0 to 1; this value was close to 1 when the partition had high accuracy. Therefore, the strength of association between phenotypes and haplotypes was considered to be proportional to the AUC value. In our algorithm, the parameters representing a degree of association between the haplotypes and phenotypes were estimated so as to maximize the AUC value; further, the haplotype with the maximum AUC value was considered to be the best haplotype associated with the phenotypes. This algorithm was implemented by using R language. The effectiveness of our algorithm was evaluated by applying it to real genotype data of the Calpine-10 gene obtained from diabetics. The results showed that our algorithm was more reasonable and advantageous for use with several quantitative phenotypes than the generalized linear model or the neural network model.

Algorithms↗
PubMed1994

An Excel program for calculating and plotting receiver-operator characteristic (ROC) curves, histograms and descriptive statistics.

Receiver-operator characteristic (ROC) curves have been used increasingly to assess the performance of clinical laboratory tests and to determine suitable positive/negative threshold values. The Microsoft Excel 4.0 program Plot.ROC was designed to provide formatted, publication quality output with minimal user involvement. It asks for a few parameters at the beginning and then runs autonomously, creating ROC charts, histograms and descriptive statistics for each test chosen, and a cumulative ROC chart to compare tests. Output can easily be formatted further using Excel's graphics and text formatting capabilities.

Biomarkers, Tumor↗
PubMedDate not supplied

Empirical Bayes combination of estimated areas under ROC curves using estimating equations.

A synthesis of the empirical Bayes method and the method of estimating equations is used to combine individual receiver operating characteristic (ROC) area estimates from different studies of the same diagnostic test into a single estimate. This single estimate represents the population mean from which individual areas under the ROC curves were sampled. The only data needed to carry out the method are estimated areas under the ROC curves and the corresponding standard errors.

Bayes Theorem↗
PubMed1989

On the statistical analysis of ROC curves.

We introduce a new accuracy index for receiver operating characteristic (ROC) curves, namely the partial area under the binormal ROC graph over any specified region of interest. We propose a simple but general procedure, based on a conventional analysis of variance, for comparing accuracy indices derived from two or more different modalities. The proposed method is related to and compared with existing methodology, and is illustrated by results from an experiment on optimization of density and contrast yielded by multiform photographic images used for scintigraphy.

Analysis of Variance↗
PubMedDate not supplied

ROC curves for classification trees.

A common problem in medical diagnosis is to combine information from several tests or patient characteristics into a decision rule to distinguish diseased from healthy patients. Among the statistical procedures proposed to solve this problem, recursive partitioning is appealing for the easily-used and intuitive nature of the rules it produces. The rules have the form of classification trees, in which each node of the tree represents a simple question about one of the predictor variables, and the branch taken depends on the answer. The authors consider the role of misclassification costs in developing classification trees. By varying the ratio of costs assigned to false negatives and false positives, a series of classification trees are generated, each optimal for some range of cost ratios, and each with a different sensitivity and specificity. The set of sensitivity-specificity combinations define a curve that can be used like an ROC curve.

Child, Preschool↗
PubMed1982

The use of an ROC curve to express quality of care results.

When a quality of care method which results in a continuous score is viewed as a screening device, it is possible to evaluate the efficiency and effectiveness of the score using assessment techniques usually applied to screening tests for disease. Using an independent standard (such as patient outcome) for determining which records truly represent inadequate medical care, for any quality score cutoff we can calculate the sensitivity and the specificity of the screening device for identifying inadequate care. An ROC curve allows the user of a screening test to select a threshold that takes into account both the estimated prevalence of inadequate care at the site and the relative importance placed on false positive and false negative errors. We used an ROC curve to examine the performance of criteria mapping, a chart review system. The criteria map method was employed to evaluate the disposition decision for 421 patients who came to 2 emergency departments with complaints of chest pain. The shape of the ROC curve generated from these data allows the user to evaluate the effect of choosing various cutpoints, acknowledging explicitly the tradeoffs between failure to review cases of inappropriate disposition and needless review of appropriate disposition.

Emergency Service, Hospital↗
PubMed2003

Three methods for analysing correlated ROC curves: a comparison in real data sets from multi-reader, multi-case studies with a factorial design.

This paper compares three published methods for analysing multiple correlated ROC curves: a method using generalized estimating equations with marginal non-proportional ordinal regression models; a method using jackknifed pseudovalues of summary statistics; a method using a corrected F-test from analysis of variance of summary statistics. Use of these methods is illustrated through six real data examples from studies with the common factorial design, that is, multiple readers interpreting images obtained with each test modality on each study subject. The issue of the difference between typical summary statistics and summary statistics from typical ROC curves is explored. The examples also address similarities and differences among the analytical methods. In particular, while point estimates of differences between test modalities are similar, the standard errors of these differences do not agree for all three methods. A simulation study supports the standard errors provided by the generalized estimating equations with marginal non-proportional ordinal regression models.

Analysis of Variance↗
PubMed1997

Statistical methodology: III. Receiver operating characteristic (ROC) curves.

Measures including sensitivity, specificity, and positive and negative predictive values have been traditionally used to assess a diagnostic test's ability to detect the presence or absence of disease. Receiver operating characteristic (ROC) curve analysis allows visual evaluation of the trade-offs between sensitivity and specificity associated with different values of the test result, or different "cutpoints" for defining a positive result. The purpose of this article is to define, construct, and interpret a ROC curve using a hypothetical example applicable to emergency medicine practice.

Acute Disease↗
PubMed1999

"Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation.

The conventional binormal model, which assumes that a pair of latent normal decision-variable distributions underlies ROC data, has been used successfully for many years to fit smooth ROC curves. However, if the conventional binormal model is used for small data sets or ordinal-category data with poorly allocated category boundaries, a "hook" in the fitted ROC may be evident near the upper-right or lower-left corner of the unit square. To overcome this curve-fitting artifact, we developed a "proper" binormal model and a new algorithm for maximum-likelihood (ML) estimation of the corresponding ROC curves. Extensive simulation studies have shown the algorithm to be highly reliable. ML estimates of the proper and conventional binormal ROC curves are virtually identical when the conventional binormal ROC shows no "hook," but the proper binormal curves have monotonic slope for all data sets, including those for which the conventional model produces degenerate fits. Copyright 1999 Academic Press.

Journal Article↗
PubMed2001

The role of scintimammography and mammography in recurrent breast cancer. Evaluation of their accuracy using ROC curves.

BACKGROUND: With the increasing demand for breast conservation surgery, the probability of recurrent tumour within the breast increases. Traditionally x-ray mammography (XMM) was used to assess the post-surgical breast, but post-surgery and radiotherapy changes have reduced the accuracy of this method. Scintimammography (SMM) has also been proposed and appears to be more accurate than XMM. MATERIAL AND METHODS: A total of 101 women received (Tc99m) MIBI SMM and 88 had a subsequent XMM. There were 142 sites suspected of loco-regional recurrence breast cancer. During the study the patients did not receive any treatment other then hormonotherapy. SMM was performed by the standard Diggles-Khalkhali method and XMM was performed using standard 2 views. Analysis was performed and the results of each type of imaging compared with histology. In the ROC curve analysis 5 points of certainty were used: from 1 being definitely normal to 5 being definitely cancer; grades 4 and 5 were counted as positive. RESULTS: The overall sensitivity value of SMM was 84% and specificity was 85%, compared with a sensitivity of 52% for XMM and a specificity of 84%. Analysis of areas under ROC curves provides statistically significant difference between SMM and XMM (p < 0.05). Combining the two tests did not significantly improve the diagnostic accuracy of sequence imaging over SMM. CONCLUSION: ROC curve analysis demonstrates that scintimammography should be the primary investigation in suspected local recurrence following breast conservation surgery.

Journal Article↗
PubMed2002

[Evaluation of diagnostic tests performance: the ROC curve].

Most of the diagnostic tests applied in clinic measure continuous or discrete ordinal variables and do not provide an "yes or no" response. Therefore, we can not assess tests nor compare them on the basis of a single pair of values of sensitivity and specificity. The ROC curve plots values of sensitivity against 1-specificity and allows evaluation of the discriminatory power of a test for all ranges of sensitivity and specificity when applied in two populations, one presenting the target disease and the other not. Evaluation of the discriminatory power or comparison of different tests are further deducted from the measure of the area under the curve. The discriminatory power of the test demonstrated, one has to determine the cutoff point on the ROC curve, that will provide the best information for clinical application.

Diagnostic Tests, Routine↗
PubMed1997

Limitations to the robustness of binormal ROC curves: effects of model misspecification and location of decision thresholds on bias, precision, size and power.

This paper concerns robustness of the binormal assumption for inferences that pertain to the area under an ROC curve. I applied the binormal model to rating method data sets sampled from bilogistic curves and observed small biases in area estimates. Bias increased as the range of decision thresholds decreased. The variance of area estimates also increased as the range of decision thresholds decreased. Together, minor bias and inflated variance substantially altered the size and power of statistical tests that compared areas under bilogistic ROC curves. I repeated the simulations by applying the binormal assumption to data sampled from binormal curves. Biases in area estimates were minimal for the binormal data, but the variance of area estimates was again higher when the range of decision thresholds was narrow. The size of tests that compared areas did not vary from the chosen significance level. Power fell, however, when the variance of area estimates was inflated. I conclude that inferences derived from the binormal assumption are sensitive to model misspecification and to the location of decision thresholds. A narrow span of decision thresholds increases the variability of area estimates and reduces the power of area comparisons. Model misspecification produces bias that alters test size and can exaggerate the loss of power that accompanies increased variability.

Bias↗
PubMedDate not supplied

Bayesian analysis of ROC curves using Markov-chain Monte Carlo methods.

The authors introduce a Bayesian approach to generalized linear regression models for rating data observed in the evaluation of a diagnostic technology. Such models were previously studied using a non-Bayesian approach. In a Bayesian analysis, the difficulties inherent in an ordinal rating scale are circumvented by using data-augmentation techniques. Posterior distributions for the regression parameters- and thereby for receiver operating characteristic (ROC) curve parameters and values, for the area under a ROC curve, differences between areas, etc.-may then be computed by Markov-chain Monte Carlo methods. Inferences are made in standard Bayesian ways. The methods are exemplified by a study of ultrasonography rating data for the detection of hepatic metastases in patients with colon or breast cancer (previously analyzed) and the results compared.

Bayes Theorem↗