Journal of Veterinary Clinics (한국임상수의학회지)

The Korean Society of Veterinary Clinics (한국임상수의학회)
권호 표지
DOI QR코드

DOI QR Code

Application of Receiver Operating Characteristic (ROC) Curve for Evaluation of Diagnostic Test Performance

진단검사의 특성 평가를 위한 Receiver Operating Characteristic (ROC) 곡선의 활용

  • Pak, Son-Il (College of Veterinary Medicine and Institute of Veterinary Science, Kangwon National University) ;
  • Oh, Tae-Ho (College of Veterinary Medicine, Kyungpook National University)
  • 박선일 (강원대학교 수의과대학 및 동물의학종합연구소) ;
  • 오태호 (경북대학교 수의과대학)
  • Received : 2015.12.09
  • Accepted : 2016.04.14
  • Published : 2016.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In the field of clinical medicine, diagnostic accuracy studies refer to the degree of agreement between the index test and the reference standard for the discriminatory ability to identify a target disorder of interest in a patient. The receiver operating characteristic (ROC) curve offers a graphical display the trade-off between sensitivity and specificity at each cutoff for a diagnostic test and is useful in assigning the best cutoff for clinical use. In this end, the ROC curve analysis is a useful tool for estimating and comparing the accuracy of competing diagnostic tests. This paper reviews briefly the measures of diagnostic accuracy such as sensitivity, specificity, and area under the ROC curve (AUC) that is a summary measure for diagnostic accuracy across the spectrum of test results. In addition, the methods of creating an ROC curve in single diagnostic test with five-category discrete scale for disease classification from healthy individuals, meaningful interpretation of the AUC, and the applications of ROC methodology in clinical medicine to determine the optimal cutoff values have been discussed using a hypothetical example as an illustration.

Keywords

References

  1. Burfeind O, Sannmann I, Voigtsberger R, Heuwieser W. Receiver operating characteristic curve analysis to determine the diagnostic performance of serum haptoglobin concentration for the diagnosis of acute puerperal metritis in dairy cows. Anim Reprod Sci 2014; 149: 145-151. https://doi.org/10.1016/j.anireprosci.2014.07.020
  2. Collinson P. Of bombers, radiologists, and cardiologists: time to ROC. Heart 1998; 80: 215-217. https://doi.org/10.1136/hrt.80.3.215
  3. Denis-Robichaud J, Dubuc J, Lefebvre D, DesCteaux L. Accuracy of milk ketone bodies from flow-injection analysis for the diagnosis of hyperketonemia in dairy cows. J Dairy Sci 2014; 97: 3364-3370. https://doi.org/10.3168/jds.2013-6744
  4. Doi K. Diagnostic imaging over the last 50 years: research and development in medical imaging science and technology. Phys Med Biol 2006; 51: R5-27. https://doi.org/10.1088/0031-9155/51/13/R02
  5. Enachescu V, Ionita M, Mitrea IL. Comparative study for the detection of antibodies to Neospora caninum in milk and sera in dairy cattle in southern Romania. Acta Parasitol 2014; 59: 5-10.
  6. Eng J. Receiver operating characteristic analysis: a primer. Acad Radiol 2005; 12: 909-916. https://doi.org/10.1016/j.acra.2005.04.005
  7. Faraggi D, Reiser B. Estimation of the area under the ROC curve. Stat Med 2002; 21: 3093-3106. https://doi.org/10.1002/sim.1228
  8. Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 2013; 4: 627-635.
  9. Lasko TA, Bhagwat JG, Zou KH, Ohno-Machado L. The use of receiver operating characteristic curves in biomedical informatics. J Biomed Inform 2005; 38: 404-415. https://doi.org/10.1016/j.jbi.2005.02.008
  10. Li J, Jiang B, Fine JP. Multicategory reclassification statistics for assessing improvements in diagnostic accuracy. Biostatistics 2013; 14: 382-394. https://doi.org/10.1093/biostatistics/kxs047
  11. Lusted LB. Logical analysis in roentgen diagnosis. Radiology 1960; 74: 178-193. https://doi.org/10.1148/74.2.178
  12. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1978; 8: 283-298. https://doi.org/10.1016/S0001-2998(78)80014-2
  13. Obuchowski NA. ROC analysis. AJR Am J Roentgenol 2005; 184: 364-372. https://doi.org/10.2214/ajr.184.2.01840364
  14. Swets JA. Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull 1986; 99: 181-198. https://doi.org/10.1037/0033-2909.99.2.181
  15. Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240: 1285-1293. https://doi.org/10.1126/science.3287615
  16. Wan S, Zhang B. Smooth semiparametric receiver operating characteristic curves for continuous diagnostic tests. Stat Med 2007; 26: 2565-2586. https://doi.org/10.1002/sim.2726
  17. Yang ZH, Li L, Pan ZS. Development of multiple ELISAs for the detection of antibodies against classical swine fever virus in pig sera. Virol Sin 2012; 27: 48-56. https://doi.org/10.1007/s12250-012-3227-z
  18. Zhou XH, Li CM, Yang Z. Improving interval estimation of binomial proportions. Philos Trans A Math Phys Eng Sci 2008; 366: 2405-2418. https://doi.org/10.1098/rsta.2008.0037
  19. Zou KH, Hall WJ, Shapiro DE. Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. Stat Med 1997; 16: 2143-2156. https://doi.org/10.1002/(SICI)1097-0258(19971015)16:19<2143::AID-SIM655>3.0.CO;2-3
  20. Zou KH, O'Malley AJ, Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 2007; 115: 654-657. https://doi.org/10.1161/CIRCULATIONAHA.105.594929
  21. Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 1993; 39: 561-577.

Cited by

  1. 고속도로 이용자의 승차감 평가특성 및 만족도 분석과 ROC 곡선을 이용한 평탄성 관리기준 적정성 검토 vol.20, pp.2, 2016, https://doi.org/10.7855/ijhe.2018.20.2.035
  2. A reliable method for colorectal cancer prediction based on feature selection and support vector machine vol.57, pp.4, 2016, https://doi.org/10.1007/s11517-018-1930-0
  3. 한국형 낙상 위험 사정도구의 타당성 평가연구 vol.26, pp.2, 2020, https://doi.org/10.22650/jkcnr.2020.26.2.265
  4. Nurses' Assessment of Sedation of Patients With Mental Illness: Scale Development and Validation vol.58, pp.12, 2016, https://doi.org/10.3928/02793695-20200918-01
  5. 외측 발목 염좌 병력에 따른 정적 하지 정렬 차이: 외측 발목 염좌의 예측인자로서 정적 하지 정렬 검사의 효용성과 한계점 vol.31, pp.1, 2016, https://doi.org/10.5103/kjsb.2021.31.1.1
  6. Determination of cut-off value by receiver operating characteristic curve of norquetiapine and 9-hydroxyrisperidone concentrations in urine measured by LC-MS/MS vol.34, pp.2, 2016, https://doi.org/10.5806/ast.2021.34.2.78
  7. Comparative Analysis of Anomaly Detection Models using AE and Suggestion of Criteria for Determining Outliers vol.26, pp.8, 2016, https://doi.org/10.9708/jksci.2021.26.08.023