Journal of the Korean Medical Association (대한의사협회지)

Korean Medical Association (대한의사협회)
권호 표지
DOI QR코드

DOI QR Code

Tools for assessing quality and risk of bias by levels of evidence

근거수준별 문헌의 질 평가 도구

  • Lim, Sun Mi (Research Institute for Healthcare Policy, Korean Medical Association) ;
  • Shin, Ein Soon (Department of Preventive Medicine, Ewha Womans University School of Medicine) ;
  • Lee, Sun Hee (Department of Preventive Medicine, Ewha Womans University School of Medicine) ;
  • Seo, Kyung Hwa (Research Institute for Healthcare Policy, Korean Medical Association) ;
  • Jung, Yu Min (Department of Preventive Medicine, Ewha Womans University School of Medicine) ;
  • Jang, Ji Eun (Department of Preventive Medicine, Ewha Womans University School of Medicine)
  • 임선미 (대한의사협회 의료정책연구소) ;
  • 신인순 (이화여자대학교 의과대학 예방의학교실) ;
  • 이선희 (이화여자대학교 의과대학 예방의학교실) ;
  • 서경화 (대한의사협회 의료정책연구소) ;
  • 정유민 (이화여자대학교 의과대학 예방의학교실) ;
  • 장지은 (이화여자대학교 의과대학 예방의학교실)
  • Published : 2011.04.10
⟨ Previous Next ⟩

Abstract

Tools for assessing methodological quality or risk of bias in randomized controlled trials (RCTs) and non-randomized studies (NRS) were reviewed. The van Tulder scale and Cochrane's assessment of risk of bias are the two most useful methodological quality evaluation tools for RCTs. Cochrane's tool includes sequence generation, allocation of sequence concealment, blinding, incomplete outcome data, selective outcome reporting, and other potential sources of bias. The Cochrane Collaboration Group recommends the Downs and Black instrument and the Newcastle-Ottawa Scale for evaluating the quality of NRS. In conclusion, this study offers useful information to physicians about tools for assessing the quality of evidence in clinical guidelines. Further research is needed to provide an essential core for evidence-based decision making regarding levels and/or grades of recommendations.

Keywords

References

  1. Higgins JP, Green S, editors. Cochrane handbook for systematic reviews of interventions. Ver. 5.1.0. The Cochrane Collaboration; 2011 [cited 2011 Jan 7]. Available from: http://www.cochrane-handbook.org.
  2. Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A. A method for assessing the quality of a randomized control trial. Control Clin Trials 1981;2:31-49. https://doi.org/10.1016/0197-2456(81)90056-8
  3. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16:62-73. https://doi.org/10.1016/0197-2456(94)00031-W
  4. Barratt A. Evidence based medicine and shared decision making: the challenge of getting both evidence and preferences into health care. Patient Educ Couns 2008;73:407-412. https://doi.org/10.1016/j.pec.2008.07.054
  5. Kranke P. Evidence-based practice: how to perform and use systematic reviews for clinical decision-making. Eur J Anaesthesiol 2010;27:763-772. https://doi.org/10.1097/EJA.0b013e32833a560a
  6. Parrilla-Castellar ER, Almeyda R, Nogales E, Velez M, Ramos M, Rivera JE, Da Vila B, Torres V, Capriles J, Adamsons K. Evidence- based medicine as a tool for clinical decision-making in Puerto Rico. P R Health Sci J 2008;27:135-140.
  7. Tiburi MF. Evidence-based medicine as viewed by key decision- makers of health plans in southern Brazil. Health Serv Manage Res 2008;21:185-191. https://doi.org/10.1258/hsmr.2007.007022
  8. Stolba N, Nguyen TM, Tjoa AM. Towards sustainable decisionsupport system facilitating EBM. Conf Proc IEEE Eng Med Biol Soc. 2007;2007:4355-4358.
  9. Wyer PC, Silva SA. Where is the wisdom? I. a conceptual history of evidence-based medicine. J Eval Clin Pract 2009;15:891-898. https://doi.org/10.1111/j.1365-2753.2009.01323.x
  10. Collins J. Evidence-based medicine. J Am Coll Radiol 2007;4:551-554. https://doi.org/10.1016/j.jacr.2006.12.007
  11. Anastasiu M, Strambu V, Popa F. Evidence-based medicine, conceptual challenge or the future of daily practice? Chirurgia (Bucur) 2007;102:527-530.
  12. Carter MJ. Evidence-based medicine: an overview of key concepts. Ostomy Wound Manage 2010;56:68-85.
  13. Maluf-Filho F. The importance of evidence-based medicine concepts for the clinical practitioner. Arq Gastroenterol 2009;46:87-89. https://doi.org/10.1590/S0004-28032009000200002
  14. Borgerson K. Valuing evidence: bias and the evidence hierarchy of evidence-based medicine. Perspect Biol Med 2009;52:218-233. https://doi.org/10.1353/pbm.0.0086
  15. Isaac CA, Franceschi A. EBM: evidence to practice and practice to evidence. J Eval Clin Pract 2008;14:656-659. https://doi.org/10.1111/j.1365-2753.2008.01043.x
  16. Kruer MC, Steiner RD. The role of evidence-based medicine and clinical trials in rare genetic disorders. Clin Genet 2008;74:197-207. https://doi.org/10.1111/j.1399-0004.2008.01041.x
  17. Rogers W, Ballantyne A. Justice in health research: what is the role of evidence-based medicine? Perspect Biol Med 2009;52:188-202. https://doi.org/10.1353/pbm.0.0082
  18. Kitto S, Petrovic A, Gruen RL, Smith JA. Evidence-based medicine training and implementation in surgery: the role of surgical cultures. J Eval Clin Pract 2010 Aug 4 [Epub]. DOI: 10.1111/j.1365-2753.2010.01526.x.
  19. Carretier J, Bataillard A, Fervers B. The patient's role in evidence-based medicine. J Chir (Paris) 2009;146:537-544.
  20. Soll RF. Evaluating the medical evidence for quality improvement. Clin Perinatol 2010;37:11-28. https://doi.org/10.1016/j.clp.2010.01.002
  21. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG; International Stroke Trial Collaborative Group; European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:iii-x, 1-173.
  22. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess 2000;4:1-154.
  23. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:1-12. https://doi.org/10.1016/0197-2456(95)00134-4
  24. van Tulder M, Furlan A, Bombardier C, Bouter L; Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976) 2003;28:1290-1299.
  25. Newell SA, Sanson-Fisher RW, Savolainen NJ. Systematic review of psychological therapies for cancer patients: overview and recommendations for future research. J Natl Cancer Inst 2002;94:558-584. https://doi.org/10.1093/jnci/94.8.558
  26. Scottish Intercollegiate Guideline Network. SIGN 50: a guideline developer's handbook 2008 [Internet]. Edinburgh: Scottish Intercollegiate Guidelines Network; 2008 [cited 2011 Jan 28]. Available from: http://www.sign.ac.uk/pdf/sign50.pdf.
  27. National Institute for Health and Clinical Excellence. The guidelines manual 2009 [Internet]. London: National Institute for Health and Clinical Excellence; 2009 [cited 2011 Jan 27]. Available from: http://www.nice.org.uk/media/5F2/44/The_guidelines_manual_2009_-_All_chapters.pdf.
  28. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377-384. https://doi.org/10.1136/jech.52.6.377
  29. Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [Inter net]. Ottawa: Ottwa Hospital Research Institute; c1996-2010 [cited 2011 Mar 28]. Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
  30. Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics 1989;84:815-827.
  31. Thomas H. Quality assessment tool for quantitative studies: effective public health practice project. Hamilton: McMaster University; 2003.
  32. Zaza S, Wright-De Agüero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, Sosin DM, Anderson L, Carande-Kulis VG, Teutsch SM, Pappaioanou M. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med 2000;18(1 Suppl):44-74. https://doi.org/10.1016/S0749-3797(99)00122-1

Cited by

  1. 간호연구 질 평가 도구 및 보고지침 고찰 vol.14, pp.3, 2012, https://doi.org/10.7586/jkbns.2012.14.3.221
  2. A New Dimension of Health Care: Systematic Review of the Uses, Benefits, and Limitations of Social Media for Health Communication vol.15, pp.4, 2013, https://doi.org/10.2196/jmir.1933
  3. 국내 간호학 분야 메타분석 논문의 질 평가 vol.43, pp.6, 2011, https://doi.org/10.4040/jkan.2013.43.6.736
  4. 기본간호학회지에 출판된 비무작위 실험연구의 방법론적 질 평가 vol.21, pp.3, 2011, https://doi.org/10.7739/jkafn.2014.21.3.311
  5. 한방안이비인후피부과학회지에 게재된 임상실험연구에 대한 고찰 vol.27, pp.4, 2014, https://doi.org/10.6114/jkood.2014.27.4.001
  6. 긴장성두통에 대한 한약치료: 체계적 문헌고찰 및 메타분석 vol.26, pp.4, 2015, https://doi.org/10.7231/jon.2015.26.4.383
  7. 국내 고위험신생아 연구 분석 및 중재 연구의 질 평가 vol.22, pp.2, 2016, https://doi.org/10.4094/chnr.2016.22.2.153
  8. 여드름의 화침치료 효과에 대한 체계적 문헌 고찰 vol.33, pp.2, 2016, https://doi.org/10.13045/acupunct.2016024
  9. Quality analysis of randomized controlled trials in the International Journal of Impotence Research: quality assessment and relevant clinical impact vol.29, pp.2, 2011, https://doi.org/10.1038/ijir.2016.48
  10. 심부전 환자 대상 국내 양적 간호연구 분석 및 중재연구의 질 평가 vol.19, pp.4, 2011, https://doi.org/10.7586/jkbns.2017.19.4.227
  11. Night Shift Work Increases the Risks of Multiple Primary Cancers in Women: A Systematic Review and Meta-analysis of 61 Articles vol.27, pp.1, 2011, https://doi.org/10.1158/1055-9965.epi-17-0221
  12. 간호교육에 적용한 가상현실 융합시뮬레이션 연구에 대한 통합적 고찰 vol.10, pp.1, 2011, https://doi.org/10.22156/cs4smb.2020.10.01.060
  13. Effectiveness of Percutaneous Nephrolithotomy, Retrograde Intrarenal Surgery, and Extracorporeal Shock Wave Lithotripsy for Treatment of Renal Stones: A Systematic Review and Meta-Analysis vol.57, pp.1, 2021, https://doi.org/10.3390/medicina57010026
  14. 뇌질환 환자에서 전산화 인지치료의 임상적 유용성 : 체계적 고찰 및 메타분석 vol.8, pp.4, 2020, https://doi.org/10.15268/ksim.2020.8.4.277