Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지

A study on validity and reliability of students' evaluation

강의평가의 타당성과 신뢰성에 관한 연구 전주대학교 강의평가 결과를 중심으로

  • Published : 2010.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

This research deals the method to assess the validity and reliability of students' evaluation for lectures. Most papers for student's evaluation have focused the procedures for controlling the external effects, but this paper is trying to answer for "How reliable is the student rating?" An empirical study shows that the evaluations in Jeonju University have the fair validity and reliability. The generalizability theory is suggested to obtain the more comprehensive results rather than Cronbach's alpha to examine internal consistency.

본 논문은 강의평가의 타당성과 신뢰성을 측정하는 방법을 소개하고 실제자료를 이용하여 타당성과 신뢰성을 평가하였다. 기존의 강의평가 관련논문이 강의평가에 미치는 외생적인 영향을 통제하는 데 주력한 데 반해, 교원의 신분에 직접적인 영향을 미칠 수 있는 중대한 강의평가가 과연 믿을 만한 평가인가에 관한 근원적인 질문에 답하려 하였다. 전주대학교의 강의평가 결과를 실증 분석한 결과 타당성과 신뢰성 면에서 어느 정도 만족할 만한 수준임을 확인할 수 있었다. 본 논문에서는 기존에 간편하게 사용되던 신뢰성 측도가 아닌 일반화가능도 계수를 이용하여 신뢰성을 평가하는 방법을 자세히 소개하고 그 장점을 설명하였다.

Keywords

References

  1. 김성연, 권치명 (2005). 통계적 기법을 활용한 균등화법에 의한 강의평가 개선방안 연구. <한국자료분석학회지>, 7, 1705-1721.
  2. 김영진 (1994). 교수강의평가제-과연 생산성이 있는가. <생산성논집>, 8, 252-235.
  3. 김종태 (2004). A study of reliability of lecture evaluation by students. <한국데이터정보학회지>, 15, 183-191.
  4. 류춘호, 이정호 (2003). 대학의 강의평가에 영향을 미치는 학생관련 요인에 관한 연구. <경영학연구>, 32, 789-807.
  5. 박노진 (2009). 핵심 문항들을 활용한 모델링-강의 평가 자료를 활용한 사례연구. <한국데이터정보학회지>, 20, 1075-1083.
  6. 성태제 (2002). <타당도와 신뢰도>, 학지사, 서울.
  7. 이기훈 (2008). 크론바하 신뢰도 계수에 관한 이해. <산경논총>, 28, 43-54.
  8. 이종승 (1995). <교육연구법>, 배영사, 서울.
  9. 조장식, 강창완, 최승배 (2009). 강의평가에 대한 균등화방법의 비교. <한국데이터정보학회지>, 20, 65-75.
  10. 한신일 (2002). 강좌규모와 강의평가의 관계분석. <고등교육연구>, 13, 155-173.
  11. Ahmadi, M., Helms, M. and Ralszadeh, F. (2001). Business students' perceptions of faculty evaluations. The International Journal of Educational Management, 15, 12-22.
  12. American educational research association, American psychological association and national council on measurement in education (AERA, APA and NCME) (1999). Standard for educational and psychological testing, American Psychological Association, Washington D. C..
  13. Baik, T. and Yang, G. (2008). Classroom lecture monitoring case study. Journal of the Korean Data & Information Science Society, 19, 1191-1200.
  14. Brennan, R. L. (2001). Generalizability theory, Springer, New York.
  15. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. https://doi.org/10.1007/BF02310555
  16. Cronbach, L. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391-418. https://doi.org/10.1177/0013164404266386
  17. Cronbach, L. J., Gleser, G. C., Nanda, H. and Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles, Wiley, New York.
  18. Cronbach, L. J., Rajaratnam, N. and Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. The British Journal of Statistical Psychology, 16, 137-163. https://doi.org/10.1111/j.2044-8317.1963.tb00206.x
  19. Crumbley, D. L. and Fliedner, E.(2002). Accounting administrators' perceptions of student evaluation of teaching(SET) information. Quality Assurance in Education, 10, 213-222. https://doi.org/10.1108/09684880210446884
  20. Kuder, G. F. and Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151-160. https://doi.org/10.1007/BF02288391
  21. Kulik, J. A. and McKeachie, W. J. (1975). The evaluation of teachers in higher education. Review of Research in Education, 3, 210-230.
  22. Lee, K. H. and Lee, S. W. (2006). A study on controlling the external effect in student evaluation of testing. The Korean Communications in Statistics, 12, 589-601. https://doi.org/10.5351/CKSS.2005.12.3.589
  23. Linn, R. L. (1997). Evaluating the validity of assessments: The consequences of use. Educational Measurement: Issue and Practice, 16, 14-15.
  24. Linn, R. L. and Gronlund, N. E. (2000). Measurement and assessment in teaching, 8th Ed., Upper Saddle River, NJ: Merrill.
  25. Mehrens, W. A. (1997). The consequences of consequential validity. Educational Measurement: Issues and Practice, 16, 16-18.
  26. Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35-44. https://doi.org/10.1023/A:1006964925094
  27. Mushquash, C. and O'Connor, B. P. (2006). SPSS and SAS programs for generalizability theory analyses. Behavior Research Methods, 38, 542-547. https://doi.org/10.3758/BF03192810
  28. Novick, M. R. and Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1. 1-13. https://doi.org/10.1007/BF02289400
  29. Popham, W. J. (1997). Consequential validity: Right concern-wrong concept. Educational Measurement: Issues and Practice, 16, 9-13.
  30. Shavelson, R. J. and Webb, N. M. (1991). Generalizability theory: A primer, Sage, Newbury Park, CA.
  31. Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16, 13-24.
  32. Spearman, C. (1910). Correlation calculated with faulty data. British Journal of Psychology, 3, 271-295.
  33. Webb, N. M., Rowley, G. L. and Shavelson, R. J. (1988). Using generalizability theory in counseling and development. Measurement & Evaluation in Counseling & Development, 21, 81-90.