Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

An Analysis of Content Validity of Behavioral Domain of Descriptive Tests and Factors that Affect Content Validity: Focus on the Fifth and Sixth Grade Science

초등학교 과학과 5, 6학년 서술형 평가문항의 행동영역 내용타당도 및 이에 영향을 미치는 요인 분석

  • Received : 2015.12.14
  • Accepted : 2016.01.12
  • Published : 2016.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

This study analyzes the content validity of descriptive tests developed for elementary schools, in order to acquire basic data to improve them. Various descriptive tests were collected and tested for differences in proportions between two-dimensional classification of educational objectives and the level of behavioral objectives. Results show that the descriptive tests developed by elementary school teachers mainly focused on "knowledge" and "understanding," and that content validity for behavioral levels to be low. Nine elementary school teachers were interviewed to understand the result. From the interviews, we found both internal and external factors that cause low content validity. The main internal factors were teachers' ability to make two-dimensional classification of educational objectives, the teachers' consideration of students' level, item level of difficulty, the ease of scoring, and path dependence. The main external factors were curriculum, parents, and administration. Based on the results, we suggested the factors related to elementary school teachers' PCK of descriptive tests.

본 연구의 목적은 초등학교에서 개발 시행된 서술형 평가 문항의 내용타당도를 분석하는데 있으며, 이 평가 문항들의 개선을 위한 기초자료를 제시하는데 연구의 의의가 있다. 이를 위하여 여러 초등학교의 서술형 평가문항을 수집하고, 이원분류표의 평가목표와 문항의 평가목표가 요구하는 행동소를 비율차 검정하였다. 분석의 결과 교사가 제작한 서술형 평가문항은 '지식', '이해'를 주로 측정하고 있으며, 행동영역의 내용타당도가 낮음을 확인하였다. 내용타당도가 낮은 결과를 설명하기 위해 9명의 초등 교사를 대상으로 면담을 실시하였다. 면담의 결과 초등학교 과학과 서술형 평가문항의 내용타당도 확보를 저해하는 요인으로 교사내적요인과 교사외적요인을 추출하였다. 교사내적요인에는 바르지 않은 이원분류표의 작성법, 초등학생의 발달 단계 고려, 난이도, 채점의 용이성, 경로의존성 등이 포함되었다. 그리고 교사외적으로는 교육과정 및 학부모 그리고 행정적 요소 등이었다. 이상의 결과를 바탕으로 과학교사의 서술형 평가전문성을 위한 요인들을 제언하였다.

Keywords

References

  1. Arthur, W. B. (1994). Increasing returns and path dependence in the economy. Ann Arbor: University of Michigan Press.
  2. Assessment Reform Group (2006). The role of teachers in the assessment of learning. London, UK: Institution of Education, University of London. Retrieved from http://www.nuffieldfoundation.org/sites/default/files/files/The-role-of-teachers-in-the-assessment-of-learning.pdf
  3. Cho, A., Moon, K., You, M., & Kim, Y. (2013). A tool for assessing the middle school student's science writing and its reliability. Biology Education, 41(4), 675-684. https://doi.org/10.15717/bioedu.2013.41.4.675
  4. Choi, E. (2011). A research on teachers and students awareness and the characteristics of question in middle school science essay test. (Master's thesis). Korea National University of Education.
  5. Choi, J., & Paik, S. (2015). A comparative analysis of achievement standards of the 2007 & 2009 revised elementary science curriculum with next generation science standards in US based on Bloom's revised taxonomy. Journal of the Korean Association for Science Education, 35(2), 277-288. https://doi.org/10.14697/jkase.2015.35.2.0277
  6. Gullickson, A. P. (1984). Teacher perspectives of their instructional use of test. Journal of Educational Research, 77(4), 244-248. https://doi.org/10.1080/00220671.1984.10885532
  7. Hahn, A., Kang, H., Kwon, C., Kim, H., & Woo, J. (1997). New elementary science teaching method. Seoul: Education science company.
  8. Hong, M., & Chung, H. (2006). The development of assessment items with essay & description type test science in biology for high school students. Journal of Science Education, 30, 65-100.
  9. Kang, H., Chong, C., & Choi, Y. (2005). An alternative exploration of Bloom's taxonomy of educational objectives : On the basis of the teachers' interview. Secondary Education Research, 53(1), 51-84.
  10. Kang, H., Gong, Y., Kwon, H., Kim, J., Bae, J., Song, M., Shin, Y., Yang, I., Yoon, H., Lee, D., Lee, M., Im, C., Im, H., Jang, S., Jeon, Y., & Chae, D. (2007). Elementary science education theory. Seoul: Education Science Company.
  11. Kim, D., & Kim, S. (2011). Curriculum and educational evaluation. Seoul: Hakjisa.
  12. Kim, E., & Han, S. (2010). The analysis of national assessment of educational achievement items using three dimensional scientific assessment framework. Journal of the Korean Society of Earth Science Education, 3(3), 248-256.
  13. Kim, E., & Kang, W. (2010). An analysis of content validity of third-grade mathematics achievement tests. Journal of Elementary Mathematics Education in Korea, 14(2), 177-196.
  14. Kim, H., & Yoo, J. (2010). An analysis on reliabilities of scoring methods and rubric ratings number for performance assessments of middle school students' science investigation activities. Journal of the Korean Association for Science Education, 30(2), 275-290.
  15. Kim, J., Boo, J., So, K., & Yang, K. (2010). Curriculum and educational evaluation. (4th ed.). Paju: Kyoyookgwahaksa.
  16. Kim, S., & Jhun, Y. (2005). For better evaluation of students in the science class (Research results, ORM 2005-51-5). Seoul: Korea Institute for Curriculum and Evaluation.
  17. Kim, Y., & Kim, Y. (2012). The development of a free-response test for the assessment of science process skills. Biology Education, 40(1), 167-177. https://doi.org/10.15717/bioedu.2012.40.1.167
  18. Klassen, S. (2006). Contextual assessment in science education: Background, issues and policy. Science Education, 90(5), 820-851. https://doi.org/10.1002/sce.20150
  19. Kwon, J., Kim, B., Choi, B., Kim, H., Paik, S., Yang, I., Kwon, Y., Cha, H., Woo, J., & Jeong, J. (2012). Science education. Seoul: Kyoyookgwahaksa.
  20. Kyungkido Education Office. (2012). Guidebook of descriptive and essay evaluation. Kyungkido: Kyungkido Education Office.
  21. Lee, D., & Jeong, E. (2014). An analysis of paper and pencil test items of life scienceI in high school. Journal of Science Education., 38(3), 670-690. https://doi.org/10.21796/jse.2014.38.3.670
  22. Lee, K., & An, H. (2005). Analysis of assessment types, scoring methods and reliability of science performance assessment in middle and high school. Journal of the Korean Association for Science Education, 25(2), 173-183.
  23. Lee, M. (2013). An analysis of the current situation regarding high school level written examinations for chemistry. (Master's thesis). Korea National University of Education.
  24. McAfee, G. (2011). Master math: AP statistics. Boston: Course Technology, a part of Cengage Learning.
  25. Ministry of Educational and Science Technology. (2009). Elementary and secondary school curriculum : General statement (Notification No. 2009-41 of the MEST). Seoul: Ministry of Educational and Science Technology.
  26. Ministry of Educational and Science Technology. (2012). Science curriculum (Notification No. 2011-361 of the MEST). Seoul: Ministry of Educational and Science Technology.
  27. National Research Council. (1996). National science education standards. Washington D.C.: National Academy Press.
  28. National Research Council. (2006). Systems for state assessment. Committee on test design for K-12 achievement. Washington D.C.: National Academy Press.
  29. Paik, S., Lee, E., Kim, J., Song, Y., Kim, Y., Chung, J., & Han, J. (2008). Analysis of the content validity of the achievement evaluation items on the "water" chapter in the high school chemistry I course. Journal of Research in Curriculum Instruction, 12(1), 55-65. https://doi.org/10.24231/rici.2008.12.1.55
  30. Paik, S., & Ryu, H. (2014). High school students' perceptions in descriptive assessment activity experiences by teacher or by peer. Journal of the Korean Association for Science Education, 34(6), 593-599. https://doi.org/10.14697/jkase.2014.34.6.0593
  31. Park, H. (2012). Analysis of summative evaluation objectives in middle school science examination by Klopfer's taxonomy of educational objectives. Journal of Science Education, 36(2), 293-302. https://doi.org/10.21796/jse.2012.36.2.293
  32. Park, I., & Kang, S. (2012). The development of assessment tools to measure scientific creative problem solving ability for middle school students. Journal of the Korean Association for Science Education, 32(2), 210-235. https://doi.org/10.14697/jkase.2012.32.2.210
  33. Pine, J., Aschbacher, P., Rothem, E., Jones, M., McPhee, C., Martin, C., Phelps, S., Kyle, T., & Foley, B. (2006). Fifth graders' science inquiry abilities: A comparative study of students in hands-on and textbook curricula. Journal of Research in Science Teaching, 43(4), 467-484. https://doi.org/10.1002/tea.20140
  34. Shim, A. (2008). The study on the recognition of teachers and students & actual conditions about middle school science essay test. (Master's thesis). Korea National University of Education.
  35. Seong, T. (2014). The foundation of educational evaluation. (2nd ed.). Seoul: Hakjisa.
  36. Song, J. (2003). An analysis of content validity of teacher-made summative evaluation. (Master's thesis). Korea National University of Education.
  37. Song, K., Lee, H., & Lim, C. (2004). Development of a test of science inquiry skills for elementary school fifth and sixth graders. Journal of the Korean Association for Science Education, 24(6), 1245-1255.
  38. Stiggins, R. J., & Bridgeford, N. J. (1985). The ecology of classroom assessment. Journal of Educational Measurement, 22(4), 271-286. https://doi.org/10.1111/j.1745-3984.1985.tb01064.x
  39. Tyler. R. W. (1949). Basic principles of curriculum and instruction. Chicago: University of Chicago Press.
  40. Yang, G. (2006). Descriptive and essay assessment source book. Seoul: Seoul Metropolitan Office of Education.
  41. Yang, I., Na, J., Lim, S., Lim, J., & Choi, H. (2008). An analysis of elementary schools' science test items by Klopfer's taxonomy of educational objectives: Focusing on the first term of the 5th grade. Journal of Korean Elementary Science Education, 27(3), 221-232.

Cited by

  1. 중등 예비 화학교사의 지필평가 문항 제작 과정에서 고려된 교과교육학 지식(PCK) 구성 요소 사이의 상호작용 vol.36, pp.5, 2016, https://doi.org/10.14697/jkase.2016.36.5.0769
  2. 발성 사고법을 이용한 예비 화학 교사의 지필평가 문항 제작 과정의 특징 분석 vol.37, pp.2, 2016, https://doi.org/10.14697/jkase.2017.37.2.0225
  3. 중학교 과학에서 서술형 평가의 실태 분석: 문항유형, 단원, 평가목표를 중심으로 vol.42, pp.2, 2016, https://doi.org/10.21796/jse.2018.42.2.198
  4. 중등 예비 화학교사의 해결자·청취자 활동을 통한 지필평가 문항 제작 과정에서 언어적 행동 및 상호작용 vol.38, pp.5, 2016, https://doi.org/10.14697/jkase.2018.38.5.611
  5. Application of Bloom’s taxonomy to formative assessment in real-time online classes in Korea vol.33, pp.3, 2016, https://doi.org/10.3946/kjme.2021.199