Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of the Characteristics of National Assessment of Educational Achievement (NAEA) Items for Science Subject through the use of Option Response Rate Distribution Curve

답지 반응률 분포 곡선을 통한 국가수준 학업성취도 과학 평가 문항 특성 분석

  • Received : 2015.01.15
  • Accepted : 2015.02.18
  • Published : 2015.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Identifying students' achievement level and reflecting it on educational policy making or instructional improvement by analyzing the results of nationwide standardized assessment is an integral part of accountability in education. On the basis of this premise, we analyzed the characteristics of National Assessment of Educational Achievement (NAEA) items for middle school science subject conducted from 2010 to 2013 by using the option response rate distribution curve, the fittest graph estimated from the response rate of correct/incorrect options by achievement score. Furthermore, we classified the type of option response rate curve in terms of correct and incorrect options. Results of the analysis of option response curve showed that five types of correct option response curve (S-shaped, J-shaped, straight-shaped, F-shaped, and step-shaped) and 4 types of incorrect option response curve (down-slope, flat, mound, and up-slope) were identified. The most common type of items was the combination of S-shaped correct option response curve and down-slope incorrect option response curve, which are considered as appropriate items to discriminate the students according to achievement level. Moreover, correct option response was found to be correlated with incorrect option response. Based on the results, we also discussed some implications on teaching-learning method and classroom assessment in science education.

국가 수준의 표준화된 학업 성취도 평가 결과를 분석하여 학생들의 성취 수준을 파악하고 이를 교육정책 수립이나 교수활동 개선에 반영하는 과정은 교육의 책무성으로 중요한 부분이다. 본 연구에서는 과학과 국가 수준 학업 성취도 평가 결과를 분석하고, 답지 반응률 분포 곡선을 활용하여 평가 문항의 특성을 분석하고자 하였다. 이를 위해 2010년부터 2013년까지 시행된 과학과 성취도 평가 결과를 성취 수준에 따라 분석하였으며, 성취도 점수에 따른 특정 문항의 정오답 반응률을 토대로 최적 곡선을 추정한 그래프인 답지 반응률 분포 곡선을 활용하여 선다형 평가 문항 112개의 정답지와 오답지 반응률 곡선의 유형을 분류하고, 유형별 문항 특성을 분석하였다. 분석 결과, 정답지는 5가지 유형(S자형, J자형, 직선형, F자형, 계단형)으로 분류하였고, 오답지는 4가지 유형(보통형, 평지형, 산지형, 상승형)으로 분류하였다. S자형의 정답지 반응 곡선과 보통형의 오답지 반응 곡선이 조합인 문항이 가장 많았으며, 성취 수준에 따라 학생들을 변별하는데 적절한 문항인 것으로 분석되었다. 또한, 정답 반응률 분포와 오답 반응률 분포가 서로 연관되는 것으로 나타났다. 연구 결과를 토대로 교수 학습, 교실 평가 등에서 함의를 논의하였다.

Keywords

References

  1. Black, P. J. (1998). Testing: friend or foe? theory and practice of assessment and testing. London: Farmer.
  2. Choi, W. (2013). Comparison of the Characteristics of each Educational Achievement Level of Elementary and Middle School Students shown in the Chemistry Items of the National Assessment of Educational Achievement. Journal of the Korean Association for Science Education, 33(6), 1186-1201. https://doi.org/10.14697/jkase.2013.33.6.1186
  3. Choi, W., & Jeong, E. (2008). Relationship between Science Achievement and Student-related Variable in National Assessment of Educational Achievement in 2006. Journal of the Korean Association for Science Education, 28(8), 848-859.
  4. Corcoran, T., & Silander, M. (2009). Instruction in high schools: The evidence and the challenge. The Future of Children: America's High Schools, 19, 157-183.
  5. Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: an evidence-based approach to reform (Consortium for Policy Research in Education Report #RR-63). Philadelphia, PA: Consortium for Policy Research in Education.
  6. Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47(2), 123-182. https://doi.org/10.1080/03057267.2011.604476
  7. Jeong, E., & Choi, W. (2008). Trend Analysis of Students' Science Achievement in National Assessment of Educational Achievement from 2003 to 2006. Journal of the Korean Association for Science Education, 28(5), 438-452.
  8. Kim, D., Kim, D., Kim, Y., Kim, J., Lee, J., Seo, M., Jo, Y., Lee, K., Lee, I., Sim, J., Pae, J., Jung, G., Choi, W., & Park, Y. (2013a). A Study of the Features of Educational Achievement and Development of the National Assessment of Educational Achievement (NAEA) 2013. Korea Institute for Curriculum and Evaluation Research Paper RRE 2013-4.
  9. Kim, D., Lee, I., Kim, M., Jeong, E., Kang, H., & Choi, Y. (2012). Analysis of the 2011 National Assessment of Educational Achievement(Science). Korea Institute for Curriculum and Evaluation Research Paper ORM 2013-37-4.
  10. Kim, D., Lee, I., Sim, J., Choi, W., Park, Y., & Joo, K. (2013b). Analysis of the 2012 National Assessment of Educational Achievement(Science). Korea Institute for Curriculum and Evaluation Research Paper ORM 2013-37-4.
  11. Kim, H., & Jeong, J. (2012). An Analysis of the Result of National Assessment of Educational Achievement in Science at Grade 9. Journal of Science Education, 36(2), 394-407.
  12. Kim, H., Lee, I., Kim, M., Jeong, J., & Sin, M. (2011c). Analysis of the 2010 National Assessment of Educational Achievement(Science). Korea Institute for Curriculum and Evaluation Research Paper ORM 2011-3-5.
  13. Kim, K., Kim, W., Choi, I., Sang, K., Kim, H., Shin, J., Kim, J., & Sohn, W. (2011a). Characteristics of Educational Academic Improvement from NAEA(National Assessment of Educational Achievement) 2010. Korea Institute for Curriculum and Evaluation Research Paper RRE 2011-2-4.
  14. Kim, M., Kim, D., Kim, Y., Chin, J, Kim, H, Lee, B., Jo, Y., Lee, I., Kim, H., Lee, Y., & Lee, C. (2011b). A Study on the Development Process fo the National Assessment of Educational Achievement. Korea Institute for Curriculum and Evaluation Research Paper RRE 2011-3-1.
  15. Lee, C., Nam, M., Nam, G., Park, E., Kim, H., Kwon, J., Choi, W., Lee, I., KIm, M., Song, M, Jeong, E., Park, J., Sin, M., Choi, J., Ko, E., & Kim, D. (2010). A Study on improving the process of developing, implementing, and scoring the National Assessment of Educational Achievement (NAEA). Korea Institute for Curriculum and Evaluation Research Paper RRE 2010-6-1.
  16. Lee, I., Sim, J., Kim, H., Lee, K., Lee, B., & Jeong, I. (2014). Analysis of the 2013 National Assessment of Educational Achievement(Science). Korea Institute for Curriculum and Evaluation Research Paper ORM 2014-30-4.
  17. Ministry of Education & Human Resources Development (2007). National Science Curriculum.
  18. Ministry of Education and Science Technology(MEST) (2009). National Science Curriculum.
  19. National Research Council (2012). A framework for K-12 Science Education: Practices, cross-cutting concepts, and core ideas. Committee on a Conceptual Framework for New K-12 Science Education Standards. Washington DC: The National Academies Press.
  20. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. committee on science learning, kindergarten through eighth grade. R.A. Duschl, H.A. Schweingruber, & A.W. Shouse (Eds.). The National Academies Press.

Cited by

  1. Analysis on the Characteristics of National Assessment of Educational Achievement (NAEA) Items for Science Subject: With a Focus on Optics vol.35, pp.3, 2015, https://doi.org/10.14697/jkase.2015.35.3.0465
  2. 학업성취도 평가 결과에 기초한 2009 개정 교육과정 시기의 중학생의 과학 학업성취 특성 분석 vol.40, pp.4, 2015, https://doi.org/10.14697/jkase.2020.40.4.429