Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

International Comparison of Cognitive Attributes using Analysis on Science Results at TIMSS 2011 Based on the Cognitive Diagnostic Theory

인지진단이론에 근거한 TIMSS 2011의 과학 결과 분석을 통한 인지 속성의 국제비교

  • Received : 2015.01.24
  • Accepted : 2015.04.13
  • Published : 2015.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

This research purports to find out the characteristics of Korean students cognitive attributes and compare it with that of high-achieving countries who took TIMSS 2011 based on the Cognitive Diagnostic Theory. Based on TIMSS 2011 Science framework, nine cognitive attributes were extracted and the researcher analyzed that 216 of the TIMSS 2011 science items require these attributes. This analysis was conducted to come up with a Q-matrix. After producing the Q-matrix, multi-level IRT was used to figure out each countries' characteristics for each of the cognitive attribute. According to the study results, four attributes, such as 'Use Models,' 'Interpret Information,' 'Draw Conclusions,' and 'Evaluate and justify' were easier attributes for Korean middle school students. However, the other five attributes such as 'Recall/Recognize', 'Explain', 'Classify', 'Integrate', 'Hypothesize and Design' were considered as harder attributes compared to other countries. Korean students also considered 'Interpret Information' as the easiest attributes, and 'Explain' as the hardest attributes of all. For Korean students, those attributes considered to be easy were the easiest and hard attributes as the hardest compared to other countries, showing very extreme cases. Therefore, to give students more meaningful learning experience, it is better to use all the attributes altogether rather than use specific attributes while constructing Science curriculum or textbooks.

본 연구의 목적은 우리나라 학생들의 인지적 속성의 특징을 인지진단이론에 근거하여 국제적인 수준에서 비교 분석하고 우리나라 교육에 주는 시사점을 찾는 것이다. TIMSS 2011의 평가틀에 근거하여 9개의 인지 속성을 추출하였고 TIMSS 2011의 과학 문항 216개에 대해 각 문항이 어떠한 인지 속성들을 요구하는지 판단하여 Q행렬을 작성하였다. 총 5차례의 검토와 수정을 반복하여 타당도를 점검하고 최종적인 Q행렬을 작성한 후, 다층 IRT 분석을 실시하여 인지 속성에 따른 국가별 특성을 파악하였다. 분석을 통해 나타난 우리나라의 인지적 속성을 TIMSS 2011의 과학성취도 상위 15개국과 비교한 결과 우리나라 학생들에게 모형사용하기, 자료분석하기, 결론도출하기, 평가 및 정당화하기는 쉬운 속성에 해당하였고 회상/인식하기, 설명하기, 분류하기, 통합하기, 가설설정 및 실험설계하기는 어려운 속성에 해당하였다. 우리나라 학생들이 가장 쉬워하는 인지 속성은 자료해석하기였고 가장 어려워하는 인지 속성은 설명하기였다. 우리나라의 경우 쉬운 인지 속성의 대부분은 비교국 중에서 가장 쉽게 여기고 어려운 인지 속성은 가장 어렵게 여기는 것으로 나타나 극단적인 특성을 보였다. 따라서 특정한 인지 속성이 많이 활용되도록 하기 보다는 여러가지 인지 속성이 골고루 활용되도록 과학 교육과정과 과학 교과서를 구성할 필요가 있겠다.

Keywords

References

  1. Bang, D., Park, E., Yoon, H., Kim, J., Lee, Y., Park, J., Song, J., Dong, H., Shim, B., Lim, H., & Lee, H. (2013). The design of curricular framework for integrated science education based on big idea. Journal of Korean Association for Science Education, 33(5), 1041-1054. https://doi.org/10.14697/jkase.2013.33.5.1041
  2. Cho, J., Kim S., Lee S., Kim, M., Ok, H., Rim, H., Park, Y., Lee, M., Han, H., & Son, S. (2011). The Trends in International Mathematics and Science Study (TIMSS 2011) : A Technical Report of the Main Survey in Korea (KICE Research Report RRE 2011-4-1). Seoul: KICE.
  3. DiBello, L., Stout, W., & Rousses, L. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In P. D. Nichols, S. F. Chipman, and R. L. Brennan (Eds.), Cognitively Diagnostic Assessment. Hillsdale, NJ: Lawrence Erlbaum Associates.
  4. Fu, J., & Li, Y. (2007). Cognitively diagnostic psychometric models: An integrative review. In annual meeting of the National Council on Measurement in Education, Chicago, IL.
  5. Hartz, S. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.
  6. Hennig, C. (2004). Cluster-wise assessment of cluster stability. Unpublished research report, Department of Statistical Science, University of College London.
  7. Jeon, K. (2003). An Analysis of Chemistry Achievement in the Third International Mathematics and Science Study-Repeat(TIMSS-R). Journal of Korean Association for Science Education, 23(3), 299-307.
  8. Jeong, E. (2005). An Analysis of Korean Middle School Students' Achievement of Life Science in TIMSS 2003. Journal of Korean Biology Science Society, 33(3), 277-290.
  9. Kim, H., & Park, D. (2009). Elementary Science Textbook Analysis of Korea and the United States. Journal of Science Education, 33(2), 258-270.
  10. Kim, H., & Yeau, S. (2004). An Analysis on Scientific Inquiry Process in the Middle School 8th Grade Science Textbook published by 7th National Curriculum and Students' Current Abilities of Scientific Inquiry Skill. Journal of Korean Biology Science Society, 32(4), 390-397.
  11. Kim, J., & Kim S. (2009). Analysis of Test Result at Secondary Science Using Cognitive Diagnosis theory. Journal of Korean Association for Science Education, 29(8), 812-823.
  12. Kim, K. M., Park, Y. S., Choe, S. U. (2008). Analysis of Scientific Inquiry Activities in the Astronomy Section of School Science Textbooks. Journal of Korean Earth Science Society, 29(2), 204-217. https://doi.org/10.5467/JKESS.2008.29.2.204
  13. Kim, K., Kim, S., Kim M., Kim S., Kang, M., Park, H., & Jung, S. (2009). Comparative analysis of curriculum and achievement characteristics between Korea and high performing countries in PISA & TIMSS (KICE Research Report RRE 2009-7-2). Seoul: KICE
  14. Kim, M., & Kim, K. (2010). A Comparative Study of Science Textbooks in Korea, Singapore, and Taiwan - Focus on the field of Biology -. Journal of Korean Association for Science Education, 30(4), 498-518.
  15. Kim, M., Kim, K. (2011). A Content Analysis of Biology Domain of Korean and Singaporean Textbooks Based on the TIMSS Framework. Journal of Korean Biology Science Society, 39(2), 217-234.
  16. Kim, S., & Seong, M. (2011). Diagnosis of knowledge states using large scale assessments - An application of DINA model -. The Journal of Curriculum and Evaluation, 14(1), 177-200.
  17. Kim, S., Kim S., Seong, M. (2008). Using Cognitive Diagnosis Theory to Analyze the Test Results of Mathematics. School Mathematics, 10(2), 259-277.
  18. Kim, T., Lee, J., Shin, K., Park, J., Kim, D., & Lee, S. (2007). A Comparative Study on Physics Inquiry Activities of Science Textbooks for Secondary School in Korea and Singapore. Journal of Korean Association for Science Education, 27(7), 547-558.
  19. Kwak, Y., & Jeong, E. Y. (2007). An analysis of Earth Science Items and Achievement in TIMSS 2003. Journal of Korean Earth Science Society, 28(4), 405-414. https://doi.org/10.5467/JKESS.2007.28.4.405
  20. Lee, B. (2006). Analysis of Inquiry Standards in Foreign National Science Curricula. Journal of Korean Association for Science Education, 25(7), 873-883.
  21. Martin, M. O., Mullis, I. V. S., & Foy, P. (2012). TIMSS 2011 International Results in Science. MA: Boston College.
  22. Ministry of Education (1997). Science Curriculum 1997-15(9). Seoul: Daehan Textbook Publishing.
  23. Ministry of Education and Human Resources Development (2007). Science Curriculum 2007-79(9). Seoul: Ministry of Education and Human Resources Development.
  24. Ministry of education, science and technology (2011). Science Curriculum 2011-361(9). Seoul: Ministry of education, science and technology.
  25. Mullis, I. V., Martin, M. O., Ruddock, G. J., O'Sullivan, C. Y., & Preuschoff, C. (2009). TIMSS 2011 Assessment Framework. MA: Boston College.
  26. Park, C. (2008). A Multilevel IRT Model for Group-Level Diagnostic Assessment with Application to TIMSS. Doctoral dissertation, University of Wisconsin-Madison.
  27. Park, C. (2013). A Comparison of the Methods of Estimating Intraclass Correlation Coefficients for Hierarchically Structured Dichotomous- Item Data. Journal of Educational Evaluation, 26(2), 459-476.
  28. Park, H., & Cho, H. (2003). Analyses of Scientific Inquiry in Science 8. Journal of Korean Association for Science Education, 23(3), 239-245.
  29. Song, M., Lee, Y., & Park, Y. (2011). Ananlysis and score reporting based on cognitive diagnostic models using the National Assessment of Educational Achievement. (KICE Research Report RRE-2011-8). Seoul: KICE.
  30. Suh, Y. (2007). A Comparative Study on Elementary Science Textbooks in Korea and the U.S. : Focusing on 3rd Grade Scientific Concepts and Inquiry Process in 'Matter' Units. Journal of Elementary Science Education, 26(5), 509-524.
  31. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345-354. https://doi.org/10.1111/j.1745-3984.1983.tb00212.x
  32. Tatsuoka, K. K. (1990). Toward integration of item response theory and cognitive error diagnoses. In N. Frederiksen, R. L. Glasser, A. M. Lesgold, and M. G. Shafto (Eds.), Diagnostic monitoring of skills and knowledge acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates.
  33. Tatsuoka, K. K. (1995). Architecture of knowledge structure and cognitive diagnosis: A statistical pattern recognition and classification approach. In P. D. Nichols, S. F. Chipman, and R. L. Brennan (Eds.), Cognitively Diagnostic Assessment. Hillsdale, NJ: Lawrence.
  34. Yang, I., Kim, S., & Cho, H. (2007). Analysis of the Types of Laboratory Instruction in Elementary and Secondary Schools Science. Journal of Korean Association for Science Education, 27(3), 235-241.
  35. Yeau, S., Kim, H., & Kim, M. (2003). An Analysis on Scientific Inquiry Process in the Elementary 5th Grade Science Textbook Published by 7th National Curriculum and Students' Current Abilities of Scientific Inquiry Skill. Journal of Korean Biology Science Society, 31(3), 214-223.
  36. You, M., & Cho, H. (2003). Analysis of Scientific Inquiry in Science VII. Journal of Korean Association for Science Education, 23(5), 494-504.
  37. Yu, S, A. (2011). An analysis of Singaporean primary-school science textbooks as a preliminary study for the development of grade-cluster textbooks in the 2009 revised Korean National Curriculum system. The Journal of Curriculum Studies, 29(3), 147-171. https://doi.org/10.15708/kscs.29.3.201109.007

Cited by

  1. 과학 교육에서의 평가 연구 동향 vol.36, pp.4, 2015, https://doi.org/10.14697/jkase.2016.36.4.0563
  2. TIMSS 2015 중학교 2학년 지구과학 영역에 대한 우리나라 학생들의 성취 특성 및 교육과정 연계성 탐색 vol.37, pp.1, 2015, https://doi.org/10.14697/jkase.2017.37.1.0009
  3. 우리나라 중학교 2학년 학생들의 과학에 대한 정의적 태도 특성 탐색 vol.37, pp.1, 2017, https://doi.org/10.14697/jkase.2017.37.1.0135
  4. 교육과정 변화에 따른 우리나라 초등학교 4학년 학생들의 TIMSS 과학 내용영역별 성취 특성 분석 vol.37, pp.4, 2015, https://doi.org/10.14697/jkase.2017.37.4.599
  5. 우리나라 중학생들의 TIMSS 2015 생명과학 영역 성취 특성 분석 vol.45, pp.3, 2015, https://doi.org/10.15717/bioedu.2017.45.3.390
  6. An International Comparative Study of Students' Scientific Explanation Based on Cognitive Diagnostic Assessment vol.12, pp.None, 2015, https://doi.org/10.3389/fpsyg.2021.795497
  7. Characterizing the Academic Achievement of Middle School Students in Science Subject based on Latent Profile Analysis vol.24, pp.2, 2015, https://doi.org/10.29221/jce.2021.24.2.177