Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Developing and Applying the Questionnaire to Measure Science Core Competencies Based on the 2015 Revised National Science Curriculum

2015 개정 과학과 교육과정에 기초한 과학과 핵심역량 조사 문항의 개발 및 적용

  • Received : 2018.05.21
  • Accepted : 2018.08.18
  • Published : 2018.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study was conducted to develop items to measure scientific core competency based on statements of scientific core competencies presented in the 2015 revised national science curriculum and to identify the validity and reliability of the newly developed items. Based on the explanations of scientific reasoning, scientific inquiry ability, scientific problem-solving ability, scientific communication ability, participation/lifelong learning in science presented in the 2015 revised national science curriculum, 25 items were developed by five science education experts. To explore the validity and reliability of the developed items, data were collected from 11,348 students in elementary, middle, and high schools nationwide. The content validity, substantive validity, the internal structure validity, and generalization validity proposed by Messick (1995) were examined by various statistical tests. The results of the MNSQ analysis showed that there were no nonconformity in the 25 items. The confirmatory factor analysis using the structural equation modeling revealed that the five-factor model was a suitable model. The differential item functioning analyses by gender and school level revealed that the nonconformity DIF value was found in only two out of 175 cases. The results of the multivariate analysis of variance by gender and school level showed significant differences of test scores between schools and genders, and the interaction effect was also significant. The assessment items of science core competency based on the 2015 revised national science curriculum are valid from a psychometric point of view and can be used in the science education field.

이 연구는 2015 과학과 교육과정에 제시된 과학과 핵심역량에 관한 서술문을 기초로 학생들의 과학과 핵심역량을 측정할 수 있는 문항을 개발하고, 그 타당도와 신뢰도를 확인하기 위하여 진행되었다. 2015 과학과 교육과정에 제시된 과학적 사고력, 과학적 탐구 능력, 과학적 문제해결력, 과학적 의사소통 능력, 과학적 참여와 평생 학습 능력에 대한 설명을 근거로 각 역량별로 5개 문항씩, 총 25개 문항이 개발되었다. 개발된 문항은 전국 17개 시도에 고루 분포한 학생 참여형 과학수업 선도학교의 초등학교, 중학교, 고등학교 학생 11,348명을 대상으로 과학과 핵심역량을 조사하여 문항의 타당도와 신뢰도를 분석하였다. 타당도는 Messick(1995)이 제안한 6가지 타당도 중 내용 타당도, 실제 타당도, 내적 구조 타당도, 일반화 타당도 등 4가지를 확인하였다. 라쉬 모델의 문항 적합도(MNSQ)를 분석한 결과, 25개 문항 중 부적합 문항은 없었으며, 구조방정식을 활용한 확인적 요인 분석 결과 5개 요인 모델은 적합한 모델로 확인되었다. 성별과 학교급별에 따른 차별적 문항 반응을 분석한 결과, 부적합 값은 175개 중 2개에서 나타났다. 성별과 학교급별 다변량 이원분산분석을 실시한 결과, 학교급 간 및 성별 간에서 통계적으로 유의미한 차이가 나타났으며, 교호작용 효과 역시 유의미하였다. 결론적으로 2015 과학과 교육과정에 제시된 과학과 핵심역량의 설명에 기초하여 개발한 과학과 핵심역량 평가문항은 정신측정학적 관점에서 타당하며, 따라서 교육 현장에서 그 활용 가치가 있을 것으로 판단된다.

Keywords

References

  1. Angner, E. (2006). Economists as experts: Overconfidence in theory and practice. Journal of Economic Methodology, 13(1), 1-24. https://doi.org/10.1080/13501780600566271
  2. Baars, M., Vink, S., van Gog, T., de Bruin, A., & Paas, F. (2014). Effects of training self-assessment and using assessment standards on retrospective and prospective monitoring of problem solving. Learning and Instruction, 33, 92-107. https://doi.org/10.1016/j.learninstruc.2014.04.004
  3. Baek, P. (2013). A study on the relationships and characteristics between korean collegiate essential skills and undergraduates’ individual variant factors. Korean Journal of General Education, 7(3), 349-387.
  4. Benbow, C. P., & Stanley, J. C. (1983). Sex differences in mathematical reasoning ability: More facts. Science, 222(4627), 1029-1031 https://doi.org/10.1126/science.6648516
  5. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. New York, NY: Springer.
  6. Cavallo, A. M., Potter, W. H., & Rozman, M. (2004). Gender differences in learning constructs, shifts in learning constructs, and their relationship to course achievement in a structured inquiry, yearlong college physics course for life science majors. School Science and Mathematics, 104(6), 288-300. https://doi.org/10.1111/j.1949-8594.2004.tb18000.x
  7. Chang, G., Seong, E., Choi, H., Jin, S., & Kim, G. (2015). Measuring Youth Competency Index and International Comparative Study II (IEA ICCS 2016): General Report. Sejong: National Youth Policy Institute.
  8. Chi, M. T. (2006). Two approaches to the study of experts' characteristics. In K.A. Ericson, N. Charness, P.J. Feltovich & R.R. Hoffman (Eds.) The Cambridge Handbook of Expertise and Expert Performance (pp.21-30). New York, NY: Cambridge University Press.
  9. Choi, K., Shin, D., & Lee, H. Y. (2008). Facts on gender difference in science education and solutional strategies. Women’s Studies Review, 25, 117-158. https://doi.org/10.18341/wsr.2008.25.2.117
  10. Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2011). Investigating the substantive aspect of construct validity for the satisfaction with life scale adapted for children: A focus on cognitive processes. Social Indicators Research, 100(1), 37-60. https://doi.org/10.1007/s11205-010-9603-x
  11. George, D., & Mallery, M. (2003). Using SPSS for Windows Step by Step: a Simple Guide and Reference. Boston, MA: Allyn & Bacon
  12. Jin, M., Sohn, Y., & Chu, H. (2011). Study on development plan of K-CESA for college education assessment. The Korean Journal of Educational Administration, 29(4), 461-486.
  13. Kang, Y., Jung, D., Park, S., Lim, H., Hwang, J., & Seol, G. (2014). 2014 KEDI Student Competency Index Survey. KEDI Report RR 2014-26
  14. Kim, C., Eom, M., Kim, K., Kim, C., Park, J., Park, H., Lee, K., & Lee, S. (2013). A Study for Developing Student Competency Index. KEDI Report RR 2013-23
  15. Kim, D., Oh, H., Song, Y., Koh, E., Park, S., & Jung, E. (2009). Exploring students’ core competencies in higher education from professors’ perspectives: A case study of Seoul National University(SNU). Asian Journal of Education, 10(2), 195-214.
  16. Kim, E, (2014). Readiness of university students in Korea for the core competencies in the 21st century. Master Thesis. Ewha Womans University.
  17. Kim, H. & Lee, S. (2012). Diagnosis and direction of the competence-based educational model: Based on the course and effects. Korean Journal of General Education, 6(4), 11-40.
  18. Kim, H. & Lee, S. (2013). Application plans of competence assessment: Focusing on the connection between assessment and curriculum. Korean Journal of General Education, 7(4), 139-172.
  19. Kim, K., Kim, J., Chang, G., So, K., Kim, J., & Kang, Y. (2008). A Study on Development and Progress Plan of Lifetime Core Competency for Adolescent I : General Report. Seoul: National Youth Policy Institute.
  20. Kim, K., Maeng, Y., Chang, G., Ku, J., Kang, Y., & Cho, M. (2009). A Study on Development and Progress Plan of Lifetime Core Competency for Adolescent II : General Report. Seoul: National Youth Policy Institute.
  21. Kostons, D., Van Gog, T., & Paas, F. (2012). Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning. Learning and Instruction, 22(2), 121-132. https://doi.org/10.1016/j.learninstruc.2011.08.004
  22. Kwak, Y. (2013). Ways of restructuring key competences for a revision of science curriculum. Journal of Korean Earth Science Society. 34(4), 368-377. https://doi.org/10.5467/JKESS.2013.34.4.368
  23. Lai, J. S., Teresi, J., & Gershon, R. (2005). Procedures for the analysis of differential item functioning (DIF) for small sample sizes. Evaluation & the health professions, 28(3), 283-294. https://doi.org/10.1177/0163278705278276
  24. Lee, G., Jeon, J., Huh, G., & Hong, W. (2009). A Study on the Vision of Elementary and Secondary School Curriculum for the Improvement of Core Competencies of Future Koreans: General Report RRC 2009-10. Seoul: Korea Institute for Curriculum and Evaluation.
  25. Lee, G., Kim, G., Kim, S., Kim, H., Lee, M., Lee, S., & Lee, I (2013). Exploring Curriculum for the Development of Core Competencies in the Future: Focusing on the Linkage of Curriculum, Teaching. Learning and Educational Evaluation. Research Report RRC 2013002. Seoul: Korea Institute for Curriculum and Evaluation.
  26. Lee, G., Kwon, H., Lee, J., & Lee, S. (2010). The study on proper way for engineering university education: Based on the perception of current competencies and expected competencies of engineering freshmen. Journal of Engineering Education Research, 13(6), 57-71. https://doi.org/10.18108/jeer.2010.13.6.57
  27. Lee, G., Min, Y., Jeon, J., & Kim, M. (2008). A Study on the Vision of Elementary and Secondary School Curriculum for the Improvement of Core Competencies of Future Koreans (II): Focusing on Sub-Element Setting by Core Competency Area. Seoul: Korea Institute for Curriculum and Evaluation.
  28. Lee, J. K., Shin, S., Rachmatullah, A., & Ha, M. (2017). Exploring the patterns of engineering college students' engineering-related creativity by gender, academic year, and engineering education accreditation program through latent class analysis. Journal of Science Education, 41(1), 16-35. https://doi.org/10.21796/jse.2017.41.1.16
  29. Lee, S. & Lee, S. (2012). A new direction of undergraduate general education: Through a case study of competence-based education. Korean Journal of General Education, 6(2), 11-42.
  30. Lee, V. E., & Burkam, D. T. (1996). Gender differences in middle grade science achievement: Subject domain, ability level, and course emphasis. Science Education, 80(6), 613-650. https://doi.org/10.1002/(SICI)1098-237X(199611)80:6<613::AID-SCE1>3.0.CO;2-M
  31. Menkhoff, L., Schmeling, M., & Schmidt, U. (2013). Overconfidence, experience, and professionalism: An experimental study. Journal of Economic Behavior & Organization, 86, 92-101. https://doi.org/10.1016/j.jebo.2012.12.022
  32. Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5-8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x
  33. MOE (Ministry of Education) (2015). 2015 revised Science National Curriculum. Ministry of Education.
  34. Neumann, I., Neumann, K., & Nehm, R. (2011). Evaluating instrument quality in science education: Rasch-based analyses of a nature of science test. International Journal of Science Education, 33(10), 1373-1405. https://doi.org/10.1080/09500693.2010.511297
  35. Noh, Y. & Lee, S. (2013a). The competency model development for Korean 4 year university students: A case study of D Women’s University. Journal of Corporate Education and Talent Research, 15(1), 79-101.
  36. Noh, Y. & Lee, S. (2013b). A study on developing and validating the instrument for diagnosing the competency of university students: A case study of D Women's University. The Korean Journal of Human Resource Development, 15(3), 273-305. https://doi.org/10.18211/kjhrdq.2013.15.3.011
  37. OECD (2003). Defining and selection of competencies: Theoretical and conceptual foundation. OECD Press.
  38. OECD (2006). Schooling for tomorrow: Thinking scenarios, rethinking education. OECD Press.
  39. Pajares, F. & Graham, L. (1999). Self-efficacy, motivation constructs, and mathematics performance of entering middle school students. Contemporary Educational Psychology, 24(2), 124-139. https://doi.org/10.1006/ceps.1998.0991
  40. Pallier, G. (2003). Gender differences in the self-assessment of accuracy on cognitive tasks. Sex Roles, 48(5), 265-276. https://doi.org/10.1023/A:1022877405718
  41. Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806-813. https://doi.org/10.1016/j.lindif.2012.04.007
  42. Park, B. T., Shin, D. (2010). Elementary school high achievers’ gender characteristics of successful and unsuccessful process in problem solving. Journal of Learner-Centered Curriculum and Instruction, 10(2), 173-189.
  43. Quirk, P. J. (2010). The trouble with experts. Critical Review, 22(4), 449-465. https://doi.org/10.1080/08913811.2010.541699
  44. Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., Gundy, C., Koller, M., Petersen, M. A., & Sprangers, M. A. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Ppidemiology, 62(3), 288-295. https://doi.org/10.1016/j.jclinepi.2008.06.003
  45. Seong, E., Baek, H., & Jin, S. (2014). Measuring Youth Competency Index and International Comparative Study I (IEA ICCS 2016): General Report. Sejong: National Youth Policy Institute.
  46. Shin, S., Lee, J. K., & Ha, M. (2017). Influence of career motivation on science learning in Korean high-school students. EURASIA Journal of Mathematics Science and Technology Education, 13(5), 1517-1538.
  47. So, K., Lee, S., Lee, J., & Heo., H. (2010). Review on curriculum reform in the New Zealand; Implementation of key competencies-based curriculum. Comparative Education Society, 20(2), 27-50.
  48. Son, Y., Song, C. & Lee, H. (2016). A Study on the Support for Core Competency of Korean Students in Collegiate Level. Sejong: Korea Research Institute for Vocational Education and Training.
  49. Stankov, L. & Lee, J. (2008). Confidence and cognitive test performance. Journal of Educational Psychology, 100(4), 961. https://doi.org/10.1037/a0012546
  50. Wright, B.D. & Linacre, J.M. (1994). Reasonable mean-square-fit values. Rasch Measurement Transactions, 8(3), 370.
  51. Yoon, H., Kim, Y., Lee, G., & Jeon, J. (2007). A Study on the Vision of Elementary and Secondary School Curriculum for the Improvement of Core Competencies of Future Koreans (I): Focusing on Core Competency Criteria and Area Setting. Research Report RRC 2007-1. Seoul: Korea Institute for Curriculum and Evaluation.

Cited by

  1. 문항 반응 분석을 활용한 초등학생과 중학생의 시스템 사고 검사 도구 타당도 검증 vol.67, pp.2, 2018, https://doi.org/10.25152/ser.2019.67.2.249
  2. 2015 개정 교육과정 과학과 교과역량에 대한 초등교사의 인식과 적용 실태 분석 vol.39, pp.2, 2018, https://doi.org/10.15267/keses.2020.39.2.219
  3. 2015 개정 교육과정 '과학탐구실험' 평가 도구 및 평가 현황 탐색 vol.40, pp.5, 2020, https://doi.org/10.14697/jkase.2020.40.5.515
  4. 역량중심 초등과학 영재교육 프로그램 지구과학 영역 분석 vol.14, pp.2, 2021, https://doi.org/10.15523/jksese.2021.14.2.136