Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Cases of Discrepancy in High School Students' Achievement in Science Education Assessment: Focusing on Testing Tool in Affective Area

과학 교육 평가에서 나타나는 고등학생들의 성취 불일치 사례 - 정의적 영역 검사 도구를 중심으로 -

  • Received : 2017.07.31
  • Accepted : 2017.09.07
  • Published : 2017.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study analyzed some of the discrepancies in quantitative and qualitative data focusing on cognitive and affective achievement in science education. Academic and affective achievement score of 308 high school students were collected as quantitative data, and 33 students were interviewed for qualitative data. We examined the causes and types of discrepancies in terms of testing tools. As a result from quantitative data, there were a large number of students with a big difference between subjects in cognitive achievement, and constructs in affective achievement. More than 20% of the students did not match tendency between achievements in two areas. Through interviews, some examples such as intentional control of science learning for future study and careers, different responses by differences in perception between school science and science, appeared. A comparison of quantitative data by testing tool between qualitative ones and interviews showed conflicting result, where most students evaluated themselves differently from their own quantitative data. That is due to the students' interaction with the testing tools. Two types of discrepancy related to testing tool are found. One is 'the concept difference between the item developer and students,' the other is 'the difference between students' exposed response and their real mindset.' These are related to the ambiguity of the terms used in the tool and response bias due to various causes. Based on this study, an effort is required to elaborate the testing item that matches students' actual perception and to apply students' science learning experience to testing items.

이 연구는 과학 교육 현장에서 인지적 정의적 평가를 중심으로 한 양적 자료와 질적 자료에서 나타나는 몇 가지 불일치 사례를 분석했다. 308명의 고등학교 2학년 학생을 대상으로 학업 성취도와 정의적 성취도를 양적 자료로 수집했고, 그 중 33명의 학생을 면담한 질적자료를 분석했다. 주로 검사 도구의 측면에서 불일치 사례의 원인과 유형을 고찰했다. 연구 결과 양적 자료인 인지적 성취와 정의적 성취 영역 각각에서 과목별, 구인별 차이가 크게 나타나는 학생들이 상당수 있었고, 특히 두 영역 간 성취도 경향이 일치하지 않는 학생들도 20% 이상 분석되었다. 선택한 진로와 진학을 위해 의도적으로 과학 학습을 조절한 사례, 학교 과학과 과학에 대한 인식 차이에 따라 다른 반응 등의 사례가 면담을 통해 발견되었다. 도구로 측정한 양적 자료와 학생들의 면담 내용인 질적 자료를 비교한 결과 스스로 반응한 양적 자료와 다르게 자신을 평가하는 학생들이 대부분이었다. 이는 다양한 특성을 지닌 학생들이 검사 도구와 상호작용하는 과정에서 비롯된다. 검사 도구와 관련된 불일치 유형은 '문항 개발자가 의도한 개념과 학생들이 이해하는 개념 간 차이'와 '표현된 반응과 속마음 간의 차이'로 나타났다. 검사 도구에서 사용한 용어가 학생들에게 모호하게 인식될 때 자의적이거나 일관성 없이 반응하는 경우가 전자에, 사회적 바람직성이나 자아 방어 기제에 의한 반응 왜곡은 후자에 해당한다. 이상 연구 결과를 바탕으로 자기보고식 검사 도구가 학생들의 실제 인식을 잘 반영하고 있는지 검토하고 정교화하려는 노력, 학습 경험을 획일적으로 고정시키는 평가 개선 등이 필요하다.

Keywords

References

  1. Abd-El-Khalick, F., Summers, R., Said, Z., Wang, S., & Culbertson, M. (2015). Development and large-scale validation of an instrument to assess Arabic-speaking students' attitudes toward science. International Journal of Science Education, 37(16), 2637-2663. https://doi.org/10.1080/09500693.2015.1098789
  2. Aikenhead, G. (2001). Students' ease in crossing cultural borders into school science. Science Education, 85, 180-188. https://doi.org/10.1002/1098-237X(200103)85:2<180::AID-SCE50>3.0.CO;2-1
  3. Aikenhead, G., & Jegede, O. J. (1999). Cross-cultural science education: A cognitive explanation of a cultural phenomenon. Journal of Research in Science Teaching, 36(3), 269-289. https://doi.org/10.1002/(SICI)1098-2736(199903)36:3<269::AID-TEA3>3.0.CO;2-T
  4. Au, Y. (2007). A search on social desirability according to administered mode and demonstrable condition of a psychology testing. Journal of Educational Evaluation, 20(4), 235-258.
  5. Bae, B., Lee, D., & Ham, K. (2015). Validation of the Korean short-version of social desirability scale(SDS-9) using the Rasch model. Korean Journal of Counseling, 16(6), 177-197. https://doi.org/10.15703/kjc.16.4.201508.177
  6. Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage.
  7. Chi, E. (2011). Applying the Rasch model to explore the differences between countries for tests administered across countries. Journal of Educational Evaluation, 24(1), 89-106.
  8. Cho, Y. (2003). A study on I-consciousness-we-consciousness-relationships between I-consciousness-we-consciousness and individuality relatedness, psychosocial maturity, and interpersonal problem. The Korean Journal of Counseling and Psychotherapy, 15(1), 91-109.
  9. Cho, J., Kim, S., Kim, M., Ok, H., Lim, H., & Son, S. (2012). Ways of improving Korean students' affective characteristic based on PISA and TIMSS results. Seoul: Korea Institute for Curriculum and Evaluation.
  10. Choe, S., Ku, J., Kim, J., Park, S., Oh, E., Kim, J., & Baek, H. (2013). Strategies for improving the affective characteristics of Korean students based on the results of PISA and TIMSS. Seoul: Korea Institute for Curriculum and Evaluation.
  11. Choi, S. (2004). Social psychology of Korean people. The Korean Psychological Association, 2, 151-162.
  12. Chung, S., & Shin, D. (2016). Trends of assessment research in science education. Journal of the Korean Association for Science Education, 36(4), 563-579. https://doi.org/10.14697/jkase.2016.36.4.0563
  13. Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory(3rd ed.). Thousand Oaks, CA: Sage.
  14. Costa, V. (1995). When science is "another world": Relationships between worlds of family, friends, school, and science. Science Education, 79(3), 313-333. https://doi.org/10.1002/sce.3730790306
  15. Cronbach, L. (1946). Response sets and test validity. Educational and Psychological Measurement, 6(3), 475-494. https://doi.org/10.1177/001316444600600405
  16. Crowne, D., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. https://doi.org/10.1037/h0047358
  17. Dudley, N., McFarland, L., Goodman, S., Hunt, S., & Sydell, E. (2005). Racial differences in socially desirable responding in selection contexts: Magnitude and consequences. Journal of Personality Assessment, 85(1), 50-64. https://doi.org/10.1207/s15327752jpa8501_05
  18. Ferrando, P., & Chico, E. (2001). Detecting dissimulation in personality test scores: A comparison between person-fit indices and detection scales. Educational and Psychological Measurement, 61, 997-1012. https://doi.org/10.1177/00131640121971617
  19. Ferrari, J., Bristow, & Cowman, S. (2005). Looking good or being good? The role of social desirability tendencies in student perception of institutional mission and values. College Student Journal, 39(1), 7-13.
  20. Fives, H., Huebner, W., Birnbaum, A., & Nicolich, M. (2014). Developing a measure of scientific literacy for middle school students. Science Education, 98(4), 549-580. https://doi.org/10.1002/sce.21115
  21. Ganster, D., Hennessey, H., & Luthans, F. (1983). Social desirability response effects: Three alternative models. Academy of Management Journal, 26(2), 321-331. https://doi.org/10.2307/255979
  22. Heine, S., Lehman, D., Peng, K., & Greenholtz, J. (2002). What's wrong with cross-cultural comparisons of subjective Likert scales?: The reference-group effect. Journal of Personality and Social Psychology, 82(6), 903-918. https://doi.org/10.1037/0022-3514.82.6.903
  23. Jűrges, H., Schneider, K., & Bűchel, F. (2005). The effect of central exit examinations on student achievement: Quasi-experimental evidence from TIMSS Germany. Journal of the European Economic Association, 3(5), 1134-1155. https://doi.org/10.1162/1542476054729400
  24. Keillor, B., Owens, D., & Pettijohn, C. (2001). A cross-cultural/cross-national study of influencing factors and socially desirable response biases. International Journal of Market Research, 43(1), 63-84.
  25. Ku, J., Kim, S., Lee, H., Cho, S., & Park, H. (2016a). OECD Programme for International Student Assessment: An analysis of PISA 2015 results. Seoul: Korea Institute for Curriculum and Evaluation.
  26. Ku, J., Kim, S., Lee, H., Cho, S., & Park, H. (2016b). OECD Programme for International Student Assessment: Establishing a foundation of PISA 2018 field trial. Seoul: Korea Institute for Curriculum and Evaluation.
  27. Kim, S., Kim, K., & Park, J. (2014). The effect of mathematics achievement on changes in mathematics interest and values for middle school students. Journal of Research in Curriculum Instruction, 18(3), 683-701. https://doi.org/10.24231/rici.2014.18.3.683
  28. Kim, K., Kim, S., Kim, M., Kim, S., Kang, M., Park, H., & Jung, S. (2009). Comparative analysis of curriculum and achievement characteristics between Korea and high performing countries in PISA & TIMSS. Seoul: Korea Institute for Curriculum and Evaluation.
  29. Kim, S. (2013). Measurement of social norms: An experimental study of response bias. Korean Political Studies, 22(2), 153-178.
  30. Kim, S., Seo, H. (2011). Self-regulated learning ability related to science inquiry skill and affective domain of science in middle school students. Journal of Science Education, 35(2), 307-323. https://doi.org/10.21796/jse.2011.35.2.307
  31. Kim, M., & Cho, J. (2013). Analysis of the properties of affective achievement in science based on TIMSS and science teachers' perception. Journal of the Korean Association for Science Education, 33(1), 46-62. https://doi.org/10.14697/jkase.2013.33.1.046
  32. Kim, S., Park, J., Kim, H., Jin, E., Lee, M., Kim, J., Ahn, Y., & Seo, J. (2012). Findings from TIMSS for Korea: TIMSS 2011 international results. Seoul: Korea Institute for Curriculum and Evaluation.
  33. Kim, Y. (2010). Development of a social desirability scale(SDS-24). Journal of Korean Social Welfare Administration, 12(3), 1-39.
  34. Kind, P., Jones, K., & Barmby, P. (2007). Developing attitudes towards science measures. International Journal of Science Education, 29(7), 871-893. https://doi.org/10.1080/09500690600909091
  35. Koballa, T. R. (1988). Attitude and related concepts in science education. Science Education, 72(2), 115-126. https://doi.org/10.1002/sce.3730720202
  36. Krynowsky, B. (1988). Problems in assessing student attitude in science education: A partial solution. Science Education, 72(4), 575-584. https://doi.org/10.1002/sce.3730720504
  37. Kwak, Y. (2017). Exploration of features of Korean eighth grade students' attitudes toward science. Journal of the Korean Association for Science Education, 37(1), 135-142. https://doi.org/10.14697/jkase.2017.37.1.0135
  38. Lee, J. (2016). Analysis of changes in the learning environments of middle school science classes. Journal of the Korean Association for Science Education, 36(5), 717-727. https://doi.org/10.14697/jkase.2016.36.5.0717
  39. Lee, M., & Kim, K. (2004). Relationship between attitudes toward science and science achievement. Journal of the Korean Association for Science Education, 24(2), 399-407.
  40. Lee, M., Sohn, W., & No, U. (2007). The results from PISA 2006. Seoul: Korea Institute for Curriculum and Evaluation.
  41. Lee, M., Park, S., Sohn, W., & Nam, M. (2007). Technical report for PISA 2006 main study. Seoul: Korea Institute for Curriculum and Evaluation.
  42. Lee, M., Choi, J., Lee, J., & Shin, M. (2016). A preliminary study of defensive response style on a self-report personality assessment. The Journal of the Korean Association of Psychotherapy, 8(2), 61-80.
  43. MaCrae, R., & Costa, P. (1983). Social desirability scales: More substance than style. Journal of Consulting and Clinical Psychology, 51, 882-888. https://doi.org/10.1037/0022-006X.51.6.882
  44. Martin, M., Mullis, I., & Foy, P. (2012). TIMSS 2011 international science report. MA: Boston College.
  45. Messick, S. (1991). Psychology and methodology of response styles. In R. E. Snow & D. E. Willey(Eds.). Improving inquires in social science. Hillsdale, NJ: Erlbaum.
  46. Ministry of Education Science, and Technology (2011). Science curriculum. Notification No. 2011-361 of MOEST. Seoul: MOEST.
  47. Mullis, I., Martin, M., & Foy, P. (2012). TIMSS 2011 international mathematics report. MA: Boston College.
  48. Murayama, K., Zhou, M., & Nesbit, J. (2009). A cross-cultural examination of the psychometric properties of response to the achievement goal questionnaire. Educational and Psychological Measurement, 69(2), 266-286. https://doi.org/10.1177/0013164408322017
  49. Myeong, J., & Crawley, F. E. (1993). Predicting and understanding Korean high school students' science track choice: Testing the theory of reasoned action by structural equation modeling. Journal of Research in Science Teaching, 30(4), 381-400. https://doi.org/10.1002/tea.3660300406
  50. Ones, D., Viswesvara, C., & Reiss, A. (1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660-679. https://doi.org/10.1037/0021-9010.81.6.660
  51. Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049-1079. https://doi.org/10.1080/0950069032000032199
  52. Osborne, J., Simon, S., & Tytler, R. (2009). Attitudes towards science: An update. Paper presented at the annual meeting of the American Educational Research Association, San Diego, California.
  53. Park, C. (2007). The trend in the Korean middle school student's affective variables toward mathematics and its effect on their mathematics achievement. The Mathematical Education, 46(1), 19-31.
  54. Park, C. Jeong, E., Kim, K., Han, K., Jun, H., & Lee, S. (2004). Teachers, instruction, and achievement based on TIMSS 1999. Seoul: Korea Institute for Curriculum and Evaluation.
  55. Park, H., (2008). Test of group invariance for the structural model among motivation, self-concept and student achievement: Using PISA 2006 data. Journal of Educational Evaluation, 21(3), 43-67. https://doi.org/10.2167/eri413.0
  56. Paulhus, D. (1984). Two component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 589-609.
  57. Paulhus, D. (1998). Interpersonal and intrapsychic adaptiveness of trait self-enhancement: A mixed blessing? Journal of Personality and Social Psychology, 74, 1197-1208. https://doi.org/10.1037/0022-3514.74.5.1197
  58. Reise, S., & Flannery, W. (1996). Assessing person-fit on measures of typical performance. Applied Measurement in Education, 9(1), 9-26. https://doi.org/10.1207/s15324818ame0901_3
  59. Saldana, J. (2009). The coding manual for qualitative researchers. Thousand Oaks, London: Sage.
  60. Sang, K., Kwak, Y., Park, J., & Park, S. (2016). The Trends in International Mathematics and Science Study (TIMSS): Findings from TIMSS 2015 for Korea. Seoul: Korea Institute for Curriculum and Evaluation.
  61. Schluf, Boaz, Hattie, J., & Dixon, R. (2008). Factors affecting responses to Likert type questionnaires: Introduction of the ImpExp, a new comprehensive model. Social Psychology of Education, 11(1), 59-78. https://doi.org/10.1007/s11218-007-9035-x
  62. Schunk, D., & Pajares, F. (2009). Self efficacy theory. In Wentzel, K., & Wigfield, A. (Eds). Handbook of motivation at school. New York: Routledge.
  63. Seo, J., Choi, J., & Kim, Y. (2007). Comparison of life learning skills of gifted science students and normal students in high school. Biology Education, 35(1), 61-72. https://doi.org/10.15717/bioedu.2007.35.1.61
  64. Shen, C., & Pedulla, J. (2000). The relationship between students' achievement and their self-perception of competence and rigour of mathematics and science: A cross-national analysis. Assessment in Education, 7(2), 237-253. https://doi.org/10.1177/107319110000700304
  65. Shin, H., & Sohn, W. (2014). Applying a mixed Rasch model to investigate response styles in TIMSS 2011 math enjoyment scale. Journal of Educational Evaluation, 27(2), 429-448.
  66. Sohn, W. (2017). Individual difference and consistency in response scale use. Korean Journal of Educational Research, 55(1), 23-43.
  67. Son, E., Cha, J., & Kim, A. (2007). Test of construct equivalence of personality inventory in low and high socially desirable responding groups. Korean Journal of Social and Personality Psychology, 21(2), 71-87.
  68. Song, H. (2010). Development of a self-reported executive function rating scale for the Korean high school students: A preliminary study. The Korean Journal of Clinical Psychology, 29(1), 109-124. https://doi.org/10.15842/kjcp.2010.29.1.007
  69. Stephen, A. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60(3), 340-360. https://doi.org/10.1177/00131640021970583
  70. Stober, J. (2001). The social desirability scale-17(SDS-17): Convergent validity, discriminant validity, and relationship with age. European Journal of Psychological Assessment, 17(3), 222-232. https://doi.org/10.1027//1015-5759.17.3.222
  71. Strauss, A. L. (1987). Qualitative analysis for social scientists. Cambridge: Cambridge University Press.
  72. Suzuki, L., & Ponterotto, J. (2007). Handbook of multicultural assessment: Clinical, psychological, and educational applications. San Francisco, CA: John Wiley & Sons, Inc.
  73. Vailant, G. (1992). Ego mechanism of defense. Washington: American Psychiatric Press.

Cited by

  1. 이공계 고등학생들의 과학 학습 경험의 의미 탐색: 인지적·정의적 성취 유형에 따른 공통점과 차이점을 중심으로 vol.22, pp.3, 2018, https://doi.org/10.24231/rici.2018.22.3.163