DOI QR코드

DOI QR Code

Analysis of weights depending on scoring domains of the mathematical creativity test

수학적 창의성 검사의 채점 영역별 가중치 분석

  • Received : 2015.11.11
  • Accepted : 2015.12.30
  • Published : 2016.05.31

Abstract

This study analyzes the mathematical creativity test as an illustrative example with scoring domains of fluency, flexibility and originality in order to make suggestions for obtaining maximum reliability based on a composite score depending on combinations of each scoring domain weights. This is done by performing a multivariate generalizability analysis on the test scores, which were allowed to access publicly, of 30 mathematically gifted elementary school students, and therefore error variances, generalizability coefficients, and effective weights have been calculated. The main results were as follows. First, the optimal weights should adjust to .5, .4, and .1 based on the maximum generalizability coefficient even though the original weights in the mathematical creativity test were equal for each scoring domain with fluency, flexibility and originality. Second, the mathematical creativity test using the three scoring domains of fluency, flexibility, and originality showed higher reliability than using one scoring domain such as fluency. These results are limited to the mathematical creativity test used in this study. However, the methodology applied in this study can help determine the optimal weights depending on each scoring domain when the tests constructed in various researchers or educational fields are composed of multiple scoring domains.

Keywords

References

  1. 강애남, 이규민 (2006). 학생들의 동료평가를 활용한 수행평가 결과의 일반화가능도 분석. 교육평가연구 19(3), 107-121. (Kang, A. N., & Lee, G. M. (2006). A generalizabiliy theory approach to investigating the generazliability of performance assessment using student peer reviews. Journal of Educational Evaluation 19(3), 107-121.)
  2. 교육부 (1992). 제6차 교육과정의 개요. 교육부 고시 제1992-11호. (Ministry of Education. (1992). The 6th curriculum. Notification No. 1992-11 of the Ministry of Education.)
  3. 교육부 (1997). 제7차 교육과정. 교육부 고시 제1997-15호. (Ministry of Education. (1997). The 7th curriculum for elementary and middle schools. Notification No. 1997-15 of the Ministry of Education.)
  4. 교육부 (2014). 2015 문.이과 통합형 교육과정 총론 주요 사항. 세종: 교육부. (Ministry of Education. (2014). The main particular of 2015 National Curriculum draft for an integration of Arts and Sciences. Sejong: Ministry of Education.)
  5. 교육부 (2015). 제2차 수학교육 종합계획. 세종: 교육부 (Ministry of Education. (2015). The 2nd curriculum for mathematical education. Sejong: Ministry of Education.)
  6. 교육과학기술부 (2009). 2009개정 교육과정 총론. 교육과학기술부 고시 제2009-41호. (Ministry of Education, Science, and Technology. (2009). The 2009 curriculum revision. Notification No. 2009-41 of the Ministry of Education, Science, and Technology.)
  7. 교육인적자원부 (2007). 2007년 개정 교육과정 총론. 교육인적자원부 고시 제 2007-79호. (Ministry of Education & Human Resources Development (2007). The 2007 curriculum revision. Notification No. 2007-79 of the Ministry of Education & Human Resources Development.)
  8. 권오남, 김정효, 윤희선, 박재희, 태혜경, 윤태숙 (2000). 창의적 문제해결력 중심의 수학 교육과정 적용 및 효과분석. 수학교육 논문집 5, 87-116.(Kwon, O. N., Kim, J. H., Youn, H. S., Park, J. H., Tae, H. K., & Youn, T. S. (2000). Application and examination the effect of mathematics curriculum to enhance creative problem solving abilities. The Mathematical Education 39(2), 81-99.)
  9. 김경선, 이규민, & 강승혜 (2010). 일반화가능도 이론을 적용한 한국어 말하기 성취도 평가의 신뢰도와 오차요인 분석. 한국어 교육 21(4), 51-75.(Kim, K. S., Lee, G. M., & Kang, S. H. (2010). Analysis of error sources and estimation of reliability in a Korean speaking achievement test by applying generalizability theory. Journal of Korean Language Education 21(4), 51-75.)
  10. 김도연, 허종관 (2002). 일반화가능도이론을 적용한 주관적 배구기능검사의 신뢰도 추정. 한국체육측정평가학회지 4(2), 15-28. (Kim, D. Y., & Heo, J. K. (2002). Generalizability coefficient estimation of subjectively volleyball test. The Korean Journal of Measurement and Evaluation in Physical Education and Sport Science 4(2), 15-28.)
  11. 김명화 (2005). 채점 자동화 시스템 구축을 위한 수학 국성형 문항 채점의 일반화가능도 연구. 교육문제연구 22, 205-222.(Kim, M. H. (2005). An application of the generalizability to constructed response items in mathematics for computer automated scoring. Journal of Research in Education, 22, 205-222.)
  12. 김명희(2000). 수행평가의 타당도 검증을 위한 측정학적 접근. 박사학위논문, 이화여자대학교.(Kim, M. H. (2000). A psychometric approaches of the validity verifications in performance assessments. (Unpublished doctoral dissertation, Ewha Woman University, Seoul, Korea).)
  13. 김보라, 이규민 (2012). 일반화가능도 이론을 적용한 초등학교 쓰기 수행평가의 총체적 채점과 분석적 채점 방식 비교. 교육학연구 50(4), 49-76.(Kim, B. R., & Lee, G. M. (2012). A comparison of holistic and analytic scoring methods for elementary school writing assessment by applying generalizability theory. Korean Journal of Educational Research 50(4), 49-76.)
  14. 김부윤, 이지성 (2005). 수학적 창의성의 평가방안에 대한 모색. 한국학교수학회논문집 8(3), 327-341. (Kim, B. Y. & Lee, J. S. (2005). A note on the assesment of mathematical creativity. Journal of the Korean School Mathematics Society 8(3), 327-341.)
  15. 김부윤, 이지성 (2006). 수학에서의 창의적 태도의 측정도구 개발과 그 적용. 수학교육 45(1), 25-34. (Kim, B. Y. & Lee, J. S. (2006). Development and its applications of the CAS-K in Mathematics. The Mathematical Education 45(1), 25-34.)
  16. 김성숙 (2011). 학문 목적 한국어 쓰기 능력에 대한 분석적 평가의 일반화가능도 검증. 한국어교육 22(3), 29-48. (Kim, S. S. (2011). Analysis of error sources and estimation of reliability in an analytic evaluation on writing ability for academic purpose Korean by applying generalizability theory. Journal of Korean Language Education 22(3), 29-48.)
  17. 김성숙, 김양분 (2001). 일반화가능도 이론. 서울: 교육과학사. (Kim, S. S. & Kim, Y. B. (2001). Generalizability theory. Seoul: Kyoyookgwahaksa.)
  18. 김성연 (2014a). 미국 수학교사의 교수 질 평가도구 분석을 통한 우리나라 수학 교원능력개발평가에서의 일반화가능도 이론 활용성 탐색. 수학교육 논문집 28(4), 431-455. (Kim, S. Y. (2014a). Exploring the application of generalizability theory to mathematics teacher evaluation for professional development instructional quality assessment of mathematics teachers in the U.S. Communications of Mathematical Education 28(4), 431-455.)
  19. 김성연 (2014b). 미국의 수업관찰평가 분석을 통한 우리나라 교원능력개발평가에서의 다변량 일반화가능도 이론 활용성 탐색. 한국교육 41(1), 5-29. (Kim, S. Y. (2014b). Exploring the application of maultivariate generalizability theory to teacher evaluation for professional development in Korea based on the analysis of classroom obsevations in the U.S. The Journal of Korean Education 41(1), 5-29.)
  20. 김성찬, 김성연, 한기순 (2012). 관찰.추천에 의한 수학영재 선발 시 사용되는 교사추천서와 자기소개서 평가에 대한 일반화가능도 이론의 활용. 수학교육 논문집 26(3), 251-271. (Kim, S. C., Kim, S. Y., & Han, K. S. (2012). An application of generalizability theory to self-introduction letter and teacher's recommendation letter used in identification of mathematically gifted students by observations and nomination. Communications of Mathematical Education 26(3), 251-271.)
  21. 김성연, 한기순 (2013). 관찰.추천제에 의한 수학영재 선발 시 사용되는 교사추천서와 자기소개서 평가에 대한 다변량 일반화가능도 이론의 활용. 영재교육연구 23(5), 671-698. (Kim, S. Y., & Han, K. S. (2013). An application of multivariate generalizability theory to teacher recommendation letters and self-introduction letters used in selection of mathematically gifted students by observation and nomination. Journal of Gifted/Talented Education 23(5), 671-698.) https://doi.org/10.9722/JGTE.2013.23.5.671
  22. 김성연, 한기순 (2014). 수학영재 선발에서 교사추천서와 자기소개서 채점내용 가중치에 따른 신뢰도 분석. 영재와 영재교육 13(1), 43-65. (Kim, S. Y., & Han, K. S. (2014). Analysis of reliability coefficients depending on different domain weights in scoring teacher recommendation letters and self-introduction letters used in selection of mathematically gifted students. The Journal of the Korean Society for the Gifted and Talented 13(1), 43-65.)
  23. 김양분 (1989). 일반화가능도이론에 의한 과학적 사고기능검사의 신뢰도 추정에 관한 연구. 교육평가연구 3(1), 183-209. (Kim, Y. B. (1989). A study of reliability estimation of a scientific critical thinking tool based on generalizability theory. Journal of Educational Evaluation 3(1), 183-209.)
  24. 김용대 (2003). 문제해결을 통한 수학적 일반성의 발견. 수학교육 논문집 15, 153-159. (Kim, Y. D. (2003). The discovery of mathematical generalization through problem solving. Communications of Mathematical Education 15, 153-159.)
  25. 김용범 (2009). 수학영재판별을 위한 창의적 수학능력 검사 문항 개발. 석사학위논문, 인천대학교.(Kim, Y. B. (2009). The development of questions to measure the creativity mathematical ability. (Unpublished master's thesis, University of Incheon, Incheon, Korea).)
  26. 김정은 (2010). 초등 수학 수업에서 수학 영재 판별 도구 개발에 관한 연구. 석사학위논문, 광주교육대학교.(Kim, J. E. (2010). A Study on the developing identifying instrument for math gifted children in elementary mathematics classes. (Unpublished master's thesis, Gwangju National University of Education, Gwangju, Korea).)
  27. 김홍원, 김명숙, 방승진, 황동주 (1997). 수학영재 판별도구 개발연구(II). 서울: 한국교육개발원. (Kim, H. W., Kim, M. S., & Bang S. H., & Hwang, D. J. (1997). A study on the development of the identification tools for mathematically gifted students(II). Seoul: Korean Educational Development Institute.)
  28. 김홍원, 김명숙, 송상헌 (1996). 수학영재 판별도구 개발 연구(I). 서울: 한국교육개발원. (Kim, H. W., Kim, M. S., & Song, S. H. (1996). A study on the development of the identification tools for mathematically gifted students(I). Seoul: Korean Educational Development Institute.)
  29. 남승인 (2007). 수학 창의성 신장을 위한 평가 문항 개발 방안. 수학교육 논문집 21(2), 271-282. (Nam, S. I. (2007). A study on the development evaluation item to extend mathematical creativity. Communications of Mathematical Education, 21(2), 271-282.)
  30. 대전광역시교육청 (2011). 창의 인성 교육을 위한 평가방법 개선. 대전: 대전광역시교육청. (Daejeon Metropolitan Office of Education. (2010). The improvement of evaluation methods for creativity and personality education. Daejeon: Dejeon Metropolitan Office of Education.)
  31. 도종훈 (2007). 학교수학에서 추측과 문제제기 중심의 수학적 탐구 활동 설계하기. 수학교육 46(1), 69-79. (Do, J. H. (2007). Designing Mathematical Activities Centered on Conjecture and Problem Posing in School Mathematics. The Mathematical Education 46(1), 69-79.)
  32. 문교부 (1987). 제 5차 중학교 교육과정. 문교부 고시 제 1987-7호. (Ministry of Education(1987). The 5th curriculum for middle schools. Notification No. 1987-7 of the Ministry of Education.)
  33. 방승진, 최중오 (2010). 수학 학문적 창의성 신장을 위한 멘토십 프로그램 모형 개발. 영재교육연구 20(1), 205-229. (Bang, S. J. & Choi, J. O. (2010). Implementations of Mentorship Program Model for the Academic Creativities of Mathematics. Journal of Gifted/Talented Education 20(1), 205-229.)
  34. 변은진, 전평국 (2001). 개방형 문제를 활용한 평가가 수학적 창의력에 미치는 효과. 수학교육논문집 11, 259-277. (Byeon, E. J. & Jeon, P. K. (2011). The effects of evaluation based on open-ended problems on mathematical creativity. Communications of Mathematical Education 21(2), 271-282.)
  35. 서울특별시교육청 (2010). 창의성 계발을 위한 평가 개선 기본계획. 서울: 서울시교육청.(Seoul Metropolitan Office of Education. (2010). The master plans of improving evaluation for creativity development. Seoul: Seoul Metropolitan Office of Education.)
  36. 성태제 (2010). 현대교육평가 (3판). 서울: 학지사. (Seong, T. J. (2010). Modern educational evaluation (3rd edition) . Seoul: Hakjisa.)
  37. 손은영 (2005). 탐구학습이 초등학교 6학년 학생들의 수학적 창의력에 미치는 효과. 석사학위논문, 한국교원대학교.(Son, E. Y. (2005). The effects of inquiry learning on the mathematical creativity of 6th students. (Unpublished master's thesis, Korea National University of Education, Chung-Buk, Korea).)
  38. 송상헌 (1998). 수학 영재성 측정과 판별에 관한 연구. 박사학위논문, 서울대학교.(Song, S. H. (1998). A study on the measurement and discrimination of the mathematical giftedness. (Unpublished master's thesis, Seoul National University of Education, Seoul, Korea).)
  39. 신동일 (2001). 일반화가능도 이론 적용을 중심으로 한 말하기평가도구 타당도 검증 연구. 응용언어학 17(1), 199-221. (Shin, D. I. (2001). Validation process of an EFL speaking test on G-theory and other analytical techniques. The Applied Linguistics Association of Korea 17(1), 199-221.)
  40. 신승윤 (2013). 초등수학영재의 수학 창의성 문제해결력과 메타인지와의 관계. 석사학위논문, 대구교육대학교. (Shin, S. Y. (2013). The relationship between mathematically gifted elementary students' math creative problem-solving ability and metacognition. (Unpublished master's thesis, Daegu National University of Education, Daegu, Korea).)
  41. 안애정 (1997). 일반화가능도 이론에 의한 체조심판 판정 오차원 분석. 박사학위논문, 명지대학교. (Ahn, A. J. (1997). Application of generalizability in analyzing sources of error variance of gymnastic judgement. (Unpublished doctoral dissertation, Myongji University, Seoul: Korea).)
  42. 양승민 (2012). 수학 창의성 발달을 위한 수학적 창의성과 수학적 사고력의 상관관계 분석. 석사학위논문, 고려대학교. (Yang, S. M. (2012). Analysis of correlation between mathematical creativity and mathematical thinking for the development of mathematical creativity. (Unpublished master's thesis, Korea University, Seoul, Korea).)
  43. 유경선 (2012). 공간능력과 수학적 창의성 신장을 위한 도형학습 과제의 효과. 석사학위논문, 서울교육대학교. (Yoo, K. S. (2012). Effects of figure learning tasks for improvement of spatial ability and mathematical creativity: focusing on a planar figure of a solid figure. (Unpublished master's thesis, Seoul National University of Education, Seoul, Korea).)
  44. 유윤재 (2002). 수학적 창의성의 검사. 한국수학교육학회 학술발표논문집 1, 1-22. (Yoo, Y. J. (2002). A mathematical creativity test. Studies in Mathematical Education, 1, 1-22.)
  45. 유윤재 (2003). 창의적 수학문제해결력 검사도구의 요소. 수학교육 논문집 17, 159-168. (Yoo, Y. J(2003). Components of a mathematical creative problem solving ability test. Communications of Mathematical Education, 17, 159-168.)
  46. 유윤재 (2007). 수학 영재 교육. 서울: 교우사. (You, Y. J. (2007). Mathematically gifted education. Seoul: Kyowoosa.)
  47. 윤기옥 (2008). 중학교 1학년 수학 교과서에 나타난 창의성 측정 문항의 분석. 석사학위논문, 단국대학교. (Yoon, K. O. (2008). Analysis for creativity factors of problems in 7th-grade mathematics textbooks. (Unpublished master's thesis, Dankook University, Seoul, Korea).)
  48. 윤지윤 (2014). 초등수학영재와 일반학생의 수학불안과 수학적 창의성의 관계. 석사학위논문, 대구교육대학교. (Yoon, J. Y. (2014). The relations between mathematics anxiety and mathematical creativity of elementary mathematically gifted and non-gifted students. (Unpublished master's thesis, Daegu National University of Education, Daegu, Korea).)
  49. 이강섭 (2010). 수학 창의성 평가에서 독창성의 점수화 방법. 수학교육 49(1), 111-118. (Lee, K. S. (2010). A scoring system for the originality in evaluation of mathematical creativity. The Mathematical Education 49(1), 111-118.)
  50. 이강섭, 심상길 (2005). 창의성 증진을 위한 수학 활동 프로그램 평가 방법의 소개. 수학교육 논문집 19(1), 101-110. (Lee, K. S., & Sim, S. K. (2005). An introduction of evaluation methods for mathematical activity program to develop creativity. Communications of Mathematical Education 19(1), 101-110.)
  51. 이강섭, 황동주, 서종진 (2003a). 개방형 문항에 대한 중학교 영재학생과 일반학생의 반응 연구. 수학교육논문집 17, 181-190. (Lee, K. S., Hwang, D. J., & Seo, J. J. (2003a). A study of responses between gifted middle school students and regular school students about open ended problems. Communications of Mathematical Education 17, 181-190.)
  52. 이강섭, 황동주, 서종진 (2003b). 일반 창의성(도형)과 수학 창의성과의 관련 연구: TTCT Figural A와 MCPSAT A를 바탕으로. 수학교육 42(1), 1-10. (Lee, K. S., Hwang, D. J., & Seo, J. J. (2003b). A study on the relationship between general creativity and mathematical creativity: based on TTCT Figural A and MCPSAT A. The Mathematical Education 42(1), 1-10.)
  53. 이경화, 고진영, 박숙희 (2007). 영재의 지능, 창의성, 자아개념간의 관계 분석. 영재와 영재교육 6(1), 147-162. (Lee, K. H., Koh, J. Y., & Park, S. H. (2007). Gender differences in creative thinking and creative personality among primary school students. The Journal of the Korean Society for the Gifted and Talented, 6(1), 147-162.)
  54. 이규민(2003). 단위검사 개념의 적용-일반화가능도 이론을 중심으로. 교육평가연구 16(1), 53-70. (Lee, G. M. (2003). An application of the testlet concept-A generalizability theory approach.. Journal of Educational Evaluation 16(1), 53-70.)
  55. 이규민, 황경현 (2007). 초등학교 과학과 수행평가의 총체적 채점과 분석적 채점 방식에 대한 일반화가능도 분석. 아동교육 16(4), 169-184. (Lee, G. M., & Hwang, K. H. (2007). A generalizability theory approach toward investigating the generalizability scores form holistic and analytic scoring methods in performance assessments of an elementary school science class. The Journal of Child Education 16(4), 169-184.)
  56. 이기영 (2004). 평가 유형과 채점 방식에 따른 중.고등학교 과학 수행 평가의 일반화가능도에 관한 연구. 박사학위논문, 서울대학교. (Lee, K. Y. (2004). A study on generalizability of science performance assessment according to assessment type and scoring method in middle & high school. (Unpublished doctoral dissertation, Seoul National University, Seoul, Korea).)
  57. 이기영, 김찬종 (2005). 한국 지구과학 올림피아드 문항 분석을 통한 문항의 질 향상 방안. 한국지구과학회지 26(6), 511-523. (Lee, K. Y., & Kim, C. J. (2005). Analysis of Korea earth science olympiad items for the enhancement of item quality. Journal of the Korean Earth Science Society 26(6), 511-523.)
  58. 이기영, 안희수 (2005). 중등학교 과학 수행평가의 평가 유형과 채점 방식 및 신뢰도 분석. 한국과학교육학회지 25(2), 173-183. (Lee, K. Y., & Ahn, H. S. (2005). Analyses of asseement types, scoring methods and reliability of science performance assessment in middle and high school. Journal of Korean Association for Science Education 25(2), 173-183.)
  59. 이대현 (2014). 다양한 해결법이 있는 문제를 활용한 수학적 창의성 측정 방안 탐색. 학교수학 16(1), 1-17. (Lee, D. H. (2014). A study on the measurement in mathematical creativity using multiple solution tasks. School Mathematics 16(1), 1-17.)
  60. 이동희, 김판수 (2010). 수학적 창의성과 태도 및 작업에 미치는 등산학습법의 적용과 효과. 한국초등수학교육학회지 14(1), 23-41. (Lee, D. H. & Kim, P. S. (2010) The Effect of Climbing Learning Method on Mathematical Creativity and Attitude toward Mathematical Creativity. Journal of Elementary Mathematics Education in Korea 14(1), 23-41.)
  61. 이영식, 신상근 (2004). 다변량 일반화가능도 이론에 의한 말하기 시험의 타당도와 신뢰도에 관한 연구. 외국어교육 11(2), 249-265. (Lee, Y. S., & Shin, S. K. (2004). An investigation into the dependability of ratings in a German speaking test using the multivariate generalizability theory. Foreign Languages Education 11(2), 249-265.)
  62. 이옥수 (2007). 자연계열 통합교과형 논술 문항 분석. 석사학위논문, 서울대학교. (Lee, O. S. (2007). An item analysis of scientific essay for science and engineering courses. (Unpublished master's thesis, Seoul National University, Seoul, Korea).)
  63. 이종희, 김기연 (2007). 창의적 생산력 신장의 교육목표 이해를 위한 수학영재의 수학적 창의성 개념 탐색. 수학교육 46(4), 445-464. (Lee, J. H., & Kim, K. Y. (2007). A study on the concept of mathematical creativity in the mathematically gifted aspect. The Mathematical Education 46(4), 445-464.)
  64. 이지연 (2005). 국가대표볼링선수 선발방법의 신뢰도 추정: 일반화가능도 이론의 적용. 박사학위논문, 한국체육대학교. (Lee, J. Y. (2005). Estimating the reliability of the selection method of the national team bowlers: application of the generalizability theory. (Unpublished doctor's dissertation, Korea National Sport University, Seoul, Korea).)
  65. 이향 (2012). 말하기 수행 평가에서 발음 범주 채점의 최적화 방안 연구-일반화가능도 이론을 활용하여. 한국어 교육 23(2), 301-329. (Lee, H. (2012). A study on the optimal conditions in rating of pronunciation scale for speaking performance assessment-using generalizability theory. Journal of Korean Language Education 23(2), 301-329.)
  66. 이현숙 (2012). 혼합형 검사의 문항 유형별 가중치에 따른 신뢰도 및 다변량 일반화가능도분석. 교육평가연구 25(1), 95-116. (Lee, H. S. (201). Multivariate generalizability analyses for mixed-format tests with various compositions of MC and CR item weights. Journal of Educational Evaluation 21(1), 95-116.)
  67. 정영한 (2009). 일반화가능도 이론을 적용한 태권도 심판판정의 신뢰도 분석. 박사학위논문, 국민대학교. (Jung, Y. H. (2009). Analysis on reliability of referees judgment in Taekwondo competition applying generalizability theory. (Unpublished master's thesis, Kookmin University, Seoul, Korea).)
  68. 정유화 (2011). 보드게임 활동이 수학적 창의성과 수학적 태도에 미치는 영향. 석사학위논문, 서울교육대학교. (Jung. (2011). The effects of boardgame activity on mathematical creativity and attitude towards mathematics. (Unpublished master's thesis, Seoul National University of Education, Seoul, Korea).)
  69. 조석희, 황동주 (2007). 중학교 수학 영재 판별을 위한 수학 창의적 문제해결력 검사 개발. 영재교육연구 17(1), 1-26. (Cho, S. H., & Hwang, D. J. (2007). Math creative problem solving ability test for identification of the mathematically gifted middle school students. Journal of Gifted/Talented Education 17(1), 1-26.)
  70. 조재윤 (2009). 일반화가능도 이론을 이용한 쓰기 평가의 오차원 분석 및 신뢰도 추정 연구. 국어교육 128, 325-357. (Cho, J. Y. (2009). A sutdy on the analysis of error source in writing assessment and reliability estimation using generalizability theory. The Education of Korean Language 128, 325-357.)
  71. 조정환 (1991). 일반화가능도이론에 의한 피하지방검사, 농구기능검사의 신뢰도 추정. 박사학위논문, 한국체육대학교. (Cho, J. H. (1991). Generalizability coefficient estimation of skinfold thickness test and basketball skill test. (Unpublished doctoral dissertation, Korean National College of Physical Education, Seoul, Korea).)
  72. 최연희 (2001). GENOVA and FACETS analysis of an English writing test: tasks, raters, and scales. 영어교육 56(2), 125-142. (Choi, Y. H. (2001). GENOVA and FACETS analysis of an English writing test: tasks, raters, and scales. English Teaching 56(2), 125-142.)
  73. 최은선 (2010). 초등수학 영재교육을 위한 수학적 창의성 개념연구. 석사학위논문, 서울교육대학교. (Choi, E. S. (2010). A study on the concept of mathematical creativity for the education of mathematically gifted children. (Unpublished master's thesis, Seoul National University of Education, Seoul, Korea).)
  74. 하수현, 이광호 (2014). Leikin 의 수학적 창의성 측정 방법에 대한 고찰. 한국초등수학교육학회지 18(1), 83-103. (Ha, S. H. & Lee, K. (2014). A study about the Leikin's method of measuring mathematical creativity. Journal of Elementary Mathematics Education in Korea 18(1), 83-103.)
  75. 한혜승 (1996). 제6차 교육과정에 따른 중학교 수학교과서 분석: 1학년을 중심으로. 석사학위논문, 이화여자대학교. (Han, H. S. (1996). Analysis of math textbooks according to the 6th curriculum in middle school: Focus on 1st-grade textbooks. (Unpublished master's thesis, Ewha Woman University, Seoul, Korea).)
  76. 한혜정.박순경.이근호.이승미 (2012). 시.도 교육청 수준 교육과정 지침 실태 분석 및 개선 방안. 서울: 한국교육과정평가원. (Han, H. J., Park, S. K., Lee, K. H., & Lee, S. M. (2012). A study on the improvement of the MPOE curriculum organization & implementation guideline through analyzing its current situation. Seoul: Korea Institute for Curriculum and Evaluation.)
  77. 홍주연, 한인기 (2014). 수학 영역에서 창의적 산출물 의미 척도. 수학교육 53(2), 291-312. (Hong, J. Y. & Han, I. K. (2014). A study on creative products semantic scale in mathematics. The Mathematical Education 53(2), 291-312.)
  78. 황우형, 이유나 (2009). 중등 영재학생과 일반학생의 완벽주의 성향과 수학교과에 대한 정의적 특성과의 관계. 수학교육 논문집 23(1), 1-38. (Hwang, W. H. & Lee, Y. N. (2009). The relationship between mathematically gifted students and regular students in perfectionism and the affective traits. Communication of Mathematical Education 23(1), 1-38.)
  79. 황혜정, 김흥원, 박경미, 김수환, 김신영, 채선희 (1997). 창의력 신장을 돕는 중학교 수학과 학습 평가 방법 연구. 서울: 한국교육개발원. (Hwang, H. J. & Kim, H. W., Park, K.M., Kim, S. H., Kim, S. Y., & Chae, S. H. (1999). A study on the evaluation methods in the middle school mathematics to improve the mathematical creativeness. Seoul: Korea Educational Development Institute.)
  80. American College Testing (1989). Preliminary technical manual for the Enhanced ACT Assessment. Iowa City, IA: American College Testing.
  81. Balka, D. S. (1974). The development of an instrument to measure creative ability in mathematics. (Unpublished doctoral dissertation, University of Missouri-Columbia).
  82. Bauer, G. R. (1971). A study of the effects of a creative classroom, creative problems, and mathematics education on the creative ability of prospective elementary teachers (Unpublished doctoral dissertation, Stanford University).
  83. Becker, J. P., & Shimada, S. (1997). The Open-Ended Approach: A New Proposal for Teaching Mathematics. VA: National Council of Teachers of Mathematics.
  84. Brandau, L. I., & Dossey, J. A. (1979). Processes involved in mathematical divergent problem-solving. San Francisco: American Educational Research Association.
  85. Brennan, R. L. (2001a). Generalizability theory. New York: Springer.
  86. Brennan, R. L. (2001b). Manual for mGENOVA version 2.1. Iowa City, IA: Center for advanced studies in measurement and assessment, The University of Iowa.
  87. Brown, J. D. (2013). Score dependability and decision consistency. In A. J. Kunnan (Ed.), The companion to language assessment. Oxford, UK: Wiley-Blackwell.
  88. Brown, G. T. L., Glasswell, K., & Harland, D. (2004). Accuracy in the scoring of writing: Studies of reliability and validity using a New Zealand writing assessment system. Assessing Writing, 9, 105-121. https://doi.org/10.1016/j.asw.2004.07.001
  89. Burjan, V. (1991). Mathematical giftedness-Some questions to be answered. In Education of the Gifted in Europe: Theoretical and Research Issues: report of the educational research workshop held in Nijmegen (The Netherlands) (pp. 165-170).
  90. Colton, D. A. (1993). A Multivariate generalizability analysis of the 1989 and 1990 AAP mathematics test forms with respect to the table of specifications. ACT research report series.
  91. Cronbach, L. J., Gleser, G. C., & Nanda, H. Rajaratnam. N.(1972). The dependability of behavioral measurements: theory of generalizability for scores and profiles. New York: Wiley.
  92. Cronbach, L. J., Linn, R. L., Brennan, R. L., & Haertel, E. H. (1997). Generalizability analysis for performance assessments of student achievement or school effectiveness. Educational and Psychological Measurement 57(3), 373-399. https://doi.org/10.1177/0013164497057003001
  93. Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991). Quality control in the development and use of performance assessments. Applied measurement in education 4(4), 289-303. https://doi.org/10.1207/s15324818ame0404_3
  94. Ervynck, G. (1991). Mathematical creativity. Advanced mathematical thinking (pp. 42-53). New York: Springer.
  95. Evans, E. W. (1964). Measuring the ability of students to respond in creative mathematical situations at the late elementary and early junior high school level. (Unpublished doctoral dissertation, University of Michigan).
  96. Foster, J. (1970). An Exploratory attempt to assess creative ability in mathematics. Primary Mathematics 8(1), 2-8.
  97. Gao, X., Shavelson, R. J., & Baxter, G. P. (1994). Generalizability of large-scale performance assessments in science: Promises and problems. Applied Measurement in Education 7(4), 323-342. https://doi.org/10.1207/s15324818ame0704_4
  98. Godbout, P., & Schutz, R. W. (1983). Generalizability of ratings of motor performances with reference to various observational designs. Research Quarterly for Exercise and Sport 54(1), 20-27. https://doi.org/10.1080/02701367.1983.10605267
  99. Guilford, J. P. (1967). The nature of human intelligence. New York: McGraw-Hill.
  100. Haylock, D. W. (1987). A framework for assessing mathematical creativity in school children. Educational Studies in Mathematics 18(1), 59-74. https://doi.org/10.1007/BF00367914
  101. Haylock, D. W. (1997). Recognizing mathematical creativity in schoolchildren. Zentralblatt fur Didaktik der Mathematik 27(2), 68-74.
  102. Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction 26(4), 430-511. https://doi.org/10.1080/07370000802177235
  103. Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: teacher observation systems and a case for the generalizability study. Educational Researcher 41(2), 56-64. https://doi.org/10.3102/0013189X12437203
  104. Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers' mathematics knowledge for teaching. The Elementary School Journal 105(1), 11-30. https://doi.org/10.1086/428763
  105. Ho, A. D., & Kane, T. J. (2013). The reliability of classroom observations by school personnel. research paper. MET Project. Bill & Melinda Gates Foundation.
  106. Hoeflinger, M. (1998). Developing mathematically promising students. Roeper Review 20(4), 244-247. https://doi.org/10.1080/02783199809553900
  107. Hollands, R. (1972). Educational technology: aims and objectives in teaching mathematics. Mathematics in School 1(6), 22-23.
  108. Imai, T. (2010). The influence of overcoming fixation in mathematics towards divergent thinking in open-ended mathematics problems on Japanese junior high school students. International Journal of Mathematical Education in Science and technology 31(2), 187-193. https://doi.org/10.1080/002073900287246
  109. Jarjoura, D., & Brennan, R. L. (1982). A variance components model for measurement procedures associated with a table of specifications. Applied Psychological Measurement 6(2), 161-171. https://doi.org/10.1177/014662168200600202
  110. Jarjoura, D., & Brennan, R. L. (1983). Multivariate generalizability models for tests developed from tables of specifications. New Directions for Testing & Measurement.
  111. Jensen, L. R. (1973). The relationships among mathematical creativity, numerical aptitude and mathematical achievement. (Unpublished doctoral dissertation, University of Texas at Austin).
  112. Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teachers: combining high-quality observations with student surveys and achievement gains. MET Project. Bill & Melinda Gates Foundation.
  113. Kim, H., Cho, S., & Ahn, D. (2004). Development of mathematical creative problem solving ability test for identification of the gifted in math. Gifted Education International 18(2), 164-174. https://doi.org/10.1177/026142940301800206
  114. Krutetskii, V. A. (1976). The psychology of mathematical abilities in school children. University of Chicago Press.
  115. Lee, G., Brennan, R. L., & Frisbie, D. A. (2000). Incorporating the Testlet Concept in Test Score Analyses. Educational Measurement: Issues and Practice 19(4), 9-15. https://doi.org/10.1111/j.1745-3992.2000.tb00041.x
  116. Lee, G., & Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets. Applied Measurement in Education 12(3), 237-255. https://doi.org/10.1207/S15324818AME1203_2
  117. Lee, K. S., Hwang, D. J., & Seo, J. J. (2003). A development of the test for mathematical creative problem solving ability. Research in Mathematical Education, 7(3), 163-189.
  118. Leikin, R. (2009). Exploring mathematical creativity using multiple solution tasks. Creativity in Mathematics and the Education of Gifted Students, 129-145.
  119. Leikin, R., Koichu, B., & Berman, A. (2009). Mathematical giftedness as a quality of problem-solving acts. Creativity in mathematics and the education of gifted students, 115-127.
  120. Leikin, R., & Lev, M. (2007). Multiple solution tasks as a magnifying glass for observation of mathematical creativity. In Proceedings of the 31st international conference for the psychology of mathematics education (Vol. 3, pp. 161-168).
  121. Li, D., & Brennan, R. (2007). A multi-group generalizability analysis of a large-scale reading comprehension test. In annual meeting of the National Council on Measurement in Education. Chicago, IL.
  122. Lynch, B. K., & McNamara, T. F. (1998). Using G-theory and many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing 15(2), 158-180. https://doi.org/10.1177/026553229801500202
  123. Mainville, W. E. (1972). A Study of the Effect of Mathematics Activity Materials: Upon Certain Aspects of Creative Thinking Ability of Prospective Elementary School Teachers. (Unpublished doctoral dissertation,, Michigan State University).
  124. Mann, E. L. (2005). Mathematical creativity and school mathematics: indicators of mathematical creativity in middle school students. (Unpublished doctoral dissertation, University of Connecticut).
  125. Matsumura, L. C., Garnier, H. E., Slater, S. C., & Boston, M. D. (2008). Toward measuring instructional interactions "at-scale". Educational Assessment 13(4), 267-300. https://doi.org/10.1080/10627190802602541
  126. Maxwell, A. A. (1974). An Exploratory Study of Secondary School Geometry Students: Problem Solving Related to Convergent-Divergent Productivity. (Unpublished doctoral dissertation, University of Marine).
  127. Measures of Effective Teaching. (2009). Measures of effective teaching. Retrieved from www.metproject.org
  128. Messick, S. (1989). Validity. in R. L. Linn (Ed.), Educational measurement (3rd ed). New York: Macmillan.
  129. Morrow Jr, J. R., Fridye, T., & Monaghen, S. D. (1986). Generalizability of the AAHPERD health related skinfold test. Research Quarterly for Exercise and Sport 57(3), 187-195. https://doi.org/10.1080/02701367.1986.10605396
  130. Polya, G. (1973). How to solve it. Princeton: Princeton University Press.
  131. Powers, S., & Brennan, R. L. (2009). Multivariate generalizability analyses of mixedformat exams. In Annual Meeting of the National Council on Measurement in Education, San Diego, CA.
  132. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement 30(3), 215-232. https://doi.org/10.1111/j.1745-3984.1993.tb00424.x
  133. Shavelson, R. J., Lung, H., & Lewin, B. (1993). On Concept Maps as Potential "Authentic" Assessments in Science. Indirect Approaches to Knowledge Representation of High School Science. National Center for Research on Evaluation Standards and Student Testing, Office of Educational Research and Improvement.
  134. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer . London: Sage Publications.
  135. Sheffield, L. J. (1994). The development of gifted and talented mathematics students and the National Council of Teachers of Mathematics Standards. PA: Diane.
  136. Shriki, A. (2010). Working like real mathematicians: developing prospective teachers' awareness of mathematical creativity through generating new concepts. Educational studies in mathematics 73(2), 159-179. https://doi.org/10.1007/s10649-009-9212-2
  137. Shriki, A. (2013). A model for assessing the development of students' creativity in the context of problem posing. Creative Education 4(07), 430. https://doi.org/10.4236/ce.2013.47062
  138. Silver, E. A. (1997). Fostering creativity through instruction rich in mathematical problem solving and problem posing. Zentralblatt fur Didaktik der Mathematik 29(3), 75-80. https://doi.org/10.1007/s11858-997-0003-x
  139. Sriraman, B. (2004). The characteristics of mathematical creativity. The Mathematics Educator 14(1). 19-34.
  140. Stamm, C. L., & Moore, J. E. (1980). Application of generalizability theory in estimating the reliability of a motor performance test. Research quarterly for exercise and sport 51(2), 382-388. https://doi.org/10.1080/02701367.1980.10605206
  141. Stratford, P. W., Norman, G. R., & McIntosh, J. M. (1989). Generalizability of grip strength measurements in patients with tennis elbow. Physical therapy 69(4), 276-281. https://doi.org/10.1093/ptj/69.4.276
  142. Taylor, J. L. (1979). Development of the physical education observation instrument using generalizability study theory. Research Quarterly. American Alliance for Health, Physical Education, Recreation and Dance 50(3), 468-481. https://doi.org/10.1080/00345377.1979.10615634
  143. Torrance, E. P. (1966). Torrance tests of creative thinking: norms technical manual (Research Edition). Princeton, NJ: Personnel Press.
  144. Torrance, E. P. (1974). Norms-technical manual: Torrance tests of creative thinking. Lexington, MA: Ginn and Company.
  145. Tuli, M. R. (1980). Mathematical creativity as related to aptitude for achievement in and attitude towards mathematics. (Unpublished doctoral dissertation, Panjab University).
  146. Wan, L., & Brennan, R. L. (2005). Reliability of scores for tests composed of testlets: A comparison of approaches in three measurement models. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Montreal, Canada.
  147. Webb, N. M., Shavelson, R. J., & Maddahian, E. (1983). Multivariate generalizability theory. In I. J. Pyans, Jr.(Ed.), Generalizability theory: Inference and practical applications (pp.67-81). San Francisco, CA: Jossey-Bass.
  148. Webb, N. M., Schlackman, J., & Sugrue, B. (2000). The dependability and interchangeability of assessment methods in science. Applied Measurement in Education 13(3), 277-301. https://doi.org/10.1207/S15324818AME1303_4
  149. Wilhelm, A. G., & Kim, S. (2015). Generalizing From Observations of Mathematics Teachers' Instructional Practice Using the Instructional Quality Assessment. Journal for Research in Mathematics Education 46(3), 270-279. https://doi.org/10.5951/jresematheduc.46.3.0270