Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지

A Meta Analysis of Using Structural Equation Model on the Korean MIS Research

국내 MIS 연구에서 구조방정식모형 활용에 관한 메타분석

  • Kim, Jong-Ki (Division of Management, College of Business, Pusan National University) ;
  • Jeon, Jin-Hwan (Institute of Management and Economics, Pusan National University)
  • Published : 2009.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, researches on Management Information Systems (MIS) have laid out theoretical foundation and academic paradigms by introducing diverse theories, themes, and methodologies. Especially, academic paradigms of MIS encourage a user-friendly approach by developing the technologies from the users' perspectives, which reflects the existence of strong causal relationships between information systems and user's behavior. As in other areas in social science the use of structural equation modeling (SEM) has rapidly increased in recent years especially in the MIS area. The SEM technique is important because it provides powerful ways to address key IS research problems. It also has a unique ability to simultaneously examine a series of casual relationships while analyzing multiple independent and dependent variables all at the same time. In spite of providing many benefits to the MIS researchers, there are some potential pitfalls with the analytical technique. The research objective of this study is to provide some guidelines for an appropriate use of SEM based on the assessment of current practice of using SEM in the MIS research. This study focuses on several statistical issues related to the use of SEM in the MIS research. Selected articles are assessed in three parts through the meta analysis. The first part is related to the initial specification of theoretical model of interest. The second is about data screening prior to model estimation and testing. And the last part concerns estimation and testing of theoretical models based on empirical data. This study reviewed the use of SEM in 164 empirical research articles published in four major MIS journals in Korea (APJIS, ISR, JIS and JITAM) from 1991 to 2007. APJIS, ISR, JIS and JITAM accounted for 73, 17, 58, and 16 of the total number of applications, respectively. The number of published applications has been increased over time. LISREL was the most frequently used SEM software among MIS researchers (97 studies (59.15%)), followed by AMOS (45 studies (27.44%)). In the first part, regarding issues related to the initial specification of theoretical model of interest, all of the studies have used cross-sectional data. The studies that use cross-sectional data may be able to better explain their structural model as a set of relationships. Most of SEM studies, meanwhile, have employed. confirmatory-type analysis (146 articles (89%)). For the model specification issue about model formulation, 159 (96.9%) of the studies were the full structural equation model. For only 5 researches, SEM was used for the measurement model with a set of observed variables. The average sample size for all models was 365.41, with some models retaining a sample as small as 50 and as large as 500. The second part of the issue is related to data screening prior to model estimation and testing. Data screening is important for researchers particularly in defining how they deal with missing values. Overall, discussion of data screening was reported in 118 (71.95%) of the studies while there was no study discussing evidence of multivariate normality for the models. On the third part, issues related to the estimation and testing of theoretical models on empirical data, assessing model fit is one of most important issues because it provides adequate statistical power for research models. There were multiple fit indices used in the SEM applications. The test was reported in the most of studies (146 (89%)), whereas normed-test was reported less frequently (65 studies (39.64%)). It is important that normed- of 3 or lower is required for adequate model fit. The most popular model fit indices were GFI (109 (66.46%)), AGFI (84 (51.22%)), NFI (44 (47.56%)), RMR (42 (25.61%)), CFI (59 (35.98%)), RMSEA (62 (37.80)), and NNFI (48 (29.27%)). Regarding the test of construct validity, convergent validity has been examined in 109 studies (66.46%) and discriminant validity in 98 (59.76%). 81 studies (49.39%) have reported the average variance extracted (AVE). However, there was little discussion of direct (47 (28.66%)), indirect, and total effect in the SEM models. Based on these findings, we suggest general guidelines for the use of SEM and propose some recommendations on concerning issues of latent variables models, raw data, sample size, data screening, reporting parameter estimated, model fit statistics, multivariate normality, confirmatory factor analysis, reliabilities and the decomposition of effects.

Keywords

References

  1. Arbuckle, J.L., AMOS Users Guide Version 3.6, Small Waters Co., 1997
  2. Anderson, J.C. and Gerbing, D.W., 'Structural Equation Modeling in Practice: A Review and Recommended Two-Step Approach,' Psychological Bulletin, Vol. 103, No. 4, 1988, pp. 411-423 https://doi.org/10.1037/0033-2909.103.3.411
  3. Bae, B.R., Structural Equation Modeling-Understanding, Applications, and Programming (2nd Eds.), ChoungRam, 2006
  4. Bagozzi, R.P. and Baumgartner, H., 'The Evaluation of Structural Equation Models and Hypothesis Testing,' Principles of Marketing Research, 1994, pp. 386-422
  5. Bagozzi, R.P. and Fornell, C., 'Theoretical Concepts, Measurement, and Meaning,' A Second Generation of Multivariate Analysis, Vol. 2, 1982, pp. 5-23
  6. Bagozzi, R.P. and Yi, Y.J., 'On the Evaluation of Structural Equation Models,' Journal of the Academy of Marketing Science, Vol. 16, No. 1, 1988, pp. 74-97 https://doi.org/10.1007/BF02723327
  7. Banvile, C. and Landry, M., 'Can the Field of MIS be Disciplined?,' Communication of th ACM, Vol. 32, No. 1, pp. 48-60 https://doi.org/10.1145/63238.63241
  8. Baron, R.M. and Kenny, D.A., 'The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations,' Journal of Personality and Social Psychology, Vol. 51, No. 6, 1986, pp. 1173-1182 https://doi.org/10.1037/0022-3514.51.6.1173
  9. Baroudi, J.J. and Orlikowski, W.J., 'The Problems of Statistical Power in MIS Research,' MIS Quarterly, Vol. 13, No. 1, 1989, pp. 87-106 https://doi.org/10.2307/248704
  10. Baumgartner, H. and Homburg, C., 'Ap plications of Structural Equation Modeling in Marketing and Consumer Research: A Review,' International Journal of Research in Marketing, Vol. 13, No. 2, 1996, pp. 139-161 https://doi.org/10.1016/0167-8116(95)00038-0
  11. Benbasat, I. and Weber, R., 'Research Commentary: Rethinking ‘Diversity’ in Information Systems Research,' Information Systems Research, Vol. 7, No. 4, 1996, pp. 389-399 https://doi.org/10.1287/isre.7.4.389
  12. Bentler, P.M., 'Comparative Fit Indexes in Structural Models,' Psychological Bulletin, Vol. 107, No. 2, 1990, pp. 238-246 https://doi.org/10.1037/0033-2909.107.2.238
  13. Bentler, P.M. and Chou, C.P., 'Practical Issues in Structural Modeling,' Sociological Methods and Research, Vol. 16, No. 1, 1987, pp. 78-117 https://doi.org/10.1177/0049124187016001004
  14. Bentler, P.M. and Bonnet, D.G., 'Significance Tests and Goodness of Fit in the Analysis of Covariance Structure,' Psychological Bulletin, Vol. 88, No. 3, 1980, pp. 588-606 https://doi.org/10.1037/0033-2909.88.3.588
  15. Bohrnstedt, G.W. and Knoke, D., Statics of Social Data Analysis, Peacock Publishers, 1994
  16. Bollen, K.A., Structural Equations with latent variables, John Wiley and Sons, 1989
  17. Bollen, K.A. and Long, J.S., Testing Structural Equation Models, Sage, 1993
  18. Browne, M.W., 'Asymptotically Distribution Free Methods for the Analysis for Covariance Structures,' British Journal of Mathematical and Statistical Psychology, Vol. 37, 1984, pp. 62-83 https://doi.org/10.1111/j.2044-8317.1984.tb00789.x
  19. Browne, M.W. and Cudeck, R., 'Alternative Ways of Assessing Model Fit,' Sociological Methods and Research, Vol. 21, No. 2, 1992, pp. 230-258 https://doi.org/10.1177/0049124192021002005
  20. Brekler, S.J., 'Applications of Covariance Structure Modeling in Psychology: Cause for Concern?,' Psychological Bulletin, Vol. 107, 1990, pp. 260-273 https://doi.org/10.1037/0033-2909.107.2.260
  21. Carmines, E. and McIver, J., Analyzing Models with Unobserved Variables: Analysis of Covariance Structures, Social Management: Current Issues, Sage, 1981
  22. Chin, W.W., 'Issues and Opinion on Structural Equation Modeling,' MIS Quarterly, Vol. 22, No. 1, 1998, pp. 7-16
  23. Chin, W.W., 'The Partial Least Squares Approach for Structural Equation Modeling,' in Modern Methods for Business Research, G.A. Marcoulides(Eds.), Lawrence Erlbaum Associates, 1998, pp. 295-336
  24. Chin, W.W. and Todd, P.A., 'On the Use, Usefulness, and Ease of Use of Structural Equation Modeling in MIS Research: A Note of Caution,' MIS Quarterly, Vol. 19, No. 2, 1999, pp. 237-246 https://doi.org/10.2307/249690
  25. Chin, W.W. and Newsted, P.R., 'Structural Equation Modeling Analysis with Small Samples Using Partial Least Squares,' in Statistical Strategies for Small Sample Research, R.H. Hoyle(Eds.), Sage, 1999, pp. 307-341
  26. Chou, C.P. and Bentler, P.M., Estimates and Tests in Structural Equation Modeling, Structural Equation Modeling: Concepts, Issues, and Applications, Sage, 1995
  27. Cliff, N., 'Some Cautions Concerning the Application of Causal Modeling Methods,' Multivariate Behavioral Research, Vol. 18, 1983, pp. 115-126 https://doi.org/10.1207/s15327906mbr1801_7
  28. Cudeck, R., 'Analysis of Correlation Matrics Using Covariance Structure Models,' Psychological Bulletin, Vol. 105, No. 2, 1989, pp. 317-327 https://doi.org/10.1037/0033-2909.105.2.317
  29. Dearden, J., 'MIS is a Mirage,' Harvard Business Review, Vol. 50, No. 1, 1972, pp. 90-99
  30. Emery, J.C. and Sprague, C.R., 'MIS: Mirage or Misconception,' Harvard Business Review, Vol. 50, No. 3, 1972, pp. 22-23
  31. Fornell, C. and Bookstein, F.L., 'Two Structural Equation Models: Lisrel and PLS Applied to Consumer Exit-Voice Theory,' Journal of Marketing Research, Vol. 19, No. 4, 1982, pp. 440-452 https://doi.org/10.2307/3151718
  32. Fornell, C. and Cha, J.S., 'Partial Least Squares,' in Advanced Methods of Marketing, R.P. Bagozzi(Eds.), Oxford, Blackwell Publishers, 1994, pp. 52-78
  33. Fornell, C. and Larcker, D.F., 'Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,' Journal of Marketing Research, Vol. 18, No. 1, 1981, pp. 39-50 https://doi.org/10.2307/3151312
  34. Garver, M.S. and Mentzer, J.T., 'Logistics Research Methods: Employing Structural Equation Modeling to Test for Construct Validity,' Journal of Business Logistics, Vol. 20, No. 1, 1999, pp. 33-57
  35. Gefen, D., 'Assessing Unidimensionality through LISREL: An Explanation and Example,' Communications of the Association for Information Systems, Vol. 12, 2003, pp. 23-47
  36. Gefen, D., Straub, D.W., and Boudreau, M.C., 'Structural Equation Modeling and Regression: Guidelines for Research Practice,' Communications of the Association for Information Systems, Vol. 4, No. 7, 2000, pp. 1-75
  37. Hair, J.F., Anderson, R.E., Tatham, R.L.,and Black, W.C., Multivariate Data Analysis(5th Eds.), Prentice Hall, 1998
  38. Hayduck, L.A., Structural Equation Modeling with LISREL: Essentials and Advances, Johns Hopkins University Press, 1987
  39. Hayduck, L.A., Cummings, G.G., Boadu, K., Pazderka-Robinson, H., and Boulianne, S., 'Testing! testing! one, two, three-Testing the theory in structural equation models!,' Personality and Individual Differences, Vol. 42, No. 5, 2007, pp. 841-850 https://doi.org/10.1016/j.paid.2006.10.001
  40. Holbert, R.L. and Stephenson, M.T., 'Structural Equation Modeling in the Communication Sciences,' Human Communication Research, Vol. 28, No. 4, 2002, pp. 531-551
  41. Hoogland, J.J. and Boomsma, A., 'Robustness Studies in Covariance Structure Modeling: An Overview and a Meta-Analysis,' Sociological Methods and Research, Vol. 26, No. 3, 1998, pp. 329-367 https://doi.org/10.1177/0049124198026003003
  42. Hoyle, R.H. and Panter, A.T., Writing about Structural Equation Modeling: Concepts, Issues, and Applications, Sage, 1995
  43. Hu, L. and Bentler, P.M., 'Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives,' Structural Equation Modeling, Vol. 6, No. 1, 1999, pp. 1-55 https://doi.org/10.1080/10705519909540118
  44. Joreskog, K.G. and Sorbom, D., Structural Equation Modeling with the SIMPLIS Command Language, Scientific Software International, 1993
  45. Joreskog, K.G. and Sorbom, D., LISREL8: Structural Equation Modeling with the SIMPLIS command language, Scientific Software International, 1996
  46. Kang, S.C., 'A Critical Evaluation of the Use of Statistical Methods in an MIS Journal,' Asia Pacific Journal of International Systems, Vol. 7, No. 2, 1997, pp. 77-102
  47. Kang, S.C., Lee, Z.K., and Choi, J.I., 'Application of Empirical Research Methods in Information Systems Research: Gaining Lessons Through Evaluation,' Asia Pacific Journal of International Systems, Vol. 16, No. 2, 2006, pp. 1-25
  48. Kelloway, E.K., Using LISREL for Structural Equation Modeling: A Researcher's Guide, Sage, 1998
  49. Kim, J.H., Hong, S.H., and Choo, B.D., 'Applications of Structural Equation Modeling in Management Studies: A Critical Review,' Korean Management Review, Vol. 36, No. 4, 2007, pp. 897-923
  50. Kim, J.K., Lim, H.S., and Lee, D.H., 'A Meta-Analysis of Reliability and Validity of Research Instruments in Korean MIS Research,' Asia Pacific Journal of International Systems, Vol. 11, No. 4, 2001, pp. 81-98
  51. Lee, S.M., Analysis of Covariance Structural Modeling, Sung Won Sa, 1990
  52. MacCallum, R.C. and Austin, J.T., 'Applications of Structural Equation Modeling in Psychological Research,' Annual Reviews of Psychology, Vol. 51, No.1, 2000, pp. 201-226 https://doi.org/10.1146/annurev.psych.51.1.201
  53. MacCallum, R.C., Brwone, M.W., and Sugawara, H.M., 'Power Analysis and Determination of Sample Size for Covariance Structural Modeling,' Psychological Methods, Vol. 1, 1996, pp. 130-149 https://doi.org/10.1037/1082-989X.1.2.130
  54. MacKinnon, D.P., 'Contrasts in Multiple Mediator Models,' in Multivariate Applications in Substance Use Research: New Methods for New Questions, J.S. Rose and L. Chassin(Eds.), Erlbaum, 2000, pp. 141-160
  55. Marsh H.W. and Hau, K.T., 'Assessing Goodness of Fit: Is Parsimony Always Desirable?' Journal Experimental Education, Vol. 64, 1996, pp. 364-390 https://doi.org/10.1080/00220973.1996.10806604
  56. McDonald, R.P., Path Analysis with Composite Variables, Multivariate Behavioral Research, Vol. 31, No. 2, 1996, pp 239-270 https://doi.org/10.1207/s15327906mbr3102_5
  57. McDonald, R.P. and Ho, M.R., 'Principles and Practice in Reporting Structural Equation Analyses,' Psychological Methods, Vol. 7, No. 1, 2002, pp. 64-82 https://doi.org/10.1037/1082-989X.7.1.64
  58. Lee, B.T., Baura, A., and Whinston, A.B., 'Discovery and Representation of Causal Relationship in MIS Research: A Methodological Framework,' MIS Quarterly, Vol. 21, No. 1, 1997, pp. 109-136 https://doi.org/10.2307/249744
  59. Long, J.S., Covariance Structure Models: An Introduction to LISREL, Sage, 1983
  60. Rigdon, E.E., 'A Necessary and Sufficient Identification Rule for Structural Models Estimated in Practice,' Multivariate Behavioral Research, Vol. 30, No. 3, 1995, pp. 359-383 https://doi.org/10.1207/s15327906mbr3003_4
  61. Park H.S., 'Research Trends on International Journal of Tourism Sciences with SEM,' Journal of Tourism Sciences, Vol. 29, No. 1, pp. 267-286
  62. Petter, S., Straub, D.W., and Rai, A., 'Specifying Formative Constructs in Information Systems Research,' MIS Quarterly, Vol. 31, No. 3, 2007, pp. 623-656 https://doi.org/10.2307/25148814
  63. Segars, A.H., 'Assessing the Unidimensionality of Measurement: A Paradigm and Illustration Within the Context of Information Systems,' Omega, Vol. 25, No. 1, 1997, pp. 107-121 https://doi.org/10.1016/S0305-0483(96)00051-5
  64. Segars, A.H. and Grover, V., 'Re-Examining Perceived Ease of Use and Usefulness: A Confirmatory Factor Analysis,' MIS Quarterly, Vol. 17, No. 4, 1993, pp. 517-525 https://doi.org/10.2307/249590
  65. Steenkamp, J.E. and Baumgartner, H., 'On the Use of Structural Equation Models for Marketing Modeling,' International Journal of Research in Marketing, Vol. 17, No. 2, 2000, pp. 195-202 https://doi.org/10.1016/S0167-8116(00)00016-1
  66. Shah, R. and Goldstein, S.M., 'Use of Structural Equation Modeling in Operations Management Research: Looking Back and Forward,' Journal of Operations Management, Vol. 24, No. 2, 2006, pp. 148-169 https://doi.org/10.1016/j.jom.2005.05.001
  67. Shook, C.L., David, J.K., Hult, G.T., and Kacmar, K.M., 'An Assessment of the Use of Structural Equation Modeling in Strategic Management Research,' Strategic Management Journal, Vol. 25, No. 4, 2004, pp. 397-404 https://doi.org/10.1002/smj.385
  68. Straub, D.W., 'Validating Instruments in MIS Research,' MIS Quarterly, Vol. 13, No. 2, 1989, pp. 147-169 https://doi.org/10.2307/248922
  69. Swanson, E.B. and Ramiller, N.C., 'Information Systems Research Thematics: Submissions to a New Journal, 1987-1992,' Information Systems Research, Vol. 4, No. 4, pp. 299-330 https://doi.org/10.1287/isre.4.4.299
  70. Tanaka, J.S., 'How Big is Big Enough? Sample Size and Goodness of Fit in Structural Equation Models with Latent Variables,' Child Development, Vol. 58, No. 1, pp. 134-146 https://doi.org/10.2307/1130296