Safety and Health at Work

Occupational Safety and Health Research Institute, Korea Occupational Safety and Health Agency (산업안전보건연구원)
권호 표지
DOI QR코드

DOI QR Code

Assessing Reliability and Validity of an Instrument for Measuring Resilience Safety Culture in Sociotechnical Systems

  • Shirali, Gholamabbas (Department of Occupational Health Engineering, Faculty of Public Health, Ahvaz Jundishapur University of Medical Sciences) ;
  • Shekari, Mohammad (Department of Occupational Health Engineering, Faculty of Public Health, Ahvaz Jundishapur University of Medical Sciences) ;
  • Angali, Kambiz Ahmadi (Department of Biostatistics and Epidemiology, Faculty of Public Health, Ahvaz Jundishapur University of Medical Sciences)
  • Received : 2016.04.24
  • Accepted : 2017.07.27
  • Published : 2018.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Background: Safety culture, acting as the oil necessary in an efficient safety management system, has its own weaknesses in the current conceptualization and utilization in practice. As a new approach, resilience safety culture (RSC) has been proposed to reduce these weaknesses and improve safety culture; however, it requires a valid and reliable instrument to be measured. This study aimed at evaluating the reliability and validity of such an instrument in measuring the RSC in sociotechnical systems. Methods: The researchers designed an instrument based on resilience engineering principles and safety culture as the first instrument to measure the RSC. The RSC instrument was distributed among 354 staff members from 12 units of an anonymous petrochemical plant through hand delivery. Content validity, confirmatory, and exploratory factor analysis were used to examine the construct validity, and Cronbach alpha and test-retest were employed to examine the reliability of the instrument. Results: The results of the content validity index and content validity ratio were calculated as 0.97 and 0.83, respectively. The explanatory factor analysis showed 14 factors with 68.29% total variance and 0.88 Kaiser-Meyer-Olkin index. The results were also confirmed with confirmatory factor analysis (relative Chi-square = 2453.49, Root Mean Square Error of Approximation = 0.04). The reliability of the RSC instrument, as measured by internal consistency, was found to be satisfactory (Cronbach ${\alpha}=0.94$). The results of test-retest reliability was r = 0.85, p < 0.001. Conclusion: The results of the study suggest that the measure shows acceptable validity and reliability.

Keywords

References

  1. Akselsson R, Koornneef F, Stewart S, Ward M. Resilience Safety Culture in Aviation Organisations. In: 17th World Congress on Ergonomics. Beijing, China: IEA; 2009.
  2. Reiman T, Rollenhagen C. Does the concept of safety culture help or hinder systems thinking in safety? Accid Anal Prev 2014;68:5-15. https://doi.org/10.1016/j.aap.2013.10.033
  3. International Atomic Energy Agency (IAEA). Safety culture in nuclear installations. Guidance for Use in the Enhancement of Safety Culture. Vienna (Austria): IAEA; 2002. p. 3-23.
  4. Clarke S. Perceptions of organizational safety: implications for the development of safety culture. J Organ Behav 1999;20:185-98. https://doi.org/10.1002/(SICI)1099-1379(199903)20:2<185::AID-JOB892>3.0.CO;2-C
  5. Hopkins A. Safety, culture and risk. The organisational causes of disaster. Sydney (Australia): CCH Australia; 2005.
  6. Richter A, Koch C. Integration, differentiation and ambiguity in safety cultures. Saf Sci 2004;42:703-22. https://doi.org/10.1016/j.ssci.2003.12.003
  7. Pidgeon NF. Safety culture and risk management in organizations. J Cross Cult Psychol 1991;22:129-40. https://doi.org/10.1177/0022022191221009
  8. Cooper MD. Towards a model of safety culture. Saf Sci 2000;36:111-36. https://doi.org/10.1016/S0925-7535(00)00035-7
  9. Gadd S, Collins AM. Safety Culture: A literature review. Harpur Hill (UK): Health and Safety Laboratory; 2002.
  10. Guldenmund FW. The nature of safety culture: a review of theory and research. Saf Sci 2000;34:215-57. https://doi.org/10.1016/S0925-7535(00)00014-X
  11. Guldenmund FW. The use of questionnaires in safety culture research-an evaluation. Saf Sci 2007;45:723-43. https://doi.org/10.1016/j.ssci.2007.04.006
  12. Pidgeon N. Safety culture: key theoretical issues. Work Stress 1998;12:202-16. https://doi.org/10.1080/02678379808256862
  13. Zhu J. Data envelopment analysis vs. principal component analysis: An illustrative study of economic performance of Chinese cities. Eur J Oper Res 1998;111:50-61. https://doi.org/10.1016/S0377-2217(97)00321-4
  14. Kines P, Lappalainen J, Mikkelsen KL, Olsen E, Pousette A, Tharaldsen J, Tomasson K, Torner M. Nordic Safety Climate Questionnaire (NOSACQ-50): A new tool for diagnosing occupational safety climate. Int J Ind Ergon 2011;41:634-46. https://doi.org/10.1016/j.ergon.2011.08.004
  15. Rall M, Dieckmann P. Safety culture and crisis resource management in airway management: general principles to enhance patient safety in critical airway situations. Best Pract Res Clin Anaesthesiol 2005;19:539-57. https://doi.org/10.1016/j.bpa.2005.07.005
  16. Antonsen S. Safety culture and the issue of power. Saf Sci 2009;47:183-91. https://doi.org/10.1016/j.ssci.2008.02.004
  17. Diaz-Cabrera D, Hernandez-Fernaud E, Isla-Diaz R. An evaluation of a new instrument to measure organisational safety culture values and practices. Accid Anal Prev 2007;39:1202-11. https://doi.org/10.1016/j.aap.2007.03.005
  18. Hopkins A, Andrew H. Lessons from Longford: the Esso gas plant explosion. Sydney (Australia): CCH Australia Limited; 2000.
  19. De Carvalho PV. The use of Functional Resonance Analysis Method (FRAM) in a mid-air collision to understand some characteristics of the air traffic management system resilience. Reliab Eng Syst Safe 2011;96:1482-98. https://doi.org/10.1016/j.ress.2011.05.009
  20. Dekker S, Hollnagel E, Woods D, Cook R. Resilience Engineering: New directions for measuring and maintaining safety in complex systems. Lund (Sweden): Lund University School of Aviation; 2008.
  21. Naevestad TO. Mapping research on culture and safety in High-Risk organizations: Arguments for a sociotechnical understanding of safety culture. J Contingencies Crisis Manage 2009;17:126-36. https://doi.org/10.1111/j.1468-5973.2009.00573.x
  22. Weick KE. The collapse of sensemaking in organizations: The Mann Gulch disaster. Admin Sci Q 1993:628-52.
  23. International Atomic Energy Agency (IAEA). IAEA, safety culture. Report 75-INSAG-4 International Nuclear Safety Advisory Group of the International Atomic Energy Agency. Vienna (Austria): IAEA; 1991.
  24. National Aeronautics and Space Administration (NASA). Report Volume 1 August 2003 from the Columbia accident investigation board. Houston (TX):NASA; 2003.
  25. Britain G, Hidden A. Investigation into the Clapham Junction railway accident. London (UK): HM Stationery Office; 1989.
  26. Kletz TA. Learning from accidents. 3rd ed. London (UK): Routledge; 2001.
  27. Shirali G, Ebrahipour V. Proactive risk assessment to identify emergent risks using functional resonance analysis method (FRAM): A case study in an oil process unit. Iran Occupational Health 2013;10:33-46.
  28. Wang R, Zheng W. Research and application of the BFM-STAMP hazard analysis method. In: Intelligent Rail Transportation (ICIRT), 2013 IEEE International Conference on IEEE, Philadelphia (PA); 2013.
  29. Hollnagel E. Barriers and accident prevention. Aldershot (UK): Ashgate; 2016.
  30. Checkland P. Systems thinking, systems practice. New York: John Wiley & Sons; 1981.
  31. Leveson NG. System safety engineering: Back to the future [Internet]. Cambridge (MA): Massachusetts Institute of Technology. 2002 [cited 2017 Jan 31]. Available from http://sunnyday.mit.edu/book2.pdf.
  32. Skyttner L. General systems theory: problems, perspectives, practice. Singapore: World scientific Publishing Co. Pte. Ltd; 2005.
  33. Shirali GA, Shekari M, Angali K. Quantitative assessment of resilience safety culture using principal components analysis and numerical taxonomy: A case study in a petrochemical plant. J Loss Prev Process Ind 2016;40:277-84. https://doi.org/10.1016/j.jlp.2016.01.007
  34. Woods DD, Leveson N, Hollnagel E. Resilience engineering: Concepts and precepts. Aldershot (UK): Ashgate Publishing, Ltd; 2007.
  35. Kao CS, Lai WH, Chuang TF, Lee JC. Safety culture factors, group differences, and risk perception in five petrochemical plants. Process Saf Prog 2008;27:145-52. https://doi.org/10.1002/prs.10246
  36. Shirali GA, Mohammadfam I, Ebrahimipour V. A new method for quantitative assessment of resilience engineering by PCA and NT approach: A case study in a process industry. Reliab Eng Syst Safe 2013;119:88-94. https://doi.org/10.1016/j.ress.2013.05.003
  37. Shirali GH, Mohammadfam I, Motamedzade M, Ebrahimipour V, Moghimbeigi A. Assessing resilience engineering based on safety culture and managerial factors. Process Saf Prog 2012;31:17-8. https://doi.org/10.1002/prs.10485
  38. Reason J. Managing the risks of organizational accidents, Vol. 6. Aldershot (UK): Ashgate; 1997.
  39. Saurin TA, Junior GC. Evaluation and improvement of a method for assessing HSMS from the resilience engineering perspective: a case study of an electricity distributor. Saf Sci 2011;49:355-68. https://doi.org/10.1016/j.ssci.2010.09.017
  40. Re A, Macchi L. From cognitive reliability to competence? An evolving approach to human factors and safety. Cogn Technol Work 2010;12:79-85. https://doi.org/10.1007/s10111-010-0148-1
  41. Lynn MR. Determination and quantification of content validity. Nurs Res 1986;35:382-6.
  42. American Educational Research Association (AERA), American Psychological Association (APA), National Council on Measurement in Education (NCME), Standards for educational and psychological tests. Washington (WA): American Psychological Association; 1974.
  43. Lawshe CH. A quantitative approach to content validity1. Pers Psychol 1975;28:563-75. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
  44. Johnson RA, Wichern DW. Applied multivariate statistical analysis. New Jersey:Prentice-Hall; 2014.
  45. Field A. Discovering statistics using SPSS. Thousand Oaks (CA): Sage Publications;2009.
  46. Kline RB. Principles and practice of structural equation modeling. New York (US): Guilford Publications; 2015.
  47. Jackson DL. Revisiting sample size and number of parameter estimates: Some support for the N: q hypothesis. Struct Equ Modeling 2003;10:128-41. https://doi.org/10.1207/S15328007SEM1001_6
  48. Shah R, Goldstein SM. Use of structural equation modeling in operations management research: Looking back and forward. J Oper Manag 2006;24:148-69. https://doi.org/10.1016/j.jom.2005.05.001
  49. Kline P. A handbook of test construction: introduction to psychometric design. London (UK): Methuen & Co. Ltd; 1986.
  50. Nunnally JC, Bernstein I. The assessment of reliability. Psychometric Theory 1994;3:248-92.
  51. Segall DO. The reliability of linearly equated tests. Psychometrika 1994;59:361-75. https://doi.org/10.1007/BF02296129
  52. Charter RA. Sample size requirements for precise estimates of reliability, generalizability, and validity coefficients. J Clin Exp Neuropsychol 1999;21:559-66. https://doi.org/10.1076/jcen.21.4.559.889
  53. Davis L. Instrument review: Getting the most from your panel of experts. Appl Nurs Res 1992;5:194-7. https://doi.org/10.1016/S0897-1897(05)80008-4
  54. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006;29:489-97. https://doi.org/10.1002/nur.20147
  55. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: Conducting a content validity study in social work research. Soc Work Res 2003;27:94-104. https://doi.org/10.1093/swr/27.2.94
  56. Vieira AL, Winklhofer H, Ennew CT. Relationship quality: a literature review and research agenda. J Cust Behav 2008;7:269-91. https://doi.org/10.1362/147539208X386833
  57. Marsh HW, Hau KT, Wen Z. In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct Equ Modeling 2004;11:320-41. https://doi.org/10.1207/s15328007sem1103_2
  58. Gomes JO, Woods DD, Carvalho PV, Huber GJ, Borges MR. Resilience and brittleness in the offshore helicopter transportation system: the identification of constraints and sacrifice decisions in pilots' work. Reliab Eng Syst Safe 2009;94:311-9. https://doi.org/10.1016/j.ress.2008.03.026
  59. Shirali GA, Motamedzade M, Mohammadfam I, Ebrahimipour V, Moghimbeigi A. Assessment of resilience engineering factors based on system properties in a process industry. Cogn Technol Work 2016;18:19-31. https://doi.org/10.1007/s10111-015-0343-1

Cited by

  1. The Relations between Safety Matters, Corporate Image and Performance in Logistics Company vol.17, pp.11, 2019, https://doi.org/10.15722/jds.17.11.201911.35
  2. Development and Validation of an Evaluation Tool to Measure the Effectiveness of a Smoking Cessation Training among Healthcare Providers in Malaysia: The Providers’ Smoking Cessation Training Ev vol.16, pp.21, 2018, https://doi.org/10.3390/ijerph16214297
  3. Sustainable performance management using resilience engineering vol.12, pp.None, 2018, https://doi.org/10.1177/1847979020976205
  4. The Influence of Organizational Culture on Resilience by Mediatory Effects of Occupational Stress, Job Satisfaction, and Burnout in Nurses: Structural Equation ModelingModeling vol.22, pp.6, 2020, https://doi.org/10.5812/ircmj.102332
  5. Measuring Resilience Engineering: An Integrative Review and Framework for Bench-Marking Organisational Safety vol.6, pp.3, 2020, https://doi.org/10.3390/safety6030037
  6. Organizational resilience in the oil and gas industry: A scoping review vol.133, pp.None, 2018, https://doi.org/10.1016/j.ssci.2020.105036
  7. Teaching-learning behavior in medicine according to students’ perspective: what most influences academic achievement? vol.10, pp.None, 2021, https://doi.org/10.12688/f1000research.73587.1
  8. The relationship between safety attitude and safety performance in the petrochemical industry: The mediating role of safety climate vol.68, pp.2, 2018, https://doi.org/10.3233/wor-203353
  9. Safety-II and Resilience Engineering in a Nutshell: An Introductory Guide to Their Concepts and Methods vol.12, pp.1, 2018, https://doi.org/10.1016/j.shaw.2020.11.004
  10. Developing a quantitative tool to measure the extent to which public spaces meet user needs vol.62, pp.None, 2018, https://doi.org/10.1016/j.ufug.2021.127152
  11. Safety culture maturity and resilience engineering in an oil drilling industry: A comparison study among government-owned and private companies vol.70, pp.2, 2018, https://doi.org/10.3233/wor-213583
  12. Prioritization of Contracting Methods for Water and Wastewater Projects Using the Fuzzy Analytic Hierarchy Process Method vol.14, pp.22, 2018, https://doi.org/10.3390/en14227815