Worldwide, millions of patients die every year as a direct result of unsafe care.1 The Institute of Medicine (IOM) of the United States released a groundbreaking report “To Err is human,” which stated that 44 000 to 98 000 people die every year from iatrogenic medical events, 7 000 of which were from medical errors.2

In Palestine, adverse events are frequent and sometimes serious but can be prevented. One out of seven patients suffers harm in Palestinian hospitals.3 Patient safety is defined as the avoidance and prevention of adverse events or injuries stemming from the processes of health care.4 Patient safety culture is an essential component of health care quality measures, and contributes to minimizing the risk of adverse events.5 Recently, a focus on patient safety culture and patient outcomes in healthcare organizations has attracted a lot of attention because of its importance in creating a work environment that enables the delivery of safe care.6 Patient safety culture is defined as a subset of organizational culture that comprises values and beliefs concerning patient safety within healthcare organizations.7 Assessment of the patient safety culture can be done either qualitatively (on site observation, interview, focus group discussion) or quantitatively (questionnaires). The most widely used approach is face-to-face interviews or self-administered questionnaires.6

While a variety of instruments exists to assess patient safety culture, the most widely used are the Safety Attitude Questionnaire (SAQ),8 and the Hospital Survey on Patient Safety Culture (HSPSC).9 The SAQ shows a positive association between obtained score and improvement in patient care delivery, and remains the most sensitive in evaluating safety culture.10 The SAQ is recommended by many authors.8,9,11

The SAQ was cross-culturally validated in different languages including English,8 Swedish,11 Turkish,12 Dutch,13 and Norwegian.14 All these studies have shown that the SAQ possesses good psychometric properties in different languages. The SAQ intensive care unit (ICU) version was translated into Arabic by Hamdan15 and was tested only for internal consistency in which Cronbach alpha ranged from 0.59 to 0.75. He suggested further revision of his translated tool. We aimed to translate the SAQ (short form 2006) into Arabic and broadly test its psychometric properties to present a validated tool to Palestinian hospitals.

Methods

The SAQ was developed to measure healthcare workers attitudes regarding safety climate. The instrument was refined from the Intensive Care Unit Management Attitudes Questionnaire (ICUMAQ),16 which was derived from the Flight Management Attitudes Questionnaire (FMAQ).17 SAQ was based on two conceptual models: the Donabedian model for quality and the Vincent framework for analyzing risk and safety.18,19 We used the SAQ (short form 2006) developed by Sexton et al,8 which comprises of 32 items constitutes six dimensions: teamwork climate, safety climate, job satisfaction, stress recognition, working condition, and perception of management, the latter of which is measured on two levels: hospital management and unit management level.

Modifications were made to the SAQ Arabic version (SAQ/AV). Items 33–36, were not part of the original SAQ scale because they are not standard items and have been used because they were relevant for the purpose of our future research project. Item 36, “communication breakdown that lead to delays in delivery of care are common”, was added to working conditions, and item 29 “the levels of staffing in this clinical area are sufficient to handle the number of patients” was moved to working conditions from perception of management because within the Palestinian culture these items are considered to be fundamental to workplace safety.

The SAQ used in this study of validity and reliability thus comprised 32 items divided into six dimensions, which was previously mentioned. Items 14, 33−35 were excluded from psychometric analysis because they are not part of the original instrument. Items 33−35 describe hospital safety behavior. Items 2, 11, and 36 are reverse scored and SAQ items are scored on five-point Likert-type scale with response choices of disagree strongly = 1, disagree = 2, neutral = 3, agree = 4, and agree strongly = 5.

Permission to use the original SAQ was obtained from the Center for Healthcare Quality and Safety, University of Texas, US. It was downloaded from https://med.uth.edu/chqs/surveys/safety-attitudes-and-safety-climate-questionnaire/. The translation process followed the guidelines of the Agency for Healthcare Research and Quality, which comprises of seven steps.20

The SAQ (short form 2006) was given separately to two independent professional translators who were native Arabic speakers, fluent in English, and had health experience. The two resulting translations were sent to colleagues in the health sector with experience in survey development. The translations were carefully reviewed for discrepancies before signing off the final draft of the Arabic SAQ. It was then sent for backward translation to two independent translators who had never seen the original questionnaire. Back-translation method is preferred because it gives an indication of semantic equivalence and can enhance the validity of SAQ.

The SAQ/AV was delivered to six nurses and six physicians for face validity. They were asked to evaluate the appearance of the questionnaire in terms of readability, consistency of style and formatting, and the clarity of the language used. Where statements were not easy understandable, they were asked to rephrase them.

Content validity was undertaken to ascertain whether the content of the questionnaire was appropriate and relevant to the study purpose. A convenient sample of 13 academics and health experts received the SAQ/AV to rate items for content validity. The raters had to meet at least one of three criteria to be chosen: knowledge of the English language; experience in the field of patient safety or previous participation in research involving the translation and validation of scales. The raters were independently asked to rate the relevance of each item to the related domain using the four points Likert scale: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = highly relevant. Researchers in favor of this scale indicated that rating of one and two considers “content invalid” while rating of three and four considers “content valid”.21 Nine raters responded, and item content validity index (I-CVI) and scale content validity index (S-CVI) were determined accordingly using the average approach (CVI/Ave).This approach estimated CVI as proportion of items that received a rating of three or four by the raters. The formula for I-CVI was:

I-CVI =number of judges rated 3 and 4 /Total number of judges

To verify inter-rater agreements, many statisticians recommended the Cohen coefficient Kappa (k).22 It represents the proportion of agreement after chance agreement is removed, and ranges from -1 to +1.

The construct validity was assessed through confirmatory and exploratory factor analysis (EFA) by a means of survey. Confirmatory factor analysis (CFA) was performed using IBM AMOS software version 22. With CFA, the researchers calculated the following indices to assess goodness of fit: comparative fit indices (CFI > 0.90),23 standardized root square residual (SRMR < 0.08),23 root mean square error of approximation (RMSEA < 0.06),23,24 Tucker Lewis Index (TLI close to 0.95),24 and its 90% confidence interval (CI).

The EFA was performed through factor analysis in SPSS Statistics (SPSS Statistics IBM Corp., Armonk, NY) version 20. The suitability of the data for carrying out such analysis was tested using the Bartlett test of sphericity and the Kaiser-Meyer-Olkin (KMO) statistic test.

The reliability of SAQ/AV was tested through measurement and assessment of internal consistency with Cronbach’s alpha value. Moreover, inter-correlation between SAQ/AV scales was tested with the Pearson correlation and half split technique was determined by measuring the intraclass correlation coefficient (ICC) to assess inter rater consistency of the raters. The benchmark for the ICC was as > 0.75 = excellent, between 0.40 and 0.75 = moderate, and < 0.40 = poor.25

This was a methodological study to evaluate the reliability and validity of the SAQ/AV (short form 2006). There are seven public general hospitals distributed in four Gaza governorates: two in the north, one in Gaza, one in the middle area, and three in the south. In order to have a representative sample from the entire Gaza Strip, four hospitals were selected, one from each governorate. Where a governorate had more than one hospital (north and south), randomization was applied. All hospital inpatient wards were included in the study.

Data were collected from June 2013 to March 2014 by six trained nursing students. The questionnaires were administered by face-to-face interview. The nurses and physicians were provided with information about the study purposes and 15 minutes was enough to complete the questionnaire, similar to Sexton et al.8A signed consent form was obtained from the participants.

Numbers of nurses and physicians working in inpatient medical, surgical, ICU, and maternity wards were obtained from hospitals’ administrations. Only full time nurses and physicians with at least six months experience were included. The total number of physicians and nurses who met the inclusion criteria were 1 295 (45% physicians and 55% nurses). Midwives were part of the study sample because they perform nursing duties.

The sampling method for this study was proportional systematic sampling. We aimed for a representative sample according to the distribution of physicians and nurses in the selected hospitals and within the selected wards. Epi info software was used to determine the sample size with 95% CI yielding an estimated 370 nurses and physicians.

Results

Of the 370 participants, 339 responded yielding a response rate of 91.6% (68.4% males, 31.6% females). The response rate was similar among nurses and physicians (91.9% and 91.8%, respectively). The average age of respondents was 36±2.6 years. Males accounted for over half of the nurses (54%) and most of the physicians (88%).

Regarding face validity, the majority of reviewers said that the general shape of the questionnaire was organized and well arranged. Moreover, the questions were clear and easy to understand, except for questions 3, 13, and 21. As a result of the feedback, changes were made to these items. In item 3, the researchers added “related to patient care” to clarify the intended meaning: disagreements related to patient care in this clinical area are resolved appropriately. The word “culture” in question 13 was not properly presented, so we added the definition of culture in brackets: the culture (shared values and beliefs within organization) in this clinical area makes it easy to learn from the errors of others. Finally, the Arabic word initially used for “fatigue” in question 21 was too vague, and an alternative Arabic word that more closely matched the English intent was used.

For content validity, the CVI/Ave and k were calculated. The I-CVI and S-CVI ranged from 0.77−1.00 and 0.85−0.97, respectively and k for the questionnaire’s items ranged from 0.76−1.00 (supplemental file).

The internal consistency of the SAQ/AV was examined through determination of the Cronbach alpha value, the inter-correlation between the scales, and the half split technique. The Cronbach alpha value for entire questionnaire was 77.7 (74.7−82.2). The half split technique showed moderate to excellent correlation as measured by the ICC and the Guttman split-half coefficient. The ICC and the Guttman split-half coefficient of SAQ/AV were 0.846 and 0.775, respectively; and ranged 0.669−0.919 and 0.727−0.927, respectively [Table 1]. The correlation between the SAQ/AV scales was positive, except for stress recognition, and ranged from 0.443−0.736 (p < 0.010) [Table 2].

Discussion

This study is unique and important since it touches on the essential components of health care quality. It is the first time that the Agency for Healthcare and Research Quality (AHRQ) guidelines have been used to translate into Arabic.

This study evaluated the psychometric properties (validity and reliability) of the SAQ/AV. The first translation of the SAQ had problems with conveying intended meanings of some words. For example, in item 1 “Nurse input is well received in this clinical area”, the word “input” was difficult to translate, although it had equivalent in Arabic. The best translation given was “Nurse’s views and suggestions are well received in this clinical area.” Similarly, the Swedish translation amended the same question of the SAQ, and the word “input” was translated as “suggestion and feedback.”26 Other words also presented challenges in the Arabic translation but not in other languages, for example “Fatigue” in item 20 and “culture” in item 13 were not amended in other studies.14,27 Generally, the Arabic wording in the translated SAQ was clear, unambiguous, and easy to understand.

The CVI for items and scales showed high relevance and excellent agreement between raters (k > 0.75). Unlike previous studies,27,28 the majority of items had I-CVI equal to or above 0.88, while the lowest six items received 0.77, which are still above the recommended value of 0.75, indicating good content validity.27 The S-CVI for all domains was above the recommended value of 0.90 except for stress recognition, which was 0.85. Polit et al,29 recommended at least 0.90 for S-CVI, while Zimmermann et al,27 revealed that a S-CVI of 0.83 indicated good content validity and suggesting participants had no problem in understanding the questionnaire items.

Compared to previous studies whose response rates ranged from 52% to 79%, our response rate was a tremendous 91.6%.8,11,13,14,28 This result could be attributed to the excellent training provided to the nursing students, and to the face-to-face interview method of collecting the data (trained students sat with the participants and provided information about the study and its objectives). Also, the encouragement of hospital leaders to staff to participate in the study, the fact that participation was anonymous, and relatively short time taken to complete the questionnaire, all contributed to the high response rate. Interestingly, several participants said, “The topic of patient safety attracted us and we would like to see the feedback as soon as you disseminate your results.”

With regard to reliability analysis, a Cronbach alpha value of < 0.70 indicates low correlation among items. The Arabic version had a Cronbach alpha equal to 77.7 (74.7−82.2). These findings are quite similar to and sometimes better than those of previous studies.12,13,26,28,30–32

The good internal consistency indicates that the SAQ/AV items measure the same concepts of attitude and behavioral aspects of health providers in view of patient safety in their clinical area. Furthermore, the high alpha value for the SAQ/AV factors may indicate good internal consistency of the whole instrument.33 The half-split technique showed high correlation as measured by ICC which exceeded 0.688. Correlation between factors showed positivity with all domains except with stress recognition. The higher the perceived level of stress, the lower the total safety culture score is. This findings are consistent with previous studies.8,11,13 Items of stress recognition address self-behavior, for instance: “fatigue impairs my performance during emergency situation.” Unlike the stress recognition items, the items of other scales address the behavior and attitudes of health workers and their effect on the safety climate. The strongest correlation (r = 0.70) was between teamwork climate and safety climate, which differs from the findings of Sexton et al,8 and Kaya et al,12 who found a strong correlation between working conditions and safety climate.

Confirmatory factor analysis of the 32 items was generally satisfactory but less than perfect compared with other results.24,30,31 The p-value of < 0.001 is against the fit of the model to the data, as TLI (0.780) and CFI (0.797) were below the recommended cut off values. RMSEA (0.085) exceeds the suggested value of 0.08 and SRMR (0.074) is below the cut off value of 0.10.

The distribution of the questions among factors was found to be slightly different from that in the original SAQ. The questions of teamwork and safety climate were merged into one factor. A new factor was appeared which included one question from teamwork climate, one from safety climate, and the question 36 of the short form 2006. This factor was named “communication openness”. This stresses the importance of communication as a driver to patient safety in Palestine and should be considered when seeking to assess the patient safety culture status quo in the future. In addition, two questions from perception of management were shifted to working condition; however, the panel of expert (survey team) had suggested before to move the question 29 to working condition because it determines the workplace safety in Palestinian culture. No changes have made in the stress recognition and job satisfaction.

In view of the changes that were necessary to make the study feasible in Palestinian settings, it appears that the SAQ/AV has generalizability limitation in the cross-cultural settings.

Conclusion

The Arabic version of the SAQ shows good reliability, face, and content validity. The analysis of construct validity through CFA and EFA presents a satisfactory model, although minor adjustment and further evaluation of some items could be considered if reconsidering or improving the psychometric soundness for future use in other contexts. Generalization could be made after verifying again the reliability and validity of the SAQ. Safety culture assessment is one of the mandated requirements for any healthcare accreditation. Therefore, this tool will be a much-needed, useful, and appropriate aid to assess the safety attitudes of healthcare workers in Palestinian hospitals.

Disclosure

The authors declared no conflicts of interest. No funding was received for this study.

Acknowledgements

We are grateful to all the nurses and physicians who participated in the study and to the Palestinian Ministry of Health and hospitals management, which facilitated the work. Special thanks to Mss Julie Webb-Pullman for revision and editorial assistance. We would like to thank IC-TUMS for their support.

references

1. 1. Jha AK, Prasopa-Plaizier N, Larizgoitia I, Bates DW. Patient safety research: an overview of the global evidence. Qual Saf Health Care 2010 Feb;19(1):42-47.
2. 2. Bates DW. Preventing medication errors: a summary. Am J Health Syst Pharm 2007 Jul;64(14)(Suppl 9):S3-S9, quiz S24-S26.
3. 3. Najjar S, Hamdan M, Euwema MC, Vleugels A, Sermeus W, Massoud R, et al. The Global Trigger Tool shows that one out of seven patients suffers harm in Palestinian hospitals: challenges for launching a strategic safety plan. Int J Qual Health Care 2013 Dec;25(6):640-647.
4. 4. Cooper JB, Gaba DM, Liang B, Woods D, Blum LN. The National Patient Safety Foundation agenda for research and development in patient safety. MedGenMed 2000 Jul;2(3):E38.
5. 5. Huang DT, Clermont G, Kong L, Weissfeld LA, Sexton JB, Rowan KM, et al. Intensive care unit safety culture and outcomes: a US multicenter study. Int J Qual Health Care 2010 Jun;22(3):151-161.
6. 6. Kline TJ, Willness C, Ghali WA. Determinants of adverse events in hospitals–the potential role of patient safety culture. J Healthc Qual 2008 Jan-Feb;30(1):11-17.
7. 7. Feng X, Bobay K, Weiss M. Patient safety culture in nursing: a dimensional concept analysis. J Adv Nurs 2008 Aug;63(3):310-319.
8. 8. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006 Apr;6:44.
9. 9. Poley MJ, van der Starre C, van den Bos A, van Dijk M, Tibboel D. Patient safety culture in a Dutch pediatric surgical intensive care unit: an evaluation using the Safety Attitudes Questionnaire. Pediatr Crit Care Med 2011 Nov;12(6):e310-e316.
10. 10. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care 2005 Oct;14(5):364-366.
11. 11. Nordén-Hägg A, Sexton JB, Kälvemark-Sporrong S, Ring L, Kettis-Lindblad A. Assessing safety culture in pharmacies: the psychometric validation of the Safety Attitudes Questionnaire (SAQ) in a national sample of community pharmacies in Sweden. BMC Clin Pharmacol 2010 Apr;10:8.
12. 12. Kaya S, Barsbay S, Karabulut E. The Turkish version of the safety attitudes questionnaire: psychometric properties and baseline data. Qual Saf Health Care 2010 Dec;19(6):572-577.
13. 13. Kristensen S, Sabroe S, Bartels P, Mainz J, Christensen KB. Adaption and validation of the Safety Attitudes Questionnaire for the Danish hospital setting. Clin Epidemiol 2015 Feb;7:149-160.
14. 14. Deilkås ET, Hofoss D. Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), Generic version (Short form 2006). BMC Health Serv Res 2008 Sep;8:191.
15. 15. Hamdan M. Measuring safety culture in Palestinian neonatal intensive care units using the Safety Attitudes Questionnaire. J Crit Care 2013 Oct;28(5):886.e7-886.e14.
16. 16. Modak I, Sexton JB, Lux TR, Helmreich RL, Thomas EJ. Measuring safety culture in the ambulatory setting: the safety attitudes questionnaire–ambulatory version. J Gen Intern Med 2007 Jan;22(1):1-5.
17. 17. Helmreich RL, Merritt AC, Sherman PJ, Gregorich SE, Wiener EL. The Flight Management Attitudes Questionnaire (FMAQ). NASA/UT/FAA Technical Report 93-4. Austin, TX: The University of Texas; 1993.
18. 18. Donabedian A. The quality of care. How can it be assessed? JAMA 1988 Sep;260(12):1743-1748.
19. 19. Vincent C, Taylor-Adams S, Stanhope N. Framework for analysing risk and safety in clinical medicine. BMJ 1998 Apr;316(7138):1154-1157.
20. 20. Agency for Healthcare Research and Quality. 2010 Translation Guidelines for the AHRQ Surveys on Patient Safety Culture [cited 2014 July 14]. Available from: http://www.ahrq.gov/sites/default/files/wysiwyg/professionals/quality-patient-safety/patientsafetyculture/resources/transguide.pdf.
21. 21. Yaghmale F. Content validity and its estimation. J Med Educ 2003;3(1):25-27.
22. 22. Cohen JA. A Coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20(1):37-46.
23. 23. Bentler PM. Comparative fit indexes in structural models. Psychol Bull 1990 Mar;107(2):238-246.
24. 24. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 1999;6(1):1-55.
25. 25. Stone AT, Bransford RJ, Lee MJ, Vilela MD, Bellabarba C, Anderson PA, et al. Reliability of classification systems for subaxial cervical injuries. Evid Based Spine Care J 2010 Dec;1(3):19-26.
26. 26. Göras C, Wallentin FY, Nilsson U, Ehrenberg A. Swedish translation and psychometric testing of the safety attitudes questionnaire (operating room version). BMC Health Serv Res 2013 Mar;13:104.
27. 27. Zimmermann N, Küng K, Sereika SM, Engberg S, Sexton B, Schwendimann R. Assessing the Safety Attitudes Questionnaire (SAQ), German language version in Swiss university hospitals–a validation study. BMC Health Serv Res 2013 Sep;13:347.
28. 28. Devriendt E, Van den Heede K, Coussement J, Dejaeger E, Surmont K, Heylen D, et al. Content validity and internal consistency of the Dutch translation of the Safety Attitudes Questionnaire: an observational study. Int J Nurs Stud 2012 Mar;49(3):327-337.
29. 29. Polit DF, Beck CT, Owen SV. Is the CVI an Acceptable Indicator of Content Validity? Appraisal and Recommendations. Res Nurs Health 2007 Aug;30(4):459-467.
30. 30. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007 Jan;60(1):34-42.
31. 31. de Carvalho RE, Cassiani SH. Cross-cultural adaptation of the Safety Attitudes Questionnaire - Short Form 2006 for Brazil. Rev Lat Am Enfermagem 2012 May-Jun;20(3):575-582.
32. 32. Lee WC, Wung HY, Liao HH, Lo CM, Chang FL, Wang PC, et al. Hospital safety culture in Taiwan: a nationwide survey using Chinese version Safety Attitude Questionnaire. BMC Health Serv Res 2010 Aug;10:234.
33. 33. Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 2003 Feb;80(1):99-103.