SYSTEMATIC REVIEW AND META-ANALYSIS


https://doi.org/10.5005/jp-journals-10005-2973
International Journal of Clinical Pediatric Dentistry
Volume 17 | Issue 10 | Year 2024

Child Oral Health Impact Profile Questionnaire: A Reliability Generalization Meta-analysis of Cronbach’s Alpha


Kalyana C Pentapati1, Deepika Chenna2, Vijay S Kumar3, Nanditha Kumar4, Saurabh Kumar5

1Department of Public Health Dentistry, Manipal College of Dental Sciences, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

2Department of Immunohematology and Blood Transfusion, Kasturba Medical College, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

3Department of Public Health Dentistry, Amrita School of Dentistry, Amrita Vishwa Vidyapeetham, Kochi, Kerala, India

4Department of Prosthodontics, JSS Dental College and Hospital, JSS Academy of Higher Education and Research, Mysuru, Karnataka, India

5Department of Pedodontics and Preventive Dentistry, Manipal College of Dental Sciences, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Corresponding Author: Kalyana C Pentapati, Department of Public Health Dentistry, Manipal College of Dental Sciences, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India, Phone: +91 9916036303, e-mail: kalyan.cp@manipal.edu

ABSTRACT

Background and aim: To evaluate the pooled estimates of Cronbach’s alpha for the Child Oral Health Impact Profile (COHIP) questionnaire using reliability generalization meta-analysis.

Methods: We have conducted a systematic search of literature from PubMed, Scopus, Excerpta Medica Database (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) from inception till 31st May 2023. Studies in English and those that have reported Cronbach’s alpha for the COHIP questionnaire were included. Screening was performed by two review authors independently. Information on authors, year, country, sample size, age, sex, target population, language and mode of administration, conditions studied for, study setting, and study design were recorded. The risk of bias was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist for internal consistency. Meta-analysis was performed using the random effects model to derive a pooled estimate of Cronbach’s alpha.

Results: A total of 626 studies were available for title and abstract screening after the removal of duplicates, and 106 studies were included for full text screening. A total of 25 estimates were obtained from 22 studies which yielded a total sample size of 10,639. The overall pooled Cronbach’s alpha was 0.87 (95% CI = 0.85–8.88) with a high heterogeneity among the included publications (I2 = 95.74%). Similarly, the parent or the caregiver reported COHIP also showed pooled Cronbach’s alpha of 0.9 (95% CI: 0.87–0.93). Meta-regression showed no significant effect of sex (coefficient: −0.12), age (coefficient: −0.007), language (coefficient: 0.003), study setting (coefficient: −0.001), risk of bias (coefficient: −0.01), and continent (coefficient: −0.04) on the overall estimate.

Conclusion: The COHIP questionnaire showed good internal consistency and can be used in research and practice among children.

How to cite this article: Pentapati KC, Chenna D, Kumar VS, et al. Child Oral Health Impact Profile Questionnaire: A Reliability Generalization Meta-analysis of Cronbach’s Alpha. Int J Clin Pediatr Dent 2024;17(10):1193-1198.

Source of support: Nil

Conflict of interest: None

Keywords: Children, Child Oral Health Impact Profile, Cronbach’s alpha, Oral health, Quality of life.

INTRODUCTION

“Oral Health-related Quality of Life” (OHRQoL) characterizes individuals’ views on how oral health could affect their overall well-being and ability to perform daily activities.1 OHRQoL assessment is considered an acceptable measure to evaluate the individual’s well-being, and outcomes of oral health care programs. The improvement in the OHRQoL can also be used to assess the success of the interventions and public health activities. The tools that are used to assess the OHRQoL have many advantages: self-reported, readily available and accessible, easily understandable content, noninvasive, inexpensive, repeatable, valid, and reliable.

Many OHRQoL instruments like “Social impacts of dental disease,” “Geriatric oral health assessment index,” “Oral health impact profile,” “dental impacts on daily living,” and “oral impacts on daily performances” have been developed in the past.2 These instruments have been validated among diverse populations, in different languages, across different age-groups, and among individuals having a variety of oral conditions. Questionnaires for children and adolescents have also been developed to evaluate the specific issues that arise during growth and development. They include the “Child Perception Questionnaire,” “the Early Childhood Oral Health Impact Score,” “Pediatric Oral Health-Related Quality of Life,” “Child Oral Impacts on Daily Performances (C-OIDP),” and the “Child Oral Health Impact Profile (COHIP).” Most of these instruments are specific to certain age-groups while C-OIDP and COHIP can be used among children and adolescents.3

The COHIP questionnaire is a self- or parent/caregiver-reported 34-item questionnaire with five subscales (“oral health,” “functional well-being,” “social-emotional well-being,” “school environment,” and “self-image”) which can be used among children and adolescents of 8–15 years with a variety of oral conditions like caries,4-8 malocclusion, overjet and orthodontic treatment need,5,9-12 and craniofacial conditions.10,13-22 The oral health subscale consists of 10 items on specific oral symptoms. The functional well-being subscale includes six items on the ability to carry out specific daily activities. Social-emotional well-being subscale has eight items related to peer interactions and mood states. The school environment incorporates four items on tasks connected with the school environment. The subscale on self-image consists of six items linked to positive feelings about self. The experiences of the participants over the past 3 months were recorded on a five-point Likert scale (“never,” “almost never,” “sometimes,” “fairly often,” and “almost all of the time”). The COHIP is a recently developed OHRQoL questionnaire and a refinement of the Child Perception Questionnaire. The COHIP questionnaire was used among diverse populations like dental out-patients,11,19,20 orthodontic patients,9,10,23 craniofacial conditions, and orofacial clefts,9,10,21,22,24 cleft lip and palate patients,14-18 and school children.4-8,10,12,24,25 It has been validated in different languages like Amharic,17 Arabic,18 Chinese,22 Dutch,11,13,14,23,24 English,8-10,16 French,7,9 Hindi,25 Italian,19,20 Korean,5,12 Malayalam,8 Persian,4,6 Spanish,9,10,16 and Thai.17 The internal consistency ranged between 0.79 and 0.98 for children and 0.83 and 0.96 for parents or caregivers reported COHIP.

Systematic reviews, reliability generalization (RG), and quality assessment reviews on various OHRQoL instruments conducted on children and adolescents exist in the literature.26-29 However, there were no reports on the pooled estimates Cronbach’s alpha of COHIP. Hence, we aimed to pool the estimates of Cronbach’s alpha of the COHIP questionnaire using RG meta-analysis (MA).

METHODS

This systematic review was reported as per the REliability GEneralization Meta-Analysis (REGEMA) guidelines.30 The protocol was registered in international prospective register of systematic reviews (PROSPERO) (CRD42023439339).

Search Strategy

We have conducted a systematic search of literature from “PubMed,” “Scopus,” “Excerpta Medica Database (EMBASE),” and “Cumulative Index to Nursing and Allied Health Literature (CINAHL)” from inception till 31st May 2023. A combination of search terms and free text were used (“Child Oral Health Impact Profile” or “COHIP” and “Child” or “Children” or “Adolescent”).

Inclusion and Exclusion Criteria

Studies in English and those that reported internal consistency of the COHIP questionnaire were included. Studies that were reported as letters, conference proceedings or abstracts, short communications, and commentaries were excluded. Studies that induced reliability were excluded.

Screening

Search results were imported to Rayyan—a web-based tool (https://rayyan.qcri.org/). Screening of titles and abstracts was performed independently by two individuals (KCP and VK; κ = 0.9). Eligible studies were screened for full text followed by data extraction by two individuals independently (κ = 0.93). Data extraction included authors’ names, year, country, and continent, total number of individuals, age, sex, language and mode of administration, conditions studied, study setting, study design, sampling, Cronbach’s alpha, and the number of items in the questionnaire. Any discrepancies between the reviewers were resolved by the third review author (CD). The quality assessment was done using the “COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias (RoB) tool” for internal consistency.31 Studies were evaluated using questions related to design requirements and statistical methods rated on a four-point scale (“very good,” “adequate,” “doubtful,” or “inadequate”). The overall score for quality assessment was obtained by worst score counts method.

Statistical Analysis

Reliability generalization-meta-analysis was conducted using “Jamovi software” (version 1.2 https://www.jamovi.org).32 Heterogeneity was assessed using I2 and Q statistics. Random effects model with restricted maximum likelihood method was used to pool the untransformed Cronbach’s alpha. Subgroup analysis was performed based on the geographic location, study setting, language, and risk of bias. Egger’s regression test and the funnel plot (x-axis Cronbach’s alpha and y-axis inverse standard error) were used for the evaluation of publication bias. Moderator analysis was conducted using the mixed effects model.

RESULTS

The search yielded a total of 1,274 studies from “PubMed” (n = 433), “Scopus” (n = 350), “EMBASE” (n = 437), and “CINAHL” (n = 54). After the removal of potential duplicates, 626 studies were available for the title and abstract screening. Overall, 106 studies were screened for full text, of which 28 had missing outcomes, 52 had used modified or shorter versions of the COHIP questionnaire, 3 were secondary data of prior publications, and 1 reported Cronbach’s alpha for a pilot study. Overall, 22 studies were included for data extraction that yielded 25 estimates (Fig. 1).

Fig. 1: REGEMA flowchart

The Cronbach’s alpha for all the studies was above the accepted cutoff (>0.7). Eight studies used parent or caregiver COHIP along with a self-reported COHIP questionnaire.10,13,14,17,20-23

Age Distribution

The ages of the participants ranged from 7 to 18 years. Age range was not reported in three reports11,14,19 and mean age was not reported in six reports (Table 1).4,8,13,15,18,25

Table 1: Characteristics of the included studies
Author Continent Sample size Age Male Female Language Number of items Child reported alpha Parent or caregiver reported alpha ROB
Broder et al. NA 524 11.6 (1.6) 255 269 English, French, or Spanish 34 0.91 0
Geels et al. Europe 498 10.99 (0.8) 250 248 Dutch 32 0.87 0
Geels et al. Europe 29 Dutch 38 0.86 0.91 0
Bos et al. Europe 182 12 (1.46) 91 91 Dutch 26 0.79 0.83 0
Ravahi et al. Asia 234 105 129 Persian 34 0.9 1
Bos and Prahl Europe 122 11.9 (2.2) 69 53 Dutch 29 0.81 0.88 0
Broder et al. NA 205 11.8 (2.9) 113 92 English/Spanish 34 0.87 1
Broder et al. NA 108 12.7 (2) 54 54 English/Spanish 34 0.86 1
Broder et al. NA 863 11.9 (3) 467 396 English/Spanish 34 0.88 0.87 1
Ahn et al. Asia 2,236 11.8 (2.5) 1,174 1,062 Korean 34 0.88 0
Asgari et al. Asia 597 14.9 (1.2) 281 316 Persian 34 0.89 0
Eslami et al. Asia 50 24 26 38 0.79 0
Broder et al. NA 1,200 11.6 (3.1) 526 674 English/Spanish 34 0.89 0
Konan et al. Asia 140 11.76 (2.29) 70 70 Thai 34 0.9 0.94 0
El et al. Europe 236 12.02 (0.31) 106 130 French 34 0.88 0
Kragt et al. Europe 243 11.6 (1.58), 12.1 (1.84) 121 122 Dutch 38 0.87 0
Kumar et al. Asia 281 123 158 English, Malayalam 34 0.81 1
Abebe et al. Africa 41 12.37 (2.5) 21 20 Amharic 38 0.979 0.958 0
Lin et al. Asia 120 11.65 71 49 Chinese 31 0.85 0.9 0
Ali et al. Africa 225 138 87 Arabic 34 0.81 0
Payak et al. Asia 300 135 165 Hindi 34 0.81 1
Defabianis et al. Europe 53 11.1 (2) 30 23 Italian 34 0.855 0.859 1
Defabianis et al. Europe 71 11.91 (2.74) 40 31 Italian 34 0.861 1
Defabianis et al. Europe 71 11.87 (2.65) 40 31 Italian 34 0.88 1
Kim et al. Asia 2,010 11.9 (2.5) 1,066 944 Korean 34 0.88 1

Sex Distribution

Only one study has not reported the male and female distribution.13 A total of 5,370 males and 5,240 females were included in this review (Table 1).

Study Setting and Design

All the included studies were cross-sectional and only nine studies were conducted in school or community settings.4-8,10,12,24,25 The cumulative alpha for school or community-based settings was 0.87 (Table 2).

Table 2: Subgroup analysis of the cumulative Cronbach’s alpha
N Estimate SE 95% CI
Continent
 Asia 9 0.863 0.0128 0.84–0.89
 Europe 9 0.857 0.0102 0.837–0.88
 North America 5 0.886 0.00806 0.87–0.9
Language
 Others 18 0.868 0.0106 0.85–0.89
 English 6 0.873 0.0134 0.85–0.9
Study setting
 School or community 9 0.868 0.00990 0.85–0.89
 Others 16 0.867 0.0125 0.84–0.89
Risk of bias
 High 10 0.863 0.00994 0.84–0.88
 Low 15 0.870 0.0127 0.85–0.9

N, number of estimates; SE, standard error; CI, confidence interval

Geographic Location

A total of nine studies were from Asia4-6,8,12,15,17,22,25 and eight were from Europe,7,11,13,14,19,20,23,24 and three were reported from North America.9,10,16 The cumulative alpha for Asian, European, and North American studies were 0.863, 0.857, and 0.886, respectively (Table 2).

Language

The COHIP questionnaire was translated into different languages like Amharic,17 Arabic,18 Chinese,22 Dutch,11,13,14,23,24 English,8-10,16 French,7,9 Hindi,25 Italian,19,20 Korean,5,12 Malayalam,8 Persian,4,6 Spanish,9,10,16 and Thai.17 Studies reported in English showed a pooled estimate of 0.87 (Table 2).

Risk of Bias

A total of 15 studies showed a low RoB (α = 0.87)5-7,9,11,13-18,21-24 and 7 studies (n = 10 estimates) showed a high RoB (α = 0.86) (Tables 1 and 2).4,8,10,12,19,20,25

Meta-analysis and Meta-regression

A total of 25 estimates were obtained from 22 studies which yielded a total sample size of 10,639 (sample size range: 29–2,236). The overall pooled Cronbach’s alpha from the MA was 0.87 (95% CI = 0.85–8.88) with a high heterogeneity (I2 = 95.74%; Q = 496.9) (Fig. 2). Similarly, the parent or the caregiver reported COHIP also showed pooled Cronbach’s alpha of 0.9 (95% CI: 0.87–0.93; n = 8; I2 = 92.58%; Q = 105.46). Meta-regression (using a mixed-method model) evaluated the potential role of moderators on the overall estimates. There was no significant effect of sex (coefficient: −0.12; 95% CI: −0.48 to 0.24), age (coefficient: −0.007; 95% CI: −0.01 to 0.03), language (coefficient: 0.003; 95% CI: −0.04 to 0.04), study setting (coefficient: −0.001; 95% CI: −0.04 to 0.03), risk of bias (coefficient: −0.01; 95% CI: −0.04 to 0.02) and continent (coefficient: −0.04; 95% CI: −0.1 to 0.02) on the overall estimate (Table 3).

Table 3: Meta-regression using mixed model
N Coefficient p-value 95% CI R2
Sex 24 −0.12 0.52 −0.48 to 0.24 0
Mean age 19 0.01 0.53 −0.01 to 0.03 0
Language 24 0.003 0.87 −0.04 to 0.04 0
Study setting 25 0.001 0.95 −0.04 to 0.03 0
Risk of bias 25 −0.01 0.56 −0.04 to 0.02 0
Continent 25 −0.04 0.21 −0.1 to 0.02 2.15

N, number of estimates; CI, confidence interval; Proportion of males

Fig. 2: Forest plot showing the cumulative Cronbach’s alpha

Publication Bias

Publication bias using Egger’s regression test showed statistical significance (coefficient = −3.79; p < 0.001). The funnel plot (x-axis Cronbach’s alpha and y-axis inverse standard error) showed asymmetry (Fig. 3).

Fig. 3: Funnel plot for the assessment of publication bias

DISCUSSION

Cronbach’s alpha is one of the most popular and important tools to assess the internal consistency of a questionnaire and is the most common metric used in psychometric research. It has gained popularity as it readily provides a reliability estimate of a questionnaire that has been administered once,33 ease of availability in the statistical software, and a commonly taught method in academia. RG is a method to combine individual reliability estimates like Cronbach’s alpha and helps us to understand the overall internal consistency of the questionnaire. Hence, in this meta-analysis, we used RG meta-analysis to understand the overall consistency of the COHIP questionnaire.

Meta-analysis showed that the cumulative Cronbach’s alpha values of the self-reported COHIP (0.87) and parent or caregiver-reported COHIP (0.9) were much above the benchmark (0.7)34 and can be graded as good to excellent based on the cutoffs provided by George and Mallery.35 These values were higher than previously reported RG estimates of the C-OIDP questionnaire which is a similar OHRQoL instrument with a smaller number of items.29

The heterogeneity was high and the findings have to be interpreted with caution. Subgroup analysis was used to identify the potential sources of heterogeneity. It was seen that there was not much variation in Cronbach’s alpha between the various subgroups when compared with the overall estimate. Meta-regression was conducted to evaluate the role of potential moderators that might influence the results. It was seen that none of the factors that were evaluated showed significant association.

Many studies were excluded due to the lack of reporting of Cronbach’s alpha (n = 26) which could have influenced the overall estimates. Future psychometric studies should report the findings based on the guidelines that incorporate the reporting of validity and reliability.36-38 Further studies are needed to evaluate the validity, reliability, and temporal stability of COHIP questionnaires among diverse populations, oral conditions, and languages to better understand the psychometric properties. One of the limitations of this review was the exclusion of non-English articles. Within the limits of this review, the COHIP questionnaire showed good internal consistency and can be used in research and practice among children.

Clinical Significance

The estimates of this meta-analysis can be helpful for researchers in selecting the choice of instruments, sample size calculations, and sources of measurement errors. It should be noted that Cronbach’s alpha although popular may not be the best tool to identify measurement errors, transient errors, or rater-specific errors. Also, it is encouraged that authors report Cronbach’s alpha along with 95% confidence intervals or standard errors of their study rather than citing from previous research. The current estimate can be used as a benchmark for the reliability of the COHIP questionnaire for comparing estimates obtained in future studies. If the reliability estimates fall outside the confidence intervals, then there is a need to examine the interitem correlations and relevant moderation.

AUTHOR CONTRIBUTIONS

Conceptualization: PKC, CD, SK, and NK; Screening: PKC, VSK, and SK; Data extraction: PKC and NK; Data analysis: PKC, CD; Initial draft: PKC, NK, SK, CD, and VSK; Final draft: PKC and CD.

REFERENCES

1. Schütte U, Heydecke G. Oral health related quality of life. Encyclopedia of Public Health. 2008. pp. 1052–1055.

2. Bennadi D, Reddy CVK. Oral health related quality of life. J Int Soc Prev Community Dent 2013;3(1):1.

3. Genderson MW, Sischo L, Markowitz K, et al. An overview of children’s oral health-related quality of life assessment: from scale development to measuring outcomes. Caries Res 2013;47(1):13.

4. Ravahi V, Mokhatry A MM, Shahriari S, et al. Comparison of the COHIP and OHIP-14 as measures of the oral health-related quality of life of adolescents. Community Dent Health 2011;28(1):82–88.

5. Ahn YS, Kim HY, Hong SM, et al. Validation of a Korean version of the Child Oral Health Impact Profile (COHIP) among 8- to 15-year-old school children. Int J Paediatr Dent 2012;22(4):292–301.

6. Asgari I, Ahmady AE, Broder H, et al. Assessing the oral health-related quality of life in Iranian adolescents: validity of the Persian version of the Child Oral Health Impact Profile (COHIP). Oral Health Prev Dent 2013;11(2):147–154.

7. Osta NE, Pichot H, Soulier-Peigue D, et al. Validation of the Child Oral Health Impact Profile (COHIP) French questionnaire among 12 years-old children in New Caledonia. Health Qual Life Outcomes 2015;13(1):176.

8. Kumar VS, Karuveettil V, Joseph J, et al. Association of dental caries and oral health impact profile in 12-year-old school children: a cross-sectional study. J Clin Diagnostic Res 2018;12(9):ZC12–ZC15.

9. Broder HL, Wilson-Genderson M. Reliability and convergent and discriminant validity of the Child Oral Health Impact Profile (COHIP child’s version). Community Dent Oral Epidemiol 2007;35(Suppl 1):20–31.

10. Broder HL, Wilson-Genderson M, Sischo L. Reliability and validity testing for the child oral health impact profile-reduced (COHIP-SF 19). J Public Health Dent 2012;72(4):302.

11. Kragt L, Tiemeier H, Wolvius EB, et al. Measuring oral health-related quality of life in orthodontic patients with a short version of the Child Oral Health Impact Profile (COHIP). J Public Health Dent 2016;76(2):105–112.

12. Kim HY, Kim J, Hong SM. Do the oral health-related quality of life and the magnitude of its association with malocclusion differ by developmental stages of schoolchildren? Am J Orthod Dentofacial Orthop 2023;163(6):777–785.

13. Geels LM, Kieffer JM, Hoogstraten J, et al. Oral health-related quality of life of children with craniofacial conditions. Cleft Palate Craniofac J 2008;45(5):461–467.

14. Bos A, Prahl C. Oral health-related quality of life in Dutch children with cleft lip and/or palate. Angle Orthod 2011;81(5):865–871.

15. Eslami N, Majidi MR, Aliakbarian M, et al. Oral health-related quality of life in children with cleft lip and palate. J Craniofac Surg 2013;24(4):e340–e343.

16. Broder HL, Norman RG, Sischo L, et al. Evaluation of the similarities and differences in response patterns to the Pediatric Quality of Life Inventory and the Child Oral Health Impact Scores among youth with cleft. Qual Life Res 2014;23(1):339–347.

17. Konan P, Manosudprasit M, Pisek P, et al. Oral health-related quality of life in children and young adolescent orthodontic cleft patients. J Med Assoc Thai 2015;98:S84–S91.

18. Ali MA, Nasir AF, Abass SK. Oral health-related quality of life among Sudanese children treated for cleft lip and palate. Cleft Palate Craniofac J 2021;58(11):1405–1411.

19. Defabianis P, Cogo C, Massa S, et al. Oral-health-related quality of life among non-syndromic school-age children with orofacial clefts: results from a cross-sectional study in northern Italy. Child (Basel) 2022;9(7):1098.

20. Defabianis P, Ninivaggi R, Romano F. Influence of cleft lip and palate on oral health-related quality of life in northern Italy: exploring both the children’s and caregivers’ perspectives. Child (Basel) 2022;9(12):1986.

21. Abebe ME, Deressa W, Oladugba V, et al. Oral health-related quality of life of children born with orofacial clefts in Ethiopia and their parents. Cleft Palate Craniofac J 2018;55(8):1153–1157.

22. Lin J, Fang X, Ha P, et al. Oral health-related quality of life in Chinese children with orofacial cleft. Cleft Palate Craniofac J 2020;57(8):931–937.

23. Bos A, Hoogstraten J, Zentner A. Perceptions of Dutch orthodontic patients and their parents on oral health-related quality of life. Angle Orthod 2010;80(2):367–372.

24. Geels LM, Hoogstraten J, Prahl-Andersen B. Confirmative factor analysis of the dimensions of the Child Oral Health Impact Profile (Dutch version). Eur J Oral Sci 2008;116(2):148–152.

25. Payak AS, Singh A, Bhadouria P, et al. Association of dental caries and oral health impact profile in children: a cross-sectional study. NeuroQuantology 2022;20(15):5801–5812.

26. Thomson WM, Broder HL. Oral-health-related quality of life in children and adolescents. Pediatr Clin North Am 2018;65(5):1073–1084.

27. Gilchrist F, Rodd H, Deery C, et al. Assessment of the quality of measures of child oral health-related quality of life. BMC Oral Health 2014;14(1):40.

28. Hettiarachchi RM, Kularatna S, Byrnes J, et al. Pediatric quality of life instruments in oral health research: a systematic review. Value Heal 2019;22(1):129–135.

29. Pentapati KC, Yeturu SK, Siddiq H. A reliability generalization meta-analysis of child oral impacts on daily performances (C-OIDP) questionnaire. J Oral Biol Craniofacial Res 2020;10(4):776–781.

30. Sánchez-Meca J, Marín-Martínez F, López-López JA, et al. Improving the reporting quality of reliability generalization meta-analyses: the REGEMA checklist. Res Synth Methods 2021;J12(4):516–536.

31. Mokkink LB, de Vet HCW, Prinsen CAC, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 2018;27(5):1171–1179.

32. Viechtbauer W. Conducting meta-analyses in R with the meta for package. J Stat Softw 2010;36(3):1–48.

33. Bruce T. Score reliability: contemporary thinking on reliability issues. Sage Publications; 2002.

34. Jum CN. Psychometric theory, 2nd edition. New York, McGraw-Hill: 1978.

35. George D, Mallery P. SPSS for Windows Step by Step: A Simple Guide and Reference, 16.0 Update, 9th edition. 2008.

36. Burns KEA, Duffett M, Kho ME, et al. A guide for the design and conduct of self-administered surveys of clinicians. Can Med Assoc J 2008;179(3):245–252.

37. Draugalis JR, Coons SJ, Plaza CM. Best practices for survey research reports: a synopsis for authors and reviewers. Am J Pharm Educ 2008;72(1):11.

38. Kelley K. Good practice in the conduct and reporting of survey research. Int J Qual Heal Care 2003;15(3):261–266.

________________________
© The Author(s). 2024 Open Access. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.