Sage Journals: Discover world-class research

Abstract

Simulations concerning the distributional assumptions of coefficient alpha are contradictory. To provide a more principled theoretical framework, this article relies on the Fréchet–Hoeffding bounds, in order to showcase that the distribution of the items play a role on the estimation of correlations and covariances. More specifically, these bounds restrict the theoretical correlation range [−1, 1] such that certain correlation structures may be unfeasible. The direct implication of this result is that coefficient alpha is bounded above depending on the shape of the distributions. A general form of the Fréchet–Hoeffding bounds is derived for discrete random variables. R code and a user-friendly shiny web application are also provided so that researchers can calculate the bounds on their data.

Keywords

reliability theory coefficient alpha item distribution classical test theory

Get full access to this article

View all access options for this article.

References

Astivia

O. L. O.

Zumbo

B. D.

(2017). Population models and simulation methods: the case of the Spearman rank correlation. British Journal of Mathematical and Statistical Psychology, 70, 347-367. https://doi.org/10.1111/bmsp.12085

Astivia

O. L. O.

Zumbo

B. D.

(2018). On the solution multiplicity of the Fleishman method and its impact in simulation studies. British Journal of Mathematical and Statistical Psychology, 71(3), 437-458. https://doi.org/10.1111/bmsp.12126

Astivia

O. L. O.

Zumbo

B. D.

(2019). A note on the solution multiplicity of the Vale–Maurelli intermediate correlation equation. Journal of Educational and Behavioral Statistics, 44(2), 127-143. https://doi.org/10.3102/1076998618803381

Barchard

K. A.

Hakstian

A. R.

(1997). The robustness of confidence intervals for coefficient alpha under violation of the assumption of essential parallelism. Multivariate Behavioral Research, 32(2), 169-191. https://doi.org/10.1207/s15327906mbr32024

Baron

R. M.

Kenny

D. A.

(1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182. https://doi.org/10.1016/j.jom.2018.04.003

Bay

K. S.

(1973). The effect of non normality on the sampling distribution and standard error of reliability coefficient estimates under an analysis of variance model. British Journal of Mathematical and Statistical Psychology, 26(1), 45-57. https://doi.org/10.1111/j.2044-8317.1973.tb00505.x

Bishop

P. A.

Herron

R. L.

(2015). Use and misuse of the Likert item responses and other ordinal measures. International Journal of Exercise Science, 8(3), 297-302.

Bollen

(1989). Structural equations with latent variables. Wiley. https://doi.org/10.1002/9781118619179

Brogden

H. E.

(1946). The effect of bias due to difficulty factors in product-moment item intercorrelations on the accuracy of estimation of reliability by the Kuder–Richardson Formula Number 20. Educational and Psychological Measurement, 6(4), 517-520. https://doi.org/10.1177/001316444600600408

10.

Carifio

Perla

(2007). Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences, 3(3), 106-116. https://doi.org/10.3844/jssp.2007.106.116

11.

Chalmers

R. P.

(2018). On misconceptions and the limited usefulness of ordinal alpha. Educational and Psychological Measurement, 78(6), 1056-1071. https://doi.org/10.1177/0013164417727036

12.

Cohen

(1988). Statistical power analysis for the social sciences. Lawrence Erlbaum.

13.

Davenport

E. C.

Jr. El-Sanhurry

N. A.

(1991). Phi/phimax: Review and synthesis. Educational and Psychological Measurement, 51(4), 821-828. https://doi.org/10.1177/001316449105100403

14.

Demirtas

(2016). A note on the relationship between the phi coefficient and the tetrachoric correlation under nonnormal underlying distributions. The American Statistician, 70(2), 143-148. https://doi.org/10.1080/00031305.2015.1077161

15.

Demirtas

Hedeker

(2011). A practical way for computing approximate lower and upper correlation bounds. The American Statistician, 65(2), 104-109. https://doi.org/10.1198/tast.2011.10090

16.

Feldt

L. S.

(1965). The approximate sampling distribution of Kuder–Richardson reliability coefficient twenty. Psychometrika, 30(3), 357-370. https://doi.org/10.1007/BF02289499

17.

Ferguson

G. A.

(1941). The factorial interpretation of test difficulty. Psychometrika, 6(5), 323-329. https://doi.org/10.1007/BF02288588

18.

Fleishman

A. I.

(1978). A method for simulating non-normal distributions. Psychometrika, 43(4), 521-532. https://doi.org/10.1007/BF02293811

19.

Fréchet

(1951). Sur les tableaux de corrélation dont les marges sont donnés. Annales de l’Université de Lyon Section A:Sciences mathématiques et astronomie [On correlation matrices with fixed margins. Annals of the University of Lyon Section A: Mathematical sciences and astronomy], 14, 53-77.

20.

Greer

Dunlap

W. P.

Beatty

G. O.

(2003). A Monte Carlo evaluation of the tetrachoric correlation coefficient. Educational and Psychological Measurement, 63(6), 931-950. https://doi.org/10.1177/0013164403251318

21.

Guilford

J. P.

(1965). The minimal phi coefficient and the maximal phi. Educational and Psychological Measurement, 25(1), 3-8. https://doi.org/10.1177/001316446502500101

22.

Headrick

T. C.

(2002). Fast fifth-order polynomial transforms for generating univariate and multivariate non-normal distributions. Computational Statistics and Data Analysis, 40(4), 685-711. https://doi.org/10.1016/S0167-9473(02)00072-5

23.

Hoeffding

(1940). Scale–invariant correlation theory. In Fisher

N. I.

Sen

P. K.

(Eds.), The collected works of Wassily Hoeffding (pp. 57-107). Springer. https://doi.org/10.1007/978-1-4612-0865-5_4

24.

Hogan

T. P.

Benjamin

Brezinski

K. L.

(2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60(4), 523-531. https://doi.org/10.1177/00131640021970691

25.

Joe

(2014). Dependence modeling with copulas. CRC Press. https://doi.org/10.1201/b17116

26.

Jöreskog

K. G.

(1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109-133. https://doi.org/10.1007/BF02291393

27.

Kelley

T. L.

(1927). Interpretation of educational measurements. World Book.

28.

Kristof

(1963). The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts. Psychometrika, 28(3), 221-238. https://doi.org/10.1007/BF02289571

29.

Levin

(1972). The occurrence of an increase in correlation by restriction of range. Psychometrika, 37(1), 93-97. https://doi.org/10.1007/BF02291414

30.

Liddell

T. M.

Kruschke

J. K.

(2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology, 79(2018), 328-348. https://doi.org/10.1016/j.jesp.2018.08.009

31.

Lord

F. M.

Novick

M. R.

(1968). Statistical theories of mental test scores. Addison-Wesley.

32.

Markon

K. E.

(2017). A generalized definition of reliability based on Lindley information. https://doi.org/10.31234/osf.io/vgpfb

33.

McNeish

(2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412-433. https://doi.org/10.1037/met0000144

34.

Nelsen

R. B.

(2010). An introduction to copulas. Springer Science & Business Media.

35.

Novick

M. R.

Lewis

(1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1-13. https://doi.org/10.1007/BF02289400

36.

Peterson

R. A.

(1994). A meta-analysis of Cronbach’s coefficient alpha. Journal of Consumer Research, 21(2), 381-391. https://doi.org/10.1086/209405

37.

Raykov

(2012). Scale development using structural equation modeling. In Hoyle

(Ed.), Handbook of structural equation modeling (pp. 472-492). Guilford Press.

38.

Raykov

Marcoulides

G. A.

(2019). Thanks coefficient alpha, we still need you!. Educational and Psychological Measurement, 79(1), 200-210. https://doi.org/10.1177/0013164417725127

39.

Rhemtulla

Brosseau-Liard

P. E.

Savalei

(2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373. https://doi.org/10.1037/a0029315

40.

Sheng

(2012). Is coefficient alpha robust to non-normal data? Frontiers in Psychology, 3, Article 34. https://doi.org/10.3389/fpsyg.2012.00034

41.

Shultz

S. G.

(1993). A Monte Carlo study of the robustness of coefficient alpha [Unpublished master’s thesis]. University of Ottawa.

42.

Sun

Chou

C. P.

Stacy

A. W.

Unger

Gallaher

(2007). SAS and SPSS macros to calculate standardized Cronbach’s alpha using the upper bound of the phi coefficient for dichotomous items. Behavior Research Methods, 39(1), 71-81. https://doi.org/10.3758/BF03192845

43.

Zimmerman

D. W.

Zumbo

B. D.

Lalonde

(1993). Coefficient alpha as an estimate of test reliability under violation of two assumptions. Educational and Psychological Measurement, 53(1), 33-49. https://doi.org/10.1177/0013164493053001003

44.

Zumbo

B. D.

Gadermann

A. M.

Zeisser

(2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21-29. https://doi.org/10.22237/jmasm/1177992180

45.

Zumbo

B. D.

Kroc

(2019). A measurement is a choice and Stevens’ scales of measurement do not help make it: A response to Chalmers. Educational and Psychological Measurement, 79(6), 1184-1197. https://doi.org/10.1177/0013164419844305

The Role of Item Distributions on Reliability Estimation: The Case of Cronbach’s Coefficient Alpha

Abstract

Keywords

Get full access to this article

References