Cronbach's alpha (Cronbach's \alpha) or coefficient alpha (coefficient \alpha), is a reliability coefficient and a measure of the internal consistency of tests and measures.[1][2] It was devised by the American psychometrician Lee Cronbach. Today it enjoys such wide-spread usage that numerous studies warn against using Cronbach's alpha uncritically.[3][4][5][6][7]

History

In his initial 1951 publication, Lee Cronbach described the coefficient as Coefficient alpha[1] and included an additional derivation.[8] Coefficient alpha had been used implicitly in previous studies,[9][10][11][12] but his interpretation was thought to be more intuitively attractive relative to previous studies and it became quite popular.[13]

  • In 1967, Melvin Novick and Charles Lewis proved that it was equal to reliability if the true scores[i] of the compared tests or measures vary by a constant, which is independent of the people measured. In this case, the tests or measurements were said to be "essentially tau-equivalent."[14]
  • In 1978, Cronbach asserted that the reason the initial 1951 publication was widely cited was "mostly because [he] put a brand name on a common-place coefficient."[2]: 263 He explained that he had originally planned to name other types of reliability coefficients, such as those used in inter-rater reliability and test-retest reliability, after consecutive Greek letters (i.e., \beta, \gamma, etc.), but later changed his mind.
  • Later, in 2004, Cronbach and Richard Shavelson encouraged readers to use generalizability theory rather than \alpha. Cronbach opposed the use of the name "Cronbach's alpha" and explicitly denied the existence of studies that had published the general formula of KR-20 before Cronbach's 1951 publication of the same name.[7]

Prerequisites for using Cronbach's alpha

To use Cronbach's alpha as an accurate estimate of reliability, the following conditions must be met:[15][16]

  1. The "parts" (i.e. items, test parts, etc.) must be essentially tau-equivalent;
  2. Errors in the measurements are independent.

However, under the definition of CTT, the errors are defined to be independent.

This is often a source of confusion for users who might consider some aspect of the testing process to be an "error" (rater biases, examinee collusion, self-report faking). Anything that increases the covariance among the parts will contribute to greater true score variance. Under such circumstances, alpha is likely to over-estimate the reliability intended by the user.[16][3]

Formula and calculation

Reliability can be defined as one minus the error score variance divided by the observed score variance:

\rho_{XX'} = \left(1 - {\sigma^2_E \over \sigma^2_X} \right)

Cronbach's alpha is best understood as a direct estimate of this definitional formula with error score variance estimated as the sum of the variances of each "part" (e.g., items or testlets):[6]

\alpha = {k \over k-1 } \left(1 - {\sum_{i=1}^k \sigma^2_{y_i} \over \sigma_X^2} \right)

where:

  • k represents the number of "parts" (items, test parts, etc.) in the measure;
  • the k/(k-1) term causes alpha to be an unbiased estimate of reliability when the parts are parallel or essentially tau equivalent;
  • \sigma_{y_i}^2 the variance associated with each part i; and
  • \sigma_X^2 the observed score variance (the variance associated with the total test scores).

The reason that the sum of the individual part variances estimates the error score variance is because \sigma_X^2 = \sigma_T^2 + \sigma_E^2 and the variance of a composite is equal to twice the sum of all covariances of the parts plus the sum of the variances of the parts: \sigma^2_{X} = \sum \sum \sigma_{y_i,y_j} + \sum \sigma^2_{y_i}. Therefore \sum \sum \sigma_{y_i,y_j} estimates \sigma_T^2 and \sum \sigma^2_{y_i} estimates \sigma_E^2. It is much easier to compute alpha by summing the part variances (to estimate error score variance) than adding up all the unique part covariances (to estimate true score variance) .

Alternatively, alpha can be calculated through the following formula:[17]

\alpha = {k \bar c \over \bar v + (k - 1) \bar c}

where:

  • \bar v represents the average variance
  • \bar c represents the average inter-item covariance (or the average covariance of "parts").

Common misconceptions

Application of Cronbach's alpha is not always straightforward and can give rise to common misconceptions.

A high value of Cronbach's alpha indicates homogeneity between the items

Many textbooks refer to \alpha as an indicator of homogeneity[18] between items. This misconception stems from the inaccurate explanation of Cronbach (1951)[8] that high \alpha values show homogeneity between the items. Homogeneity is a term that is rarely used in modern literature, and related studies interpret the term as referring to uni-dimensionality. Several studies have provided proofs or counterexamples that high \alpha values do not indicate uni-dimensionality.[19][20][21][22][23] See counterexamples below.

Uni-dimensional data
X_1X_2X_3X_4X_5X_6
X_11033333
X_23103333
X_33310333
X_43331033
X_53333103
X_63333310

\alpha=0.72 in the uni-dimensional data above.

Multidimensional data
X_1X_2X_3X_4X_5X_6
X_11066111
X_26106111
X_36610111
X_41111066
X_51116106
X_61116610

\alpha=0.72 in the multidimensional data above.

Multidimensional data with extremely high reliability
X_1X_2X_3X_4X_5X_6
X_11099888
X_29109888
X_39910888
X_48881099
X_58889109
X_68889910

The above data have \alpha=0.9692, but are multidimensional.

Uni-dimensional data with unacceptably low reliability
X_1X_2X_3X_4X_5X_6
X_11011111
X_21101111
X_31110111
X_41111011
X_51111101
X_61111110

The above data have \alpha=0.4, but are uni-dimensional.

Uni-dimensionality is a prerequisite for \alpha. One should check uni-dimensionality before calculating \alpha rather than calculating \alpha to check uni-dimensionality.

A high value of Cronbach's alpha indicates internal consistency

The term "internal consistency" is commonly used in the reliability literature, but its meaning is not clearly defined. The term is sometimes used to refer to a certain kind of reliability (e.g., internal consistency reliability), but it is unclear exactly which reliability coefficients are included here, in addition to \alpha. Cronbach (1951)[8] used the term in several senses without an explicit definition. Cortina (1993)[19] showed that \alpha is not an indicator of any of these.

Removing items using "alpha if item deleted" always increases reliability

Most psychometric software will produce a column labeled "alpha if item deleted" which is the coefficient alpha that would be obtained if an item were to be dropped. For good items, this value is lower than the current coefficient alpha for the whole scale. But for some weak or bad items, the "alpha if item deleted" value shows an increase over the current coefficient alpha for the whole scale.

Removing an item using "alpha if item deleted" may result in 'alpha inflation,' where sample-level reliability is reported to be higher than population-level reliability.[24] It may also reduce population-level reliability.[25] The elimination of less-reliable items should be based not only on a statistical basis but also on a theoretical and logical basis. It is also recommended that the whole sample be divided into two and cross-validated.[24]

Notes

  1. ^ The true score is the difference between the score observed during the test or measurement and the error in that observation. See classical test theory for further information.

References

  1. ^ Cronbach, Lee J. (1951). "Coefficient alpha and the internal structure of tests". Psychometrika. 16 (3): 297–334. Springer Science and Business Media LLC. doi:10.1007/bf02310555. hdl:10983/2196. S2CID 13820448
  2. ^ Cronbach, L. J. (1978). "Citation Classics". Current Contents. 13: 263. Archived 2022-01-20 at the Wayback Machine. Retrieved 2021-03-22.
  3. ^ Sijtsma, K. (2009). "On the use, the misuse, and the very limited usefulness of Cronbach's alpha". Psychometrika. 74 (1): 107–120. doi:10.1007/s11336-008-9101-0. PMC 2792363. PMID 20037639
  4. ^ Green, S. B. & Yang, Y. (2009). "Commentary on coefficient alpha: A cautionary tale". Psychometrika. 74 (1): 121–135. doi:10.1007/s11336-008-9098-4. S2CID 122718353
  5. ^ Revelle, W. & Zinbarg, R. E. (2009). "Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma". Psychometrika. 74 (1): 145–154. doi:10.1007/s11336-008-9102-z. S2CID 5864489
  6. ^ Raykov, T. & Marcoulides, G. A. (2017). "Thanks coefficient alpha, we still need you!". Educational and Psychological Measurement. 79 (1): 200–210. doi:10.1177/0013164417725127. PMC 6318747. PMID 30636788
  7. ^ Cronbach, L. J. & Shavelson, R. J. (2004). "My Current Thoughts on Coefficient Alpha and Successor Procedures". Educational and Psychological Measurement. 64 (3): 391–418. doi:10.1177/0013164404266386. S2CID 51846704
  8. ^ Cronbach, L.J. (1951). "Coefficient alpha and the internal structure of tests". Psychometrika. 16 (3): 297–334. doi:10.1007/BF02310555. hdl:10983/2196. S2CID 13820448
  9. ^ Hoyt, C. (1941). "Test reliability estimated by analysis of variance". Psychometrika. 6 (3): 153–160. doi:10.1007/BF02289270. S2CID 122361318
  10. ^ Guttman, L. (1945). "A basis for analyzing test-retest reliability". Psychometrika. 10 (4): 255–282. doi:10.1007/BF02288892. PMID 21007983. S2CID 17220260
  11. ^ Jackson, R. W. B. & Ferguson, G. A. (1941). "Studies on the reliability of tests". University of Toronto Department of Educational Research Bulletin. 12: 132.
  12. ^ Gulliksen, H. (1950). Theory of mental tests. Wiley. doi:10.1037/13240-000
  13. ^ Cronbach, Lee (1978). "Citation Classics". Current Contents. 13 (8). Archived 2022-10-22 at the Wayback Machine. Retrieved 2022-10-21.
  14. ^ Novick, M. R. & Lewis, C. (1967). "Coefficient alpha and the reliability of composite measurements". Psychometrika. 32 (1): 1–13. doi:10.1007/BF02289400. PMID 5232569. S2CID 186226312
  15. ^ Spiliotopoulou, Georgia (2009). "Reliability reconsidered: Cronbach's alpha and paediatric assessment in occupational therapy". Australian Occupational Therapy Journal. 56 (3): 150–155. doi:10.1111/j.1440-1630.2009.00785.x. PMID 20854508. Archived 2022-10-21 at the Wayback Machine. Retrieved 2022-10-21.
  16. ^ Cortina, Jose M. (1993). "What is coefficient alpha? An examination of theory and applications.". Journal of Applied Psychology. 78 (1): 98–104. doi:10.1037/0021-9010.78.1.98. ISSN 1939-1854. Archived 2023-08-13 at the Wayback Machine. Retrieved 2022-10-21.
  17. ^ DATAtab (October 27, 2021). Cronbach's Alpha (Simply explained). YouTube. pp. 4:08. Retrieved 2023-08-01.
  18. ^ "APA Dictionary of Psychology". dictionary.apa.org. Archived 2019-07-31 at the Wayback Machine. Retrieved 2023-02-20.
  19. ^ Cortina, J. M. (1993). "What is coefficient alpha? An examination of theory and applications". Journal of Applied Psychology. 78 (1): 98–104. doi:10.1037/0021-9010.78.1.98
  20. ^ Green, S. B.; Lissitz, R. W.; Mulaik, S. A. (1977). "Limitations of coefficient alpha as an Index of test unidimensionality". Educational and Psychological Measurement. 37 (4): 827–838. doi:10.1177/001316447703700403. S2CID 122986180
  21. ^ McDonald, R. P. (1981). "The dimensionality of tests and items". The British Journal of Mathematical and Statistical Psychology. 34 (1): 100–117. doi:10.1111/j.2044-8317.1981.tb00621.x
  22. ^ Schmitt, N. (1996). "Uses and abuses of coefficient alpha". Psychological Assessment. 8 (4): 350–3. doi:10.1037/1040-3590.8.4.350
  23. ^ Ten Berge, J. M. F. & Sočan, G. (2004). "The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality". Psychometrika. 69 (4): 613–625. doi:10.1007/BF02289858. S2CID 122674001
  24. ^ Kopalle, P. K. & Lehmann, D. R. (1997). "Alpha inflation? The impact of eliminating scale items on Cronbach's alpha". Organizational Behavior and Human Decision Processes. 70 (3): 189–197. doi:10.1006/obhd.1997.2702
  25. ^ Raykov, T. (2007). "Reliability if deleted, not 'alpha if deleted': Evaluation of scale reliability following component deletion". The British Journal of Mathematical and Statistical Psychology. 60 (2): 201–216. doi:10.1348/000711006X115954. PMID 17971267