The Distress Tolerance Scale (DTS) is a self-report measure of perceived capacity to withstand aversive emotions. Initial factor analysis of this measure suggested a structure comprising one higher-order factor and four lower-order domain-specific factors. However, there is limited evidence in support of the DTS’s purported multidimensionality, and despite use of the DTS subscales, research has yet to assess their incremental utility. The current investigation sought to rectify the paucity of evidence in support of the DTS’s factor structure and independent use of DTS subscales via bifactor analysis. In the present study (N = 826 community adults), a bifactor model of the DTS provided the best fit to the data. However, an examination of statistical indices associated with bifactor modeling, as well as results from an examination of incremental utility, suggest that the domain-specific factors are largely redundant with the general factor and do not provide incremental utility in predicting relevant clinical constructs beyond the general factor. Measurement invariance between sexes was confirmed. Taken together, results support use of a DTS total score, but not subscale scores.
AbdiH.WilliamsL. J. (2010). Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2, 433-459.
2.
AkaikeH. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.
3.
AmeralV.Palm ReedK. M.CameronA.ArmstrongJ. L. (2014). What are measures of distress tolerance really capturing? A mixed methods analysis. Psychology of Consciousness: Theory, Research, and Practice, 1, 357-369.
4.
AnestisM. D.BaggeC. L.TullM. T.JoinerT. E. (2011). Clarifying the role of emotion dysregulation in the interpersonal-psychological theory of suicidal behavior in an undergraduate sample. Journal of Psychiatric Research, 45, 603-611.
5.
AnestisM. D.LavenderJ. M.Marshall-BerenzE. C.GratzK. L.TullM. T.JoinerT. E. (2012). Evaluating distress tolerance measures: Interrelations and associations with impulsive behaviors. Cognitive Therapy and Research, 36, 593-602.
6.
AntonyM. M.BielingP. J.CoxB. J.EnnsM. W.SwinsonR. P. (1998). Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychological Assessment, 10, 176-181.
7.
BardeenJ. R.FergusT. A. (2016). Emotional distress intolerance, experiential avoidance, and anxiety sensitivity: The buffering effect of attentional control on associations with posttraumatic stress symptoms. Journal of Psychopathology and Behavioral Assessment, 38, 320-329.
8.
BardeenJ. R.FergusT. A.HannanS. M.OrcuttH. K. (2016). Addressing psychometric limitations of the Difficulties in Emotion Regulation Scale through item modification. Journal of Personality Assessment, 98, 298-309.
9.
BardeenJ. R.FergusT. A.OrcuttH. K. (2013a). Experiential avoidance as a moderator of the relationship between anxiety sensitivity and perceived stress. Behavior Therapy, 44, 459-469.
10.
BardeenJ. R.FergusT. A.OrcuttH. K. (2013b). Testing a hierarchical model of distress tolerance. Journal of Psychopathology and Behavioral Assessment, 35, 495-505.
11.
BardeenJ. R.FergusT. A.OrcuttH. K. (2014). The moderating role of experiential avoidance in the prospective relationship between anxiety sensitivity and anxiety. Cognitive Therapy and Research, 38, 465-471.
12.
BardeenJ. R.KumpulaM. J.OrcuttH. K. (2013). Emotion regulation difficulties as a prospective predictor of posttraumatic stress symptoms following a mass shooting. Journal of Anxiety Disorders, 27, 188-196.
13.
BardeenJ. R.MichelJ. S. (2017). The buffering effect of religiosity on the relationship between intolerance of uncertainty and depressive symptoms. Psychology of Religion and Spirituality, 9, 90-95.
14.
BehrendT. S.SharekD. J.MeadeA. W.WiebeE. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43, 800-813.
15.
BernsteinA.MarshallE. C.ZvolenskyM. J. (2011). Multi-method evaluation of distress tolerance measures and construct(s): Concurrent relations to mood and anxiety psychopathology and quality of life. Journal of Experimental Psychopathology, 2, 386-399.
16.
BernsteinA.VujanovicA. A.LeyroT.ZvolenskyM. J. (2011). Distress tolerance: Research synthesis and future directions. In ZvolenskyM. J.BernsteinA.VujanovicA. (Eds.), Distress tolerance: Theory, research, and clinical applications (pp. 263-280). New York, NY: Guilford Press.
17.
BonifayW.LaneS. P.ReiseS. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5, 184-186.
18.
BrownT. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: Guilford Press.
19.
BrownT. A.MooreM. T. (2012). Confirmatory factor analysis. In HoyleR. H. (Ed.), Handbook of structural equation modeling (pp. 361-379). New York, NY: Guilford Press.
20.
BuhrmesterM.KwangT.GoslingS. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science, 6, 3-5.
21.
BurnhamK. P.AndersonD. R.HuyvaertK. P. (2011). AIC model selection and multimodel inference in behavioral ecology: Some background, observations, and comparisons. Behavioral Ecology and Sociobiology, 65, 23-35.
22.
ChandlerJ.ShapiroD. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53-81.
23.
CheungC. W.RensvoldR. B. (2002). Evaluating good- ness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
24.
CorstorphineE.MountfordV.TomlinsonS.WallerG.MeyerC. (2007). Distress tolerance in the eating disorders. Eating Behaviors, 8, 91-97.
25.
CougleJ. R.BernsteinA.ZvolenskyM. J.VujanovicA. A.MacateeR. J. (2013). Validation of self-report measures of emotional and physical distress tolerance. Journal of Psychopathology and Behavioral Assessment, 35, 76-84.
26.
DennhardtA. A.MurphyJ. G. (2011). Associations between depression, distress tolerance, delay discounting, and alcohol-related problems in European American and African American college students. Psychology of Addictive Behaviors, 25, 595-604.
27.
DueberD. M. (2016, December). Bifactor indices calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models. Retrieved from http://sites.education.uky.edu/apslab/resources
28.
EbesutaniC.McLeishA. C.LubertoC. M.YoungJ.MaackD. J. (2014). A bifactor model of anxiety sensitivity: Analysis of the Anxiety Sensitivity Index-3. Journal of Psychopathology and Behavioral Assessment, 36, 452-464.
29.
EbesutaniC.SmithA.BernsteinA.ChorpitaB. F.Higa-McMillanC.NakamuraB. (2011). A bifactor model of negative affectivity: Fear and distress components among younger and older youth. Psychological Assessment, 23, 679-691.
30.
GorsuchR. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
31.
GrahamJ. W.SchaferJ. L. (1999). On the performance of multiple imputation for multivariate data with small sample sizes. In HoyleR. (Ed.), Statistical strategies for small sample research (pp. 1-29). London, England: Sage.
32.
HancockG. R.MuellerR. O. (2001). Rethinking construct reliability within latent variable systems. In CudeckR.Du ToitS.SorbomD. (Eds.), Structural equation modeling: Present and future (pp. 195-216). Lincolnwood, IL: Scientific Software International.
33.
HenryJ. D.CrawfordJ. R. (2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227-239.
34.
HsuS. H.CollinsS. E.MarlattG. A. (2013). Examining psychometric properties of distress tolerance and its moderation of mindfulness-based relapse prevention effects on alcohol and other drug use outcomes. Addictive Behaviors, 38, 1852-1858.
35.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
KlineR. B. (2016). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford Press.
38.
LeyroT.BernsteinA.VujanovicA. A.McLeishA.ZvolenskyM. J. (2011). Distress Tolerance Scale: A confirmatory factor analysis among daily cigarette smokers. Journal of Psychopathology and Behavioral Assessment, 33, 47-57.
39.
LeyroT. M.ZvolenskyM. J.BernsteinA. (2010). Distress tolerance and psychopathological symptoms and disorders: A review of the empirical literature among adults. Psychological Bulletin, 136, 576-600.
40.
LinehanM. (1993). Cognitive-behavioral treatment of borderline personality disorder. New York, NY: Guilford Press.
41.
LovibondS. H.LovibondP. F. (1995). Manuals for the Depression Anxiety Stress Scales (2nd ed.). Sydney, New South Wales, Australia: Psychology Foundation.
42.
McHughR. K.KertzS. J.WeissR. B.Baskin-SommersA. R.HearonB. A.BjörgvinssonT. (2014). Changes in distress intolerance and treatment outcome in a partial hospital setting. Behavior Therapy, 45, 232-240.
43.
McHughR. K.OttoM. W. (2012). Refining the measurement of distress tolerance. Behavior Therapy, 43, 641-651.
44.
MuthénB.KaplanD.HollisM. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431-462.
45.
MuthénL. K.MuthénB. O. (2015). MPlus (Version 7.4) [Computer software]. Los Angeles, CA: Muthén & Muthén.
46.
OppenheimerD. M.MeyvisT.DavidenkoN. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867-872.
47.
PaolacciG.ChandlerJ. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184-188.
48.
PaolacciG.ChandlerJ.IpeirotisP. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411-419.
49.
PatrickC. J.HicksB. M.NicholP. E.KruegerR. F. (2007). A bifactor approach to modeling the structure of the Psychopathy Checklist-Revised. Journal of Personality Disorders, 21, 118-141.
50.
PaulhusD. L.RobinsR. W.TrzesniewskiK. H.TracyJ. L. (2004). Two replicable suppressor situations in personality research. Multivariate Behavioral Research, 3, 303-328.
51.
ReiseS. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696.
52.
ReiseS. P.BonifayW. E.HavilandM. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140.
53.
ReiseS. P.KimD. S.MansolfM.WidamanK. F. (2016) Is the bifactor model a better model or is it just better at modeling implausible responses? Application of iteratively reweighted least squares to the Rosenberg Self-Esteem Scale. Multivariate Behavioral Research, 51, 818-838. doi:10.1080/00273171.2016.1243461
54.
ReiseS. P.MooreT. M.HavilandM. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559.
55.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150.
56.
SatorraA.BentlerP. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In CloggC. C.von EyeA. (Eds.), Latent variables analysis: Applications for developmental research (pp. 399-419). Thousand Oaks, CA: Sage.
57.
ShoreyR. C.StraussC.ElmquistJ.AndersonS.CorneliusT. L.StuartG. L. (2017). Distress tolerance and intimate partner violence among men in substance use treatment. Journal of Family Violence, 32, 317-324.
58.
SimonsJ.GaherR. (2005). The Distress Tolerance Scale: Development and validation of a self-report measure. Motivation and Emotion, 29, 83-102.
59.
StuckyB. D.EdelenM. O. (2015). Using hierarchical IRT to create unidimensional measures from multidimensional data. In ReiseS. P.RevickiD. A. (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp. 183-206). New York, NY: Routledge.
60.
StuckyB. D.ThissenD.EdelenM. O. (2013). Using logistic approximations of marginal trace lines to develop short assessments. Applied Psychological Measurement, 37, 41-57.
61.
SugiuraN. (1978). Further analysts of the data by Akaike’s information criterion and the finite corrections. Communications in Statistics-Theory and Methods, 7, 13-26.
TimpanoK. R.BucknerJ. D.RicheyJ. A.MurphyD. L.SchmidtN. B. (2009). Exploration of anxiety sensitivity and distress tolerance as vulnerability factors for hoarding behaviors. Depression and Anxiety, 26, 343-353.
64.
VeilleuxJ. C.PollertG. A.ZielinskiM. J.ShaverJ. A.HillM. A. (2019). Behavioral assessment of the negative emotion aspect of distress tolerance: Tolerance to emotional images. Assessment, 26(3), 386-403. doi:10.1177/1073191116689819
65.
VujanovicA. A.BernsteinA.LitzB. (2011). Traumatic stress. In ZvolenskyM.BernsteinA.VujanovicA. (Eds.), Distress tolerance: Theory, research, and clinical applications (pp. 126-148). New York, NY: Guilford Press.
66.
VujanovicA. A.Bonn-MillerM. O.PotterC. M.MarshallE. C.ZvolenskyM. J. (2011). An evaluation of the relation between distress tolerance and posttraumatic stress within a trauma-exposed sample. Journal of Psychopathology and Behavioral Assessment, 33, 129-135.
67.
WangM.RussellS. S. (2005). Measurement equivalence of the Job Descriptive Index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement, 65, 709-732.
68.
WolgastM. (2014). What does the Acceptance and Action Questionnaire (AAQ) really measure?Behavior Therapy, 45, 831-839.
69.
ZvolenskyM. J.BernsteinA.VujanovicA. A. (Eds.). (2011). Distress tolerance: Theory, research, and clinical applications. New York, NY: Guilford Press.
70.
ZvolenskyM. J.OttoM. W. (2010). Affective intolerance, sensitivity, and processing: Advances in clinical science introduction [Special series]. Behavior Therapy, 38, 228-233.
71.
ZvolenskyM. J.VujanovicA. A.BernsteinA.LeyroT. (2010). Distress tolerance: Theory, measurement, and relations to psychopathology. Current Directions in Psychological Science, 19, 406-410.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
