What’s Wrong With the Reference Group Effect in Cross-Cultural Research?

Abstract

Cross-cultural researchers often compare mean scores across cultures using subjective Likert scales, but this approach faces several methodological challenges. One alleged problem is the reference group effect (RGE), which posits that varying normative standards distort cultural comparisons, potentially suppressing or even reversing true differences. In this study, we revisit this issue and challenge its premise. Leveraging data from three large-scale international datasets, we examined the relationship between subjective socioeconomic status (SES) and its criterion variable, national wealth (GDP per capita). According to the RGE, respondents’ judgments based on their local peers would result in those from wealthy countries rating themselves as less affluent and those from developing countries rating themselves as relatively more affluent. Such biased evaluations would suppress true differences and lead to a null correlation between SES and GDP across countries. However, our findings consistently revealed a strong positive correlation: respondents from wealthier countries rated themselves higher, while those from developing countries rated themselves lower. The majority of respondents appear to interpret numerical anchors of Likert scales in a single, universal, and absolute manner—“middle class” carries the same meaning regardless of varying economic standards. This consistent interpretation enables valid cross-cultural differences to emerge. Our findings encourage cross-cultural researchers to interpret observed cultural differences with greater confidence when using Likert scales.