Abstract
Keywords
Introduction
With the growing influence of online reviews on consumer decisions and platform operations (Abrahams et al., 2015; Ceran et al., 2016; Cui et al., 2018; Kumar et al., 2018), one of the major operational challenges that online review platforms face is ensuring the quality of reviews (Chen et al., 2016; Lee et al., 2018). Therefore, if these platforms wish to succeed, they must establish mechanisms that motivate reviewers to contribute not only frequently but also with high quality (Dellarocas, 2010; Sun et al., 2021). To this end, platforms have devised various incentives to attract and retain reviewers who produce high-quality reviews (Burtch et al., 2018). While financial incentives for writing reviews are available (e.g., Khern-am-nuai et al., 2018), several review platforms commonly use non-financial incentives that recognize reviewer contributions (e.g., Goes et al., 2014). Among these non-financial incentives, recognition-based incentives (e.g., badges, status, and rank) are now widely employed by online review platforms (Anderson et al., 2013; Cavusoglu et al., 2015).
While considerable research has explored the effectiveness of recognition-based incentives (Anderson et al., 2013; Goes et al., 2016), recent studies have only begun to examine the impact of a transient, status-based incentive system on user contributions (Bhattacharyya et al., 2020; Zhang et al., 2020). In a transient, status-based incentive system, user statuses are not permanent, as they can be promoted or demoted from their statuses. In turn, platforms also struggle to maintain the exclusivity of status while minimizing the risk of alienating demoted reviewers. Although previous studies have investigated how a reviewer’s performance changes after achieving a certain status (Bhattacharyya et al., 2020), the impact of losing status on review quality has been largely unexplored. In this article, we address such a research gap by proposing the following research questions:
RQ1: What is the impact of status loss on the quality of reviews written by users who lose their status? RQ2: How do readers perceive the change in review quality following the loss of a reviewer’s status?
In our context, we empirically investigate the implications of status loss on review quality in the context of a third-party, online review platform by using a unique approach that distinguishes between intrinsic and perceived measures of quality. Intrinsic quality measures reflect the objective quality of reviews and are a supply-side construct on an online platform, while the perceived quality of reviews is a demand-side manifestation and is quantified by how informative and helpful review readers (i.e., platform consumers) perceive these reviews to be. This distinction in quality measures, although rarely addressed in the literature, is nonetheless important because the intrinsic and perceived quality metrics of a review come from separate entities, reviewers, and a platform’s review consumers (hereafter referred to as
We analyze the impact of status loss on these intrinsic and perceived qualities of reviews by using Yelp as the context. Yelp confers the transient performance-based status
Our main findings have direct implications for platform operations, which have emerged to be one of the key issues in the operations management literature (Khern-am-nuai et al., 2024; Sun and Xu, 2018; Yan et al., 2019). The apparent loss in intrinsic quality would be less of a problem for platforms if these reviews were consumed less or perceived to be commensurately less helpful. However, we find that reviews written by demoted reviewers are asymmetrically perceived to have higher quality by platform consumers. We use inequity theory and the elaboration likelihood model (ELM) to theoretically explain our empirical findings. In particular, inequity theory suggests that the decline in intrinsic quality may be driven by the feeling of unfairness perceived by demoted reviewers, who respond by posting lower-quality reviews. Using mechanism analyses, we rule out the possibility that this result could be driven by alternative explanations such as a pre-demotion loss of interest in the platform among demoted reviewers or change in the review-writing strategies of demoted reviewers post-demotion (i.e., by opting to write reviews on business units distinct from those they previously covered before their demotion). We conclude that the disparity (i.e., the perceived quality of reviews posted by demoted reviewers is unjustly higher than their intrinsic quality) can be attributed to the platform’s strategy of displaying these demoted reviewers’ past Elite status. The ELM suggests that such a difference in the two quality measures could arise if review consumers give more weight to peripheral cues (i.e., the badge that shows that a demoted reviewer once held status) than to central cues (i.e., the intrinsic quality of the reviews). We use two additional analyses to lend support to this mechanism. First, we compare if reviews of similar intrinsic quality on Yelp were perceived differently if they were written by demoted reviewers as opposed to reviewers who never held Elite status. We find that the perception of review quality, after accounting for intrinsic quality measures, is statistically higher for demoted reviewers. Second, we use a randomized experiment, which conclusively proves that the perception of review quality is indeed guided by peripheral cues rather than central cues of quality. The experiment also demonstrates that our results from the main analyses are not driven by the potential failure to capture intrinsic quality by the measures that we use in this study. Additionally, we study the role of temporal effects by using two different moderators: how long a reviewer has been with a platform and how long a reviewer held a certain status before losing it. We find that neither of the moderators plays a significant role in moderating the effect of status loss. Next, we review the related literature and discuss the theoretical underpinning of our article.
Background Literature and Theoretical Underpinning
Related Literature
UGC Platform Strategies and User Performance
The literature in this stream of research usually examines UGC from a supply-side perspective (e.g., Duan et al., 2008; Hu et al., 2009; Pavlou and Gefen, 2004; Wu and Zhao, 2023). For instance, past literature has identified factors that platforms can use to drive individuals to ask or answer questions in knowledge-sharing communities such as StackOverflow (e.g., Pu et al., 2022). Meanwhile, researchers have also studied the designs of UGC platforms. For example, Burtch et al. (2022) studied the impact of peer awards on the contribution behavior of Reddit users. In the same vein, Rishika and Ramaprasad (2019) demonstrated that the contribution behavior is significantly affected by the symmetricity of social ties, social embeddedness, and tie strengths in the online community.
Another sub-stream of research in this area focuses on platform strategies regarding the use of incentives to encourage content contributions. Particularly, numerous prior works study the implications of incentives on the content contribution behavior, including monetary incentives (Burtch et al., 2018; Khern-am-nuai et al., 2018; Qiao et al., 2020) and non-monetary ones (Rasool and Pathania, 2023; Yu et al., 2023; Zhang et al., 2020), and how incentives impact sellers on the platform (Fradkin and Holtz, 2023; Qiao and Rui, 2023). Our study is closely related to prior works in this sub-stream that involve the status of platform contributors. In particular, the literature suggests that status is a strong motivational factor (e.g., Lefevere, 1983). Relatedly, Cheng et al. (2020) empirically demonstrated the economic implications of the social reputation system. They find that the reputation badges of the sellers have significant and positive impacts on sales. Outside the e-commerce context, the implications of status loss point to conflicting results. Notably, most of these studies have investigated domains in which individuals are bound by employment or a commitment that makes individuals’ participation non-voluntary; in such settings, the effects of losing status are negative (e.g., Marr and Thau, 2014). Meanwhile, status loss involving participants who act voluntarily has been studied in contexts such as gaming platforms and loyalty programs where the loss of status can either yield negative effects (e.g., Duguid and Goncalo, 2015; Wagner et al., 2009) or positive effects (e.g., Deodhar et al., 2019; Pettit et al., 2010).
Our study contributes to the literature on platform strategies and user performance by examining the effectiveness of transient status awards in two distinct ways. First, we investigate the impact of status demotion in online review platforms, for which the primary benefit of status is social recognition, and in which individual engagement is mainly driven by psychological factors such as altruism and moral responsibility, rather than the financial incentives featured in loyalty programs (Ke et al., 2020; Lampel and Bhalla, 2007). In the context of our study, Yelp explicitly prohibits participating businesses from offering monetary incentives for reviews. 1 This nuance in our empirical setting ensures that the identification of the effects of status loss on review quality are not confounded by the financial motivation of users to write reviews. Second, we study a status evaluation system in which the criteria for change in status is endogenous to the evaluator (i.e., the platform), and, thus, unknown to the reviewers. Prior studies on status loss have typically used settings in which the criteria for gaining and maintaining status are publicly known (Liang et al., 2017). However, when the criteria for demotion are endogenous, a loss of status can breed perceptions of unfairness and injustice among affected reviewers, which can impact their subsequent contributions. We expand upon this perspective in Section 2.2 where we discuss theories that underpin our empirical investigations.
Display of Status and Perceived Review Quality
User-generated reviews play an important role in the decision-making process of potential consumers, who rely heavily on UGC in the absence of other information (Dellarocas, 2003). The volume of reviews, tone of reviews, star ratings, and intrinsic as well as perceived quality of reviews all seem to affect the potential sales of the focal product/service (Floyd et al., 2014). However, a potential consumer using these reviews may not be acquainted with the reviewer(s). Thus, that consumer may question the credibility of the content generated by the latter. Therefore, the large variance in the veracity of this indirect experience coupled with the ever-growing volume of reviews requires consumers to use cues to quickly identify the most credible and relevant content (Yin et al., 2014). To help consumers search for credible information more efficiently, these platforms have instituted several measures based on peer evaluations of reviews or
Intricately linked with perceived review quality is reviewer credibility. With respect to designing online review platforms, Dellarocas (2010) underscores the importance of “what information should be included” in a reviewer’s profile. As mentioned previously, recognition-based incentive systems are prevalent in online review platforms, and it is a common practice to show reviewers’ statuses or badges on the reviewers’ profile pages (e.g., “Top Contributor” on TripAdvisor, “Elite” on Yelp). Many platforms that employ a transient status-based reward system often show the past status of reviewers. For example, Yelp shows the years during which reviewers held “Elite” status, even if they currently do not hold that status. Similarly, HackerRank displays top leaderboard accolades that a user received previously. Such a practice is also commonly observed in massively multiplayer online role-playing games, such as
UGC Quality Control
While consumers have relied on high-quality reviews to make future consumption decisions, companies for whom these reviews have been written have used such responses to mitigate consumer concerns, thereby increasing consumer satisfaction (Abrahams et al., 2015). However, operationally, there are quality-related challenges that an online review platform faces. Chen et al. (2016) observed that online reviews are at best incomplete, for they lack the opinions of consumers who never write reviews, which could lead to reporting bias. Chen et al. (2016) also find that positive experiences are reported more than negative experiences. Online review platforms are also subjected to sentiment manipulation from strategic parties, as shown by Lee et al. (2018) in the context of movie operations, and such manipulation could lead to a decline in information quality and loss of consumer welfare.
Our work contributes to the literature on the quality management of online reviews by providing insights into how the dynamics of intrinsic quality and perceived quality are impacted differently by a platform’s strategies on review quality management, specifically the decision to demote a reviewer from a transient status (“Elite” status in our empirical context). First, we empirically demonstrate that a platform’s strategy to demote reviewers leads to a decline in the intrinsic quality of reviews. Second, we show that a platform’s strategy to display the past status of the demoted reviewers leads to perception bias, in which a “poorer” review written by a demoted reviewer is unduly perceived to be of higher quality. In addition, our study contributes to the literature on the unintended consequences of platform policy (e.g., Joglekar et al., 2016; Anderson et al., 2023; Mayya and Viswanathan, 2024) by demonstrating that the policy to demote Elite reviewers not only directly impacts the quality of reviews written by demoted reviewers but also indirectly impacts how these lower-quality reviews are perceived by review readers. Last, our work is also related to the preferred partnership literature (Meehan and Wright, 2013; Sahaym et al., 2023) as we unveil how the removal of the “preferred partner” tag (i.e., the Elite tag in our empirical context) by one entity impacts relevant stakeholders.
Underpinning Theories
In our review of the literature, two relevant theories emerge as potential mechanisms that could inform our empirical analyses. In this subsection, we expand upon these theories and discuss how they underpin the contribution behavior of demoted reviewers and how consumers perceive the quality of reviews.
Inequity Theory
Recall that in our empirical setting, the criteria for status demotion are endogenous to the platform, which may induce perceptions of unfairness among demoted reviewers. Such perceptions and corresponding changes in behavior are typically explained by inequity theory.
Inequity theory, developed by Adams (1963) and based on Festinger’s theory of cognitive dissonance (Festinger, 1957), explains how individuals perceive and evaluate fairness with respect to their inputs and outcomes compared to those of others. The theory states that individuals evaluate the value (
In the context of our study, reviewers expect to be recognized with fairness by the platform to which they are contributing. Hence, if reviewers believe that their contributions are comparable to those of other reviewers who held similar status in a given period, then the former would expect the platform to allow them to retain their status in the next period. Inequity theory in online community settings has been applied primarily to identify different factors that motivate contribution behavior. For example, Chou et al. (2016) showed that a sense of virtual community is fostered by the perception of online justice or fairness, which leads to value co-creation behavior. Drawing upon inequity theory, Feng and Ye (2016) suggested that in online communities, individuals who consume reviews perceive themselves to have gained knowledge unfairly from the efforts of others and, therefore, engage in reciprocal contributions to restore equity. Conversely, Bhattacharyya et al. (2020) used the inequity theory to explain that eligible but unacknowledged members of an online review platform may reduce their contributions because of a sense of recognition inequality. In our context, we employ inequity theory to illustrate how the intrinsic quality of reviews is affected for individuals when they are demoted from a status on an online community platform.
Elaboration Likelihood Model
As previously discussed, consumers of an online review platform typically develop their perceptions of review quality based on various factors. Our study focuses on one of these factors, reviewer status, as the primary variable of interest in our empirical analysis. In that regard, previous studies have extensively drawn upon the ELM to explain the formation of perceptions among consumers.
ELM proposes that a recipient’s perception of information is influenced by both the content of the information and the context in which it is presented (Petty et al., 1986). Zhu et al. (2014) draw upon ELM to theorize that consumers seeking information could focus on “central cues” and be influenced by “
In the context of online review platforms, we posit that the credibility of the reviewer (or source) plays a critical role in shaping platform consumers’ perceptions of a review’s quality. In the online context, while a potential consumer may not have any personal acquaintance with a reviewer, the social status or recognition conferred by peers or the platform can act as a direct substitute for all three parameters (expertise, trustworthiness, and source attractiveness) of the source credibility model. In a similar vein, signaling theory suggests trust and credibility about others’ actions (e.g., written reviews) are reinforced when the superiority of the other is demonstrated through a signal (e.g., platform-conferred status recognition) that is difficult to replicate and is costly to obtain (Donath, 2007). Therefore, while “argument quality” determines the intrinsic quality of reviews, “source credibility” impacts perceived quality. In our specific context, we leverage ELM to discuss how a change in reviewer status, a
In summary, our empirical investigations are guided by the inequity theory and the ELM. We summarize how these theories underpin the constructs used in our empirical analyses in Figure 1.
Conceptual model of our study.
In this section, we describe our research context and the data we use in our empirical analyses.
Empirical Setting
We use Yelp as our focal online review platform. With over 70 million active reviewers and 200 million reviews, it is one of the largest review platforms on the Internet (Smith, 2021). Reviewers on Yelp can leave a review for a product or service that they have used and can help the business with feedback and word of mouth. Around 60% of Yelp businesses are restaurants, cafes, or food and beverage services.
At the end of each year, reviewers can nominate themselves for “Elite” status. The nominations are evaluated by Yelp’s local area manager, and this process is known to be subjective (Nilsson et al., 2018). Therefore, it is not guaranteed that two reviewers with similar contributions will be conferred (or not conferred) status similarly. Each year, Elite reviewers account for about 2% of Yelp’s reviewer base, and the reviewers who receive this status retain it for the next calendar year. These reviewers are automatically renominated for the continuance of their status. However, some of them may not be conferred the same status in the following year if they do not meet the criteria that are endogenously decided upon by Yelp. More importantly, a reviewer who is declined a promotion (or is demoted from an existing status) is unaware of the exact criteria behind the platform’s decision (Bhattacharyya et al., 2020). We leverage this process to develop a quasi-experiment-based identification strategy in our subsequent empirical analyses. We primarily use Yelp’s academic dataset, 2 which provides information for different business units from 11 cities between 2006 and 2018. There are over 6 million reviews written by 1.2 million reviewers for 192,609 businesses in total. For each review, the dataset provides the date of a review, text written by the reviewer, the star rating conferred by the reviewer, the compliments that a review received (e.g., useful, funny, and cool), and the business for which the review is written, as well as the business’s characteristics and categories. Reviewer details consist of the reviewer’s location, years of activity on Yelp, number of reviews written, years the reviewer maintained Elite status (if any), and overall compliments received.
Our dataset consists of reviews written in 11 cities only. However, it is important to note that reviewers in our dataset may write reviews in other locations that are not a part of the Yelp academic dataset, which imposes two issues. First, we are interested in studying reviewer behavior by using an observable dataset, and it would be problematic to include reviewers whose behavior is largely unobserved. For example, we find reviewers for whom we observe only one review in our dataset. However, our dataset is restricted to only 11 cities, and it is possible that such reviewers wrote many more reviews for businesses outside of these 11 cities. Second, the literature has shown that review-generating behavior is significantly different when reviewers write reviews outside their base location (Kokkodis and Lappas, 2020). We take two steps to alleviate these issues. First, we follow the approach used by Bhattacharyya et al. (2020) to filter out reviewers who have written > 25% of their reviews outside our dataset (i.e., outside the 11 cities captured in the Yelp Academic dataset). This practice ensures that the reviewers included are not tourists. In addition, we use the unique reviewer ID information from the dataset and program a web crawler that collects additional data for each reviewer. These additional data include all the reviews written by focal reviewers beyond the cities covered in our primary data. We augment our existing dataset with these additional data and use it for our main analysis. In total, there are 495,430 reviews written by reviewers who have achieved Elite status at least once.
Intrinsic and Perceived Review Quality Measures
In the extant literature on online reviews, researchers have used several variables to proxy the intrinsic quality of reviews. In this article, we use the following six variables that are commonly used in the literature to capture intrinsic review quality:
Review length: This measures the number of alphanumeric characters in a review. The past literature has shown that longer reviews contain more information and therefore are commonly considered to have higher quality (Mudambi and Schuff, 2010). Flesch reading ease: This is a score that measures the ease of reading a text. It considers the total number of sentences, words, and syllables included in a document to generate this score. A higher score means that the textual content is simple to read. As such, prior studies commonly denote reviews with low Flesch reading ease scores as high-quality reviews (Garnefeld et al., 2021; McCloskey, 2021). Flesch-Kincaid reading grade: This metric measures the grade level of education required to produce textual content. Accordingly, prior studies tend to interpret a review with a higher Flesch-Kincaid grade (i.e., a review produced by a writer with a higher level of education) as a higher-quality review (Carlson et al., 2015; Manchaiah et al., 2020). Gunning fog index: Similar to the Flesch-Kincaid reading grade, the Gunning fog index measures the education grade of the writer based on the text. Accordingly, the literature has associated a review with a higher Gunning fog index score (i.e., higher-education grade of the review writer) with higher review quality (Khern-am-nuai et al., 2018; Yin et al., 2016). Dale Chall readability score: This score also measures the grade level required to write the textual content. A score below 4.9 means that the review is written by someone with grade 4 or below capabilities, while a score of above 10 means that the review is written by someone with grade 16 or above capabilities. As such, prior studies usually treat reviews with a high Dale Chall readability score as high-quality reviews (Khreiche, 2020; Zhang et al., 2022). Lexical density: This measures how many different lexical words are present in a text divided by the total number of words in the review. Therefore, a higher quality review tends to have a higher lexical density score (McCarthy and Jarvis, 2010).
We extract these intrinsic quality measures from the review text using a package called TextStat in Python (Bansal and Aggarwal, 2021). The operationalization of these variables is summarized in Section A of the E-Companion, where we also provide visualization examples of these measures.
To measure the perceived quality of a review, we create two measures. On Yelp, review readers can vote reviews as “useful,” “cool,” and “funny.” In existing literature on online reviews, these votes are typically considered collectively as indications of review quality (e.g., Bakhshi et al., 2015; Li et al., 2019). Therefore, our first measure, termed
In Table 1, we present the summary statistics of our variables of interest, that is, the intrinsic quality measures and perceived quality measures for the 9,879 reviewers who lost statuses during our observation period. We also include our control variables, namely, the average star rating, the number of years since the reviewer has contributed to Yelp, and the number of years since the reviewer has/had Elite status.
Summary statistics of the variables of interest (per reviewer).
Note: The summary statistics above are generated from 9,879 reviewers who lost their statuses during our observation period.
Summary statistics of the variables of interest (per reviewer).
Note: The summary statistics above are generated from 9,879 reviewers who lost their statuses during our observation period.
Using the data we described in the previous section, we construct our empirical models. Since our first objective is to establish the causal effect of status loss on the intrinsic quality of future reviews, we face the endogeneity issue of who gets demoted. Fortunately, Yelp uses an endogenous promotion/demotion process (i.e., it does not reveal the exact criteria it uses to promote a reviewer to Elite status or to demote a current Elite reviewer). We leverage this selection process to set up our research framework as a quasi-experiment, wherein we use PSM to control for potential endogeneity concerns and subsequently a DiD technique to estimate the treatment effect(s). These techniques have been extensively used in studies that analyze causal inferences (e.g., Kokkodis and Lappas, 2020; Mayya et al., 2021; Sharma et al., 2020). Within this framework, our analysis is akin to a two-group experiment, in which the reviewers who lost status are considered to have received the treatment while those who did not lose status constitute our control group. When we compare both the difference between the treatment and control groups and the corresponding difference between the pre-treatment and post-treatment periods, we can causally identify the effect of the treatment (i.e., loss of status) on intrinsic quality and the perceived quality of reviews. It is also worth noting that Yelp does not distinguish reviews written by friends and those written by other reviewers, which ensures that our estimations are not impacted by potential peer effects.
Data Structure
In our article, we construct the data at the reviewer level (i.e., each observation corresponds to a reviewer). For each time a reviewer lost status in our dataset, we create an observation that consists of two time periods. The first time period (
Meanwhile, to create the control group for our analyses, we implement the following procedures. A reviewer who never lost status during the time frame of our data set becomes a candidate for the control group. Moreover, in our setting we allow a demoted reviewer to be a control-group candidate, albeit not in the same year that the reviewer lost Elite status. More specifically, for these reviewers we only consider the not-yet-demoted years as part of the potential control group observation. Furthermore, for such a reviewer, the following years cannot be observations for the pre-year in the control group: the first year that the reviewer achieved Elite status, the year at the end of which that reviewer lost status, and the year after the reviewer lost status. With these heuristics in place, for the qualified control-group candidates, we construct an observation per reviewer per year that consists of two time periods. The first time period (
Propensity-Score Matching
To perform our empirical analyses, we must control for the endogeneity related to how an Elite reviewer is demoted. In parity with usual practices in quasi-experimental settings, we use PSM to find the matched control group of current Elite reviewers to compare with our treatment group of reviewers who had lost Elite status. Since our objective is to identify a pair of reviewers who write reviews of similar quality in the pre-treatment period, we follow prior studies and use variables that represent review quality as matching covariates (e.g., Qiao et al., 2020). Specifically, as we discussed earlier in Section 3.1.1, these variables include review length, the Flesch reading ease, the Flesch-Kincaid reading grade, the Gunning fog index, the Dale Chall readability score, and lexical density, all of which capture intrinsic review quality. We also include usefulness votes and total compliments, which represent perceived quality. 3 In addition, we include more covariates to improve the robustness of our matching by considering variables used in studies that employed reviewer-level matching in the context of online reviews. These variables include average star rating, the number of years since a reviewer has contributed to Yelp, and the number of years since a reviewer gained/lost status (e.g., Khern-am-nuai et al., 2018). We performed this matching using all data in the pre-treatment period, for which we had 8,723 treatment reviewers and 15,689 potential control group reviewers. We estimate the propensity score using a logit function and use nearest-neighbor matching without replacement to obtain our final matched data, which contain 8,723 unique treatment reviewers and 5,925 unique control reviewers. Given that the majority of reviewers experienced a status loss within our 12-year observation window, we permit demoted reviewers to serve as control users during the time they retained Elite status. Consequently, a demoted reviewer may function as a control user(s) multiple times in our matching process, with their Elite years serving as control observations. Thus, the overall count of unique control reviewers in our matched dataset is lower than that of treated users.
We next validate our matching by testing the balance between the variables of the treatment group against those of the matched control group. It is important to note that we perform PSM because we aim to use our DiD analysis as our primary empirical specification. However, because our dataset consists of only two time periods, the balance test essentially serves as a test for the parallel trend assumption as well. Here, we evaluate the balance between the treatment group and the control group in PSM-based matching using the standardized mean difference (SMD) (Cohen, 1988). As shown in Table 2, we achieve excellent matching between the treatment group (reviewers who lost Elite status) and the control group (current Elite reviewers) since none of the variables has an SMD higher than 0.20, which is the threshold that indicates a lack of balance between the treatment group and the control group (Zhu et al., 2024).
Covariate balance between treated and control groups.
SMD = standardized mean difference.
Covariate balance between treated and control groups.
SMD = standardized mean difference.
Having identified our matched data, we now have measures for the intrinsic quality and perceived quality of reviews written before versus after the treatment for both the treatment group and the control group. Here, we utilize the DiD regression specification that is commonly used in the literature (e.g., Kumar et al., 2022) with the following equation:
In the context of a transient, status-based reward system, researchers have underscored the importance of studying the temporal effects on the quality of reviews. For instance, Zhang et al. (2020) showed that an individual’s review quality stabilizes over time, as supported by learning theory, which proposes that repeated experiences linger in the subconscious memory (Hofer et al., 2009; Mowrer, 1960). Interestingly, Bhattacharyya et al. (2020) find that reviewers who hold status for long periods tend to be less productive, which negatively impacts quality. This drop in performance is attributed to reinforcer satiation that diminishes the strength of the reinforcement (i.e., the urge to maintain status) on an individual’s behavior with the repeated occurrence of the reinforcement (Bhattacharyya et al., 2020; Murphy et al., 2003). Therefore, the intrinsic quality of reviews after a status loss could be dependent on temporal associations between a reviewer and a platform. In our context, these temporal associations could include the length of time a reviewer has participated with a platform, or it could reflect how long a reviewer held Elite status before losing it. We call these time-related variables “temporal moderators.” Furthermore, features related to temporal associations with an online review platform are commonly highlighted characteristics of a reviewer. When reviewers exhibit these characteristics, they strengthen peripheral cues, which possibly then affects a reviewer’s expertness-trustworthiness-attractiveness. Therefore, we examine two temporal moderators (expressed as 
In this section, we present the results from our empirical exercises. For ease of exposition, we express
Main Results
In this subsection, we discuss two sets of results. First, we report findings related to the effect of changes in status on the intrinsic review quality of demoted reviewers, as well as how consumers’ perceived quality of reviews written by the demoted users is affected. To obtain these results, we use the DiD specification in equation (1). Table 3 shows the estimated effect of status loss on the six intrinsic quality measures alongside the two perceived quality measures.
Effect of status loss on review quality of lost Elite compared to current Elite reviewers.
Note: *
; **
; ***
; HC1 robust standard errors are in parentheses.
Effect of status loss on review quality of lost Elite compared to current Elite reviewers.
Note: *
With regards to the impact of status loss on the intrinsic quality measures of a review, Table 3 reveals that reviewers who have been demoted tend to produce reviews with lower intrinsic quality compared to those who retain their status. Such a decrease is consistent across all intrinsic quality measures we employ. We note that all but one of these intrinsic measures are statistically significant at
With regards to the impact of status loss on the perceived quality of a review, Table 3 shows that the
Our main results demonstrate that demoted reviewers tend to contribute reviews with significantly lower quality after a loss of status. However, despite this lower intrinsic quality, there are no significant changes to the perceived quality of reviews written by demoted reviewers from the perspective of review consumers. A natural question thus arises: Are these effects moderated by the experience of demoted reviewers? To answer this question, we identify two moderators commonly used in the literature to represent reviewer experience (e.g., Bradley, 2007; English et al., 2010). Here, we explore the effect of two temporal moderators: (i) the number of years a reviewer contributes to a platform (
Heterogeneous effect on review quality of lost Elite compared to current Elite reviewers, based on the number of years a reviewer contributes to Yelp.
Note: *
; **
; ***
; HC1 robust standard errors are in parentheses.
Heterogeneous effect on review quality of lost Elite compared to current Elite reviewers, based on the number of years a reviewer contributes to Yelp.
Note: *
Heterogeneous effect on review quality of lost Elite compared to current Elite reviewers, based on the number of years a reviewer held Elite status.
Note: *
First, we examine the effect that both of these moderators have on intrinsic quality in two sets of regressions presented in Tables 4 and 5, wherein the effects of the moderators are shown as interactions with
Second, we analyze the effect of temporal moderators on how a status loss affects the perceived quality of reviews, also shown in Tables 4 and 5 for moderators
We note that the main effects of both the temporal moderators, captured by the coefficients of
Our empirical findings demonstrate that when reviewers experience a loss of status, their posted reviews are intrinsically lower quality. However, consumers of these reviews continue to perceive them as having relatively high quality. Additionally, the effects of the status loss do not seem to be influenced by reviewers’ experience with the platform. In this section, we examine the empirical evidence supporting two potential underlying mechanisms that explain these results: (1) the perception of inequity among demoted reviewers to explain the decline in the intrinsic quality of reviews and (2) the predominance of
Why Do Demoted Reviewers Post Reviews With Lower Quality?
Inequity theory states that when individuals compare their own outcome-input ratios to those of relevant others, the former may perceive unfairness if their outcomes were lower than individuals with seemingly similar or lower inputs (Adams, 1963, 1965). We argue that in our context since the status-demotion criteria were endogenously determined by the platform (i.e., not publicly announced), a demotion could lead to perceived unfairness among demoted individuals. In accordance with inequity theory, demoted individuals who perceive such unfairness can react by (a) distorting either their own or others’ inputs or outcomes cognitively, (b) acting in a manner that causes others to change their own inputs or outcomes, (c) behaving in a way that changes their own inputs or outcomes (e.g., by working harder), (d) selecting a different comparison individual (presumably someone whose outcome/input ratio is considered equal to theirs), or (e) exiting the situation altogether, such as quitting the job (Pritchard et al., 1972). In our context, Yelp does not reveal the criteria it follows to make status decisions. Therefore, a demoted reviewer is likely to perceive unfairness. However, in such instances, a reviewer cannot directly change the effort level or the outcome of other reviewers. Instead, demoted reviewers can either increase their own effort to regain their lost status, or these reviewers can decrease their effort when they make future contributions. Previous studies have shown that individuals who feel undervalued often express decreased job satisfaction and demonstrate lower performance (Adams, 1963; Pritchard et al., 1972). Therefore, our findings, which show a decline in intrinsic quality measures following status loss, are consistent with the results. In the next step of our analysis, we examine two alternative explanations for the decrease in the intrinsic quality of reviews following a loss of status.
Do Demoted Reviewers Post Reviews With Lower Quality Because They Have Lost Interest in the Platform Even Before the Demotion?
We made an implicit assumption in using inequity theory to explain the decline in intrinsic quality: that the observed effect on demoted reviewers was solely due to their loss of status. However, it is plausible that reviewers who lost their status may have lost interest in contributing reviews even before the status loss occurred. This could have resulted in the production of inferior reviews and the subsequent loss of status. To ensure the robustness of our results against such self-selection issues, we present two arguments.
First, we recall that our identification strategy relies on a matching technique to ensure that the demoted reviewers (i.e., treatment group) and the current Elite reviewers (i.e., control group) have similar characteristics before the treatment (i.e., the loss of status). Our balance test in Table 2 ensures that both groups have reasonably similar characteristics. In other words, we found no evidence that demoted reviewers (treatment group) demonstrate a loss of interest differently from matched current Elite reviewers (control group) in contributing reviews before the treatment.
Second, we employ an alternative specification to be more conservative. We focus only on demoted reviewers who, in the pre-demotion period, wrote more reviews than the median number of reviews written by Elite reviewers. This way, we restrict the demoted reviewers to only those who did not show signs of losing interest in writing reviews on the platform before they lost status. We rerun the DiD regression specified in equation (1) on a subset of these demoted reviewers and their corresponding matched control group of current Elite reviewers. The results, shown in Table 6, demonstrate that even among this subset of demoted reviewers, the negative effect of status loss on intrinsic review quality persists.
Effect of status loss on the intrinsic quality of reviews by lost Elite compared to current Elite reviewers for reviewers who make significant contributions (in volume) in the pre-treatment period.
Note: *
0.10; **
0.05; ***
; HC1 robust standard errors are in parentheses.
Effect of status loss on the intrinsic quality of reviews by lost Elite compared to current Elite reviewers for reviewers who make significant contributions (in volume) in the pre-treatment period.
Note: *
In the previous section, we presented empirical evidence that even among demoted reviewers who post reviews at a higher level than the average contribution level of all Elite reviewers, a decrease in intrinsic review quality is observed after status loss. Hence, it is unlikely that the decline in intrinsic review quality is driven by reviewers who lost interest in the platform even before their loss in status. In this subsection, we explore an alternative explanation in which the decrease in review quality may be caused by a change in the review-writing strategies of demoted reviewers after losing their status. Specifically, after demotion, a reviewer may strategically write reviews for certain types of restaurants that differ from the ones they wrote about before demotion. As the set of focal restaurants changes, the intrinsic quality of the reviews written for this new set of restaurants may also change.
To investigate such a potential alternative explanation of our results, we rerun our DiD analysis specified in equation (1) on intrinsic review quality measures after incorporating more controls for restaurant-related and peer reviewer-related characteristics. Specifically, we have added the following covariates as additional control variables: the average ratings assigned to restaurants, the age of the restaurants, the review volume of the restaurants, the average menu price of the restaurants reviewed, the number of restaurants in the area, and the average ratings assigned to the businesses by other reviewers. As observed in Table 7, even after we control for these attributes, we still find that the intrinsic quality declines for demoted reviewers. Therefore, we posit that the decrease in the intrinsic quality of the reviews, which contributed to the loss of status, is unlikely to be driven by the change in the type of restaurants reviewed by the demoted reviewers.
Effect of status loss on the intrinsic quality of reviews by lost Elite reviewers compared to current Elite after controlling for additional restaurant and reviewer-related characteristics.
Note: *
; **
; ***
; HC1 robust standard errors are in parentheses.
Effect of status loss on the intrinsic quality of reviews by lost Elite reviewers compared to current Elite after controlling for additional restaurant and reviewer-related characteristics.
Note: *
Our main findings suggest that the intrinsic quality of reviews written by demoted individuals declines after their status loss. However, we also find that the perceived quality of these reviews does not decrease after the loss of status. To explain this result, we turn to the ELM, which posits that evaluations of information are guided by a combination of central and peripheral cues, with the former based on the intrinsic quality and the latter based on the source credibility (Petty et al., 1981, 1986; Sussman and Siegal, 2003). To evaluate the quality of reviews, consumers on the platform may depend on peripheral cues as they might not personally know the reviewers or their level of expertise. In such cases, platform recognition in the form of badges and awards can reinforce these peripheral cues and enhance the consumers’ confidence in the quality of reviews (Donath, 2007). In our context, even if a reviewer has been demoted, the platform still displays whether a reviewer previously held Elite status, which enhances weight of the peripheral cue of quality and increases consumer confidence in the credibility of reviews written by demoted reviewers. Correspondingly, the dominance of peripheral cues over central cues causes consumers to not recognize the decline in the intrinsic quality of reviews written by demoted reviewers. Furthermore, it is possible that consumers may not easily recognize the decrease in intrinsic quality, leading to a less significant effect on the perceived central cue. Thus, consumers perceive that the quality of reviews written by demoted reviewers is statistically similar to those written by current Elite reviewers, even though the intrinsic quality is lower in reviews written by the former compared to the latter.
Perceived Quality Comparison Between Demoted Reviewers and Reviewers With No Past Status
To further support the idea that the display of past statuses affects perceived quality, we conduct an additional comparison between demoted reviewers and reviewers who never held statuses, so we may determine how their review quality perceptions differ. If the display of past statuses truly has a disproportionate effect on perceived quality, we would then expect reviews written by demoted reviewers to be perceived as higher quality compared to reviews of similar intrinsic quality written by reviewers who never had statuses. To test this conjecture, we run the same DiD estimation stated in equation (1) and use demoted reviewers as our treatment group against a matched set of reviewers who never held statuses as our control group.
5
Our results, presented in Table 8, indicate that, even after we control for intrinsic review quality, both
Effect of status loss on perceived quality in reviews by lost-Elite reviewers compared to reviewers who never held status.
Note: *
; **
; ***
; HC1 robust standard errors are in parentheses.
Effect of status loss on perceived quality in reviews by lost-Elite reviewers compared to reviewers who never held status.
Note: *
However, one can still argue that our use of six distinct intrinsic quality measures may not fully capture the true meaning of “information” for a consumer. Thus, the information content of reviews written by demoted reviewers might remain unchanged, yet our intrinsic measures might fail to account for this. To address such concerns, we conducted a randomized experiment on Amazon Mechanical Turk to investigate the underlying mechanism that guides consumers’ perception of quality, showing that it is indeed peripheral cues, rather than central cues, that play a significant role in online review platforms.
The objective of our randomized experiment is to cleanly capture the impact of displaying past and present statuses (or no status) on the perceived quality of reviews. In our context, we are interested in evaluating the impact of displaying the status of reviewers to platform consumers on how consumers’ perceptions of quality changes based on the status, itself.
We recruited 480 subjects in September 2022 using Amazon Mechanical Turk (MTurk) to conduct our experiment. Our subjects were required to rate the quality of the review text on a scale of 1–10. Each subject received seven reviews to rate and a few demographic questions to answer, including gender, age, and level of education. We randomly selected a subset of 50 reviews from our reviews dataset. Each review could appear to a subject with only one of the randomly chosen possible treatments: written by a reviewer who currently holds Elite status (
We did not collect any personally identifiable information from the subjects. At the end of the task, each subject was paid $0.5 for participation. It took an average of 4 min and 37 s for each subject to finish the task. To ensure the quality of our subjects, we only recruited subjects who had participated in at least a hundred similar approved tasks in the past. Our detailed instructions and the demographic questions given to subjects are included in Figure A5 in E-Companion Section B.2.
Out of the data collected from our experiment, we observed a few systematic inconsistencies in our data-generation process. First, we observed that some of the responses may be generated by bots. This can be determined by the time that a subject would take to log in and submit the Recaptcha for a form. We found 28 such anomalies and excluded their observations. Second, we noticed that some subjects chose to rate all the reviews at the highest quality of 10. It is clear that these subjects did not follow the instructions and were participating just for the reward; hence, we excluded observations from 14 such subjects. In total, we have 438 subjects who evaluated a total of 50 unique reviews. Among these, we have 1,064 observations for which reviews show no status (
In Table 9, we find that both the treatments that display the reviewer status (current Elite status
Effect of display of different status on perceived review quality.
Note: *
Difference in the effect of Elite status display on perceived review quality w.r.t. to past-Elite status display.
Note: *
In summary, we use this section to draw upon inequity theory and explain the decline in intrinsic quality of reviews contributed by demoted reviewers after their loss of status. We also extend our empirical analyses to rule out two alternative explanations that may explain the decline in intrinsic quality after status loss. In addition, we utilize the ELM to theoretically explain why the perceived quality of reviews does not change for the reviews posted after status loss, even when those reviews have significantly lower intrinsic quality. We also provide further support to our proposed mechanism by conducting a randomized experiment to cleanly identify the change in perceived quality (or the lack thereof) among consumers with respect to the change in reviewers’ statuses.
To ensure that our results are robust, we perform several additional analyses. Details of these analyses and corresponding results are reported in Section D of the E-Companion. The analyses are as follows:
To ensure that our results are not driven by the matching method employed in our empirical analysis, we consider an alternative matching algorithm based on an entropy balancing technique. We report our results in Section D.1 of the E-Companion. In our main specification, a reviewer can act as both a treated individual and a control reviewer, depending on the timing of their demotion from the Elite status. Thus, we match on each instance of a reviewer for our analysis. As a robustness check, we match at the reviewer level instead of reviewer instance, excluding We consider an alternative identification strategy to ensure that our results are not driven by our primary identification strategy: matching and DiD. Specifically, we employ a causal forest algorithm (Wager and Athey, 2018) and report our results in Section D.3 of the E-Companion. We perform a falsification test using a placebo test in which we use a “fake” treatment date instead of the actual treatment to observe whether our results are simply spurious. The fake treatment date is the same date as the actual treatment date but one calendar year prior. We report our results in Section D.4 of the E-Companion. We perform another falsification test using a randomized treatment test. Specifically, we randomly assign users in our dataset to either a treatment group or a control group and run our regression models for 10,000 iterations. We report our results in Section D.5 of the E-Companion. In our main analysis, we excluded the In our main analysis, we addressed self-selection bias by employing PSM. As a robustness check, we employ a Heckman two-stage estimation. In the first stage of the model, along with the review quality measures used in PSM, we incorporate three location-based variables. These variables, influencing Yelp managers’ demotion decisions, are less likely to have a direct impact on an individual’s review quality, making them suitable for fulfilling the exclusion restrictions. Detailed results can be found in Section D.7 of the E-Companion. In our main analyses, we set up the treatment-effect model as a “canonical” version of DiD, that is, two groups in two time periods. In this robustness check, we implement a staggered DiD approach following Callaway and Sant’Anna (2021). We report our results in Section D.8 of the E-Companion. We provide predictive analytics on the Yelp data, as a substitute to the Amazon Turk Experiment, so we may establish that peripheral cues indeed guide the perceived quality. We report our results in Section D.9 of the E-Companion.
All results are qualitatively and quantitatively similar to our main results. Notably, the treatment group in these robustness exercises is reviewers who lost Elite status, and the control group is reviewers who currently hold Elite status.
Discussion and Conclusion
Consumer reliance on peer information in the digital age has led to the dominance of UGC platforms. For these platforms, balancing content quality and user engagement is a key operational challenge. While the impact of status rewards for high-quality content have been studied, the impact of losing status remains unexplored. This paper addresses this gap by analyzing observational data from Yelp to examine the consequences of status loss on UGC platforms.
This study explores the impact of exclusivity in performance-based rewards on the quality of UGC. Particularly, we focus on the effect of losing status on review quality within an online review platform. We examine whether demoting reviewers leads to lower quality reviews and, consequently, harms the platform. Our findings reveal a two-fold effect. First, the intrinsic review quality of reviewers diminishes significantly upon demotion, potentially due to dissatisfaction with the platform stemming from the perceived inequity related to demotion. We rule out alternative explanations of a decline in intrinsic quality stemming from reviewers losing interest in the platform before demotion or from opting to review different types of businesses post-demotion. Second, the platform’s policy of displaying a reviewer’s past status disproportionately affects the perceived review quality. Even with the lower intrinsic quality, consumers continue to consider these reviews helpful due to the dominance of peripheral cues (i.e., a reviewer’s past status) over central cues (i.e., the intrinsic review quality) in their evaluations. This disparity between the intrinsic and perceived quality of a review undermines the value and sustainability of the platform in the long run. We utilize inequity theory and the ELM to explain these findings. The summary of our findings is provided in Table 11.
Our study offers significant research contributions. First, it provides valuable insights on a previously unexplored area: the effect of losing status on reviewer behavior and platform welfare. While previous studies have focused on the impact of status acquisition on review-generating behavior, the implications of status loss in these transient, status-based incentive systems have been less explored. This study addresses this gap by empirically examining how status loss affects the behavior of demoted reviewers.
Second, we go beyond user behavior to explore the operational implications for platforms. Review platforms rely on both reviewer effort (intrinsic quality) and consumer perception (perceived quality) to thrive. These distinct aspects, often conflated in the existing literature, influence platform value differently. Here, we analyze how status loss affects the interplay between intrinsic and perceived quality measures. Interestingly, our findings reveal a decline in intrinsic quality post-demotion, but no corresponding decline in perceived quality. This highlights a critical gap that future platform designs should address.
Finally, we introduce a novel theoretical framework involving inequity theory and the ELM. Specifically, we explain the decline in intrinsic quality with inequity theory, positing that demotion triggers the perception of unfairness, leading to lower effort. The gap between intrinsic and perceived quality is explained by the ELM. We argue that consumers rely more heavily on peripheral cues (past status badges) than central cues (review content) when evaluating reviews due to the platform’s practice of displaying past statuses. This integration of inequity theory and the ELM provides a unique lens to understand the dynamics of intrinsic and perceived quality in online reviews.
Managerial Implications
Our study presents several important and relevant insights for practitioners who develop and manage the quality of content for UGC platforms. First, our empirical evidence not only is statistically significant but also carries meaningful economic implications. For example, our main results demonstrate that demoted reviewers tend to write reviews with significantly lower quality. Such an impact has direct and negative implications for a platform’s welfare, as reported by Chan et al. (2022). Meanwhile, our results on the discrepancy between perceived review quality and intrinsic review quality point to potential long-term negative implications for customer satisfaction (Hong et al., 2018), which directly impacts platform economics (Amorim and Pratas, 2022).
Second, our empirical findings highlight the potential negative consequences of displaying past badges on profiles of demoted users within a transient, status-based, user recognition system. Such a system can result in consumers perceiving content generated by demoted users to be of better quality than it is, leading to reduced consumer satisfaction with the platform in the long term and impacting its welfare and sustainability. Thus, our results suggest that platform managers should reconsider policies that entail displaying the past statuses of demoted reviewers.
Third, our findings underscore the importance of potential design mechanisms and intervention strategies to platform managers who must address issues of status loss. For example, offering incentives such as participation in social events, exclusive invitations, or a dedicated forum to address perceptions of inequity (similar to the support forum in Meta Stack Overflow by Bhattacharyya et al., 2020) could motivate demoted reviewers to generate higher-quality reviews. Moreover, our findings demonstrate that even though demoted users tend to produce lower intrinsic quality reviews following the loss of their statuses, the perceived quality of their reviews remains unchanged. Therefore, consumers on UGC platforms should be made aware of this possible bias (e.g., offer them intrinsic quality metrics of reviews), so they can make well-informed decisions (Goes et al., 2014). Alternatively, platforms could display intrinsic quality metrics to assist users during their content consumption (e.g., Hou and Ma, 2022).
Fourth, insights from our study can be used by operation managers in UGC platforms to develop predictive analytics frameworks for identifying potential reviewers of interest. In Section E of the E-Companion, we demonstrate how a simple predictive analytics framework can be developed using a scalable, off-the-shelf, machine learning model that can consistently identify, more than a year before a reviewer loses status, whether the reviewer is likely to reduce the quality of his/her reviews after demotion. By proactively identifying such reviewers, appropriate actions can be taken in the form of incentives or retention mechanisms to alleviate potential adverse impacts that the platform may face in the future. Depending on the platform’s objective, a similar predictive tool can be developed by leveraging the insights provided in this study.
Limitations and Future Research
This article is subject to certain limitations, which, simultaneously, offer an excellent avenue for future research. First, our findings are based on observational data from Yelp, a platform that is known for online restaurant reviews. Future studies could examine whether other factors (e.g., culture and content type) moderate outcomes. For example, research could be conducted on platforms that operate in different cultural contexts (e.g., Asian platforms) or have different types of content (e.g., Q&A platforms). Second, it would be interesting to study how status loss can be designed in ways that can motivate the demoted reviewers to work harder to regain their statuses. Third, although we have framed this study as a quasi-experimental study to enhance its external validity, there are still opportunities to decipher more with respect to internal validity and underlying mechanisms. Furthermore, qualitative research methodologies such as focus group interviews or surveys could be employed in future studies to explore the relationship between intrinsic review quality and perceived review quality in the context of status loss. These approaches would provide valuable insights to extend the literature. Similarly, accessing review writers’ thought processes to obtain insights on the underlying psychological drivers of demoted users (e.g., whether demoted users learn to change their behavior post-demotion) would also be an excellent avenue for future qualitative research. Last, future research may examine additional variables, such as the characteristics of the business units receiving reviews, as potential moderators that influence the impact of status loss on review-generating behavior.
Supplemental Material
sj-pdf-1-pao-10.1177_10591478241279801 - Supplemental material for Status Downgrade: The Impact of Losing Status on a User-Generated Content Platform
Supplemental material, sj-pdf-1-pao-10.1177_10591478241279801 for Status Downgrade: The Impact of Losing Status on a User-Generated Content Platform by Vandith Pamuru, Wreetabrata Kar and Warut Khern-am-nuai in Production and Operations Management
Footnotes
Declaration of Conflicting Interests
Funding
ORCID iD
Supplemental Material
).How to cite this article
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
