Abstract
Keywords
Introduction
Information technology has become a central part of the lives of the world’s communities, not only in developed countries, but also in developing ones such as Indonesia (Rachmawati et al., 2021). As of January 2022, the Internet penetration in Indonesia was 73.7%, which is about 204.7 of the 277.7 million people in the country (DataReportal, 2022). Internet access using a web browser through desktops has steadily declined over the years, and as of February 2022, Internet access from web browsers through mobile phones had increased to 68% of total Internet access (Statcounter, 2022).
Increased use of the Internet through mobile phones or smartphones cannot be separated from the development and penetration of mobile phones. Modern mobile devices, especially smartphones and tablets, are no longer used only for voice communication, but also for completing online payment transactions (Tew et al., 2021). This situation also exists in Indonesia, as smartphone penetration in Indonesia has reached 133% (DataReportal, 2022). The growth in the number of smartphone users drives the development of e-commerce. The increasing popularity of smartphones in the last few decades has introduced consumers to commerce services that are available anytime and anywhere.
These statistics indicates abundance of opportunities for the development of mobile Internet consumption and mobile applications in Indonesia. An increase in middle-class purchasing power and the rapid adoption of technology are also driving online payment trends in Indonesia (Google et al., 2021). This condition has led analysts to predict that Indonesia continues to be one of the most vibrant digital financial markets in the world. It is also predicted this e-commerce trend will positively impact mobile payments, where user behavior will shift to mobile payments using a mobile wallet or mobile bank account (Ryza, 2015).
In Indonesia, the development of mobile payment can be observed from the intense competition between mobile payment providers. From recent data, two mobile payment applications have the highest user penetration: Go-Pay and OVO (Rahardyan, 2022). Since 2020, the market seems to favor Go-Pay over OVO (Mulia, 2020). Go-Pay, an online payment service provided by the parent company and application Go-Jek, initially started as an online transportation or ride-hailing company. Go-Jek is now engaged in various areas of business, including logistics and mobile payment in Indonesia. Go-Pay acts as a digital wallet that can be topped up from and integrated with local banks and convenience stores. Exclusively accessible from the Go-Jek application, Go-Pay enables users to pay not only online merchants on e-commerce websites or applications, but also offline merchants such as street food vendors. For users with verified identity, Go-Pay enables users to transfer money deposited in their wallets to other users. With such features, Go-Pay managed to obtain a 60% market share as of 2020 (Cordon, 2020).
Seeing the success of Go-Pay as a mobile payment service, the opportunities in the emerging financial technology industry attracted massive investments from other big technology companies such as Google, Tencent, and JD.com. At present, mobile payment services with the most users in Indonesia are Go-Pay, T-cash (rebranded as LinkAja as of 2019), PayPro, OVO, Mandiri e-cash, XL Cash, and Sakuku. Among these competitors, Go-Pay offers a complete user experience with a broad coverage of mobile payment services.
User experience is an important concern for application development. A good user experience is expected to maintain the emotions of its users (Hsu & Chen, 2018). User emotion needs to be considered in technology development because the decision to continue using technology post-adoption also involves user emotions (Chea & Luo, 2008). Current technology is expected to provide emotive connections with its users so that in the long term, customer engagement and loyalty from its users can be generated (Chea & Luo, 2008; Tarute et al., 2017). The ability to engage consumers is needed to capture and maintain market share in a competitive market (Kim & Baek, 2018; Rasool et al., 2020; Tarute et al., 2017). For this reason, mobile payment service providers need to understand the concepts and theories of emotions to use the users’ emotions in increasing their loyalty to the mobile payment services offered.
Although the user experience factor is essential in the formation of user loyalty when using mobile payment technology, unfortunately, not many studies have reviewed the impact of the user experience on human psychology, especially regarding emotions. This encouraged us to investigate the psychological processes built from the user experience and their impact on emotions, customer engagement, and loyalty in the context of mobile payment. In particular, we focused on two attributes of user experience: application design and application performance. Both of these attributes are regarded as critical external stimuli influencing the users’ psychological involvement, leading to their behavioral engagement (Fang et al., 2017).
In this study, we chose to investigate the Go-Pay mobile payment service since the company has experienced significant development compared to other mobile payment services in the past few years. The company often introduces innovations that offer convenience for its users. In addition, Go-Pay is a relatively new mobile payment service with currently the largest user base, so it is crucial to assess the factors that influence the continuance intention of these users. The emergence of Go-Pay’s competitors, which provide attractive offers and services, might reduce user intention to continue using Go-Pay. Therefore, the purpose of this study was to investigate the factors that influence user intentions to continue using Go-Pay, focusing on the application’s design and performance.
In this section, we also review related works that serve as the theoretical background of this study. The review includes factors influencing mobile application’s continuance intention. Subsequently, we propose a research model and advance corresponding hypotheses.
In general, mobile payment is defined as a cashless payment method using a mobile device to pay for goods, services, or bills. Mobile payment, or m-payment, is a payment activity using a mobile device following a process that starts from payment initiation to confirmation (Abrahão et al., 2016). Mobile payment is a natural evolution of electronic payment, which allows users to pay for goods and services with mobile devices wherever they are (Tew et al., 2021). The mobile payment process allows the purchase, payment, or transfer of certain values through mobile devices without cash transactions (Abrahão et al., 2016; Oliveira et al., 2016). Using a smartphone to make payments is expected to be favored in the future compared to using cash or credit cards. Mobile payment has quickly developed and has become a global trend in recent years.
Several studies have reported that users’ emotions play a significant role in decision making, including mobile payment (Cernea & Kerren, 2015; Ou & Verhoef, 2017; Rychalski & Hudson, 2017). Emotion is a subjective experience that depends on the context of its emergence (Arguedas et al., 2016). Empirically, user emotions directly influence user satisfaction and loyalty to continue using products or certain services (Ou & Verhoef, 2017; Rychalski & Hudson, 2017). Therefore, it is important to make emotional connections with customers as an effective strategy and be different from other competitors to increase customer loyalty, which can be formed through relationships that are built by involving emotions (Chea & Luo, 2008).
Studies of customer emotions can be classified into two types: incidental emotion and integral emotion. Incidental emotion is an emotion that is formed from the user’s mood, whereas integral emotion is an emotion that is relevant to the decision object, such as consumption emotions and emotions that arise from advertising. In this study, we focused on integral emotion since our purpose was to discuss how users’ emotions affect their loyalty or continuance intention to use a certain product or service (Ou & Verhoef, 2017).
Continuance intention is an essential aspect that determines the long-term success of an online service (Mouakket, 2015; Pereira et al., 2015), including mobile payment service (Liébana-Cabanillas et al., 2019). The increasingly intensive development of the mobile payment industry necessitates mobile payment service providers to maintain their market to survive in the industry (Pal et al., 2021; Tew et al., 2021). Thus, mobile payment service providers need to know the factors that influence user decision to continue using their mobile payment service (Oghuma et al., 2016). User intention to use mobile payment might be influenced by the perceived value of the service, which increases as the number of users grows, that is, network externalities (Qasim & Abu-Shanab, 2016). Research on continuance intention is generally conducted to investigate user behavior toward information systems in the context of post-adoption (Mouakket, 2015; Oghuma et al., 2016) to identify the factors that influence user continuance intention of an information system. In an expectation confirmation model (ECM) study Lee and Kwon (2011) categorized the factors that influence user continuance intention into cognitive factors and affective factors. Cognitive factors are related to mental processes in learning while affective factors are related to specific emotions or states of feelings.
Another study (Chea & Luo, 2008) described the relationship between cognition–emotion interaction, satisfaction, and post-adoption behavior in the consumer retention model of e-service use. The aim was to explain the determinants of three post-adoption behaviors: recommendation intention, complaint intention, and continuance intention, which are used to understand and connect emotionally with customers. A model was formulated using expectations confirmation theory (ECT), which added an affective construct: positive affection and negative affection. The construct was added to measure customer loyalty based on emotions arising from customer’s experience with mobile payment. To measure the positive and negative emotions of respondents, the researchers used the positive affective and negative affective schedule (PANAS), which consists of 10 items each of positive and negative emotion expressions. The results of the model showed that the negative affection related to the use of e-services directly predicted compliant behavior, but positive affection and negative affection did not affect customer satisfaction. The researchers found that the interaction between perceived usefulness and positive affection was not significant. The interaction between perceived usefulness and negative affection with respect to satisfaction was also not significant. They proposed the best method for e-service providers to address the three post-adoption behaviors by maintaining customer satisfaction, as satisfaction leads to higher continuance intentions, fewer complaints, and more customer recommendations (Chea & Luo, 2008).
Another study described uses of customer data with 102 companies in eight different industries in the Netherlands (Ou & Verhoef, 2017). To measure users’ positive and negative emotions, the researchers asked the customers the extent to which they felt six specific emotions based on their experience as corporate customers. The measurement items for positive emotions included happiness, joy, and enthusiasm; for negative emotions, the measurement items were anger, regret, and distrust. From the analysis of the data, they found that positive and negative emotions had additional effects on loyalty intentions: positive emotions weakened positive links (negative interactions), and negative emotions strengthened positive links for brands and relationship equity (positive interactions) (Ou & Verhoef, 2017).
In addition to user emotions, mobile application features may influence engagement, which leads to the application’s continuous use (Tarute et al., 2017), resulting in higher engagement, and can direct users to continuously use mobile applications. In addition, consumer engagement positively influences user intentions to reuse mobile applications, whereas consumer interaction and functionality features do not positively influence consumer engagement with mobile applications (Tarute et al., 2017).
In a study conducted in the context of mobile travel applications, Fang et al. (2017) explored how application attributes, namely, application design and application performance, stimulate user engagement. These two attributes were included as multidimensional constructs consisting of application design attributes, encompassing three sub-attributes (compatibility, complexity, and relative advantages) and app performance attributes, comprising three sub-attributes (user interface attractiveness, privacy and security, and portability). Based on the stimulus organism response (SOR) model, these sub-attributes were found to function as a stimulus or signal to trigger psychological engagement and benefit evaluation (utilitarian, hedonic, and social benefits). The data were then analyzed using covariance-based structural equation modeling techniques to test structural models. The results showed that two application design features (user interface attractiveness and privacy and security) as well as three application performance sub-attributes (compatibility, ease of use, and relative advantage) were important drivers of user behavioral engagement for mobile travel applications, whereas psychological engagement and three types of benefit perception (utilitarian benefits, hedonic benefits, and social benefits) positively affected user behavioral engagement (Fang et al., 2017).
This literature review illustrates the studies on factors that influence mobile application’s continued use in various contexts. It might be instructive to explore how the same attributes and factors affect customer behavior with regard to mobile payment use in Indonesia. The findings from the literature review guided us to construct a conceptual research model for studying the impact of a mobile payment user experience on consumers’ emotions and, consequently, continuance intention.
The research model for this study was developed using theories and concepts from previous research. The factors considered in the proposed research model were designed and selected from previous research (Chea & Luo, 2008; Fang et al., 2017; Liébana-Cabanillas et al., 2021; Ou & Verhoef, 2017; Tarute et al., 2017). We compiled the application attributes that measure the aspects of positive and negative emotions of users based on previous research that analyzed the factors driving consumer engagement using mobile travel applications using the application design and application performance attributes (Fang et al., 2017). Each application’s design and performance has sub-attributes, namely, user interface attractiveness, privacy and security, and portability for application design attributes, as well as compatibility, ease of use, and relative advantages for application performance (Fang et al., 2017).
In this study, however, we did not include portability in application design because not all mobile payment users have more than one device with a different system to operate the mobile payment application. Subsequently, we added the convenience sub-attribute to the model. Convenience was drawn from a study by J. Park et al. (2010), who analyzed the intention to use mobile payment based on the ease of use of the mobile payment system (MPS). In the study, MPS consisted of mobility, reachability, compatibility, and convenience. For application performance attributes, we added service quality sub-attributes. The service quality sub-attributes were added since they are considered the most important predictor of the use of mobile payment services (Liébana-Cabanillas et al., 2019; Oghuma et al., 2016).
We integrated application attributes with valence emotions, namely, positive emotions and negative emotions, into the model. The goal was to determine how the mobile payment application attributes affect user psychology, especially in terms of emotions. In this study, we used positive and negative emotions by referring to previous research (Chea & Luo, 2008), which separated negative from positive emotions since these two emotions are mutually independent dimensions (Uddin et al., 2014). Each of the positive and negative emotions is related to the variables of continuance intention and consumer engagement. This combination is based on Tarute et al. (2017), who showed that maintaining a long-term relationship with consumers requires an increase in consumer engagement. The research model proposed for this study is shown in Figure 1.

Proposed research model.
The hypotheses formulated for this study were thus based on factors included in the proposed research model, which are briefly explained in the following sub-sections.
Relationship Between Consumer Engagement and Continuance Intentions
Kim & Baek (2018) stated that there is a strong connection between engagement and the ability to earn profits from the following benefits, such as increasing positive attitudes, commitment to the brand, as well as increasing purchases (McLean, 2018). When mobile payment service providers succeed in increasing user engagement, they can significantly increase the chances of successful mobile payment services (Kim et al., 2013). For this reason, mobile payment service providers need to pay attention to the importance of consumer engagement in developing and retaining loyal customers while attracting new customers (Kim & Baek, 2018). Tarute et al. (2017) defined consumer engagement as the intensity of participation and individual connections with various mobile payment services offered by service providers. The effect of the features of the mobile payment application on consumer engagement leads to continuance intention. From this study, they found that consumer engagement positively affects user intentions to continue using mobile payments. This is inline with another study on NFC-based mobile payment, which found consumer engagement to be a significant antecedent of continuance intention (Liébana-Cabanillas et al., 2021). We thus hypothesized the following:
H1:
Relationship Between Positive and Negative Emotions and Continuance Intention
Emotions are closely related to loyalty (Rychalski & Hudson, 2017). Several studies have found that positive emotions have a positive direct relationship to loyalty (Rychalski & Hudson, 2017). Ou and Verhoef (2017) in their research proves that the positive and negative emotions that are generated gradually affect the continuance intention of using mobile payments. The results of this study are supported by Chaparro-Peláez et al. (2015) in his research which aims to measure the role of emotion and trust in service recovery in B2C e-commerce. The results showed that positive and negative emotions had a positive effect on continuance intention. More specifically, Chaparro-Peláez et al. (2015) stated that positive emotions when using mobile payments have a stronger effect on continuance intention to use them compared to negative emotions. Research conducted by Razzaq et al. (2017) show in more detail the influence of each positive and negative emotion. This research which aims to study the contribution of emotions to loyalty intention shows that positive emotions have a positive effect on continuance intention and negative emotions have a negative effect on continuance intention. This also applies to Go-Pay users who always experience positive emotions because of the convenience offered by Go-Pay services, it will affect the user’s decision to use Go-Pay on an ongoing basis. Therefore, we propose the following two hypotheses:
H2a:
H2b:
Relationship Between Positive and Negative Emotions and Consumer Engagement
Based on a study conducted by Martínez-López et al. (2017) who studied consumer engagement in online brand communities, showed that there was a positive relationship between positive emotions and consumer engagement. When customers experience and share positive experiences, a strong emotional bond and identity are formed between customers and mobile applications (Laurence et al., 2015). The bond formed will build customer confidence in the mobile payment service. A higher level of trust will build relationships that are no longer based on cognition but emotions. Martínez-López et al. (2017) added that consumer engagement increases with increasing emotional relationships and limits cognition relationships with online services. This statement is in line with research conducted by Arguedas et al. (2016) exploring more specifically about the effect of emotions on consumer engagement in their study. Arguedas et al. (2016) showed that the more often positive emotions are formed, the more consumer engagement will increase, inversely proportional to negative emotions which result in lower consumer engagement. Hence, we hypothesized the following:
H3a:
H3b:
Relationship Between Attractiveness User Interface and Positive and Negative Emotions
An attractive user interface can encourage users to form positive emotions such as being attracted, fascinated, etc. Emotion is a factor that connects the user interface with the ability to operate mobile payment applications. Kumar et al. (2016) emphasized the importance of esthetics in an application because a pleasant esthetic can change someone’s emotional state and change the assessment of the mobile payment application. In other words, paying attention to aspects of the user interface can increase the user’s positive emotions (Kumar et al., 2016). The design of an easy-to-use interface will help users operate mobile payments with simple steps and create easier interaction between users and the mobile payment application. Positive interactions will also create positive emotions (H. Park & Song, 2015), contrary to the negative emotions formed due to negative interactions with the user. Kumar et al. (2016) stated that the user interface aspect has an impact on emotions that are formed based on user perceptions and depend on user ratings of the Go-Pay mobile payment user interface. Kumar et al. (2016) also emphasized the importance of esthetics in applications because pleasing esthetics can change a person’s emotional state and change the assessment of mobile payment applications. In other words, paying attention to aspects of the user interface can increase the positive emotions of users. Consequently, we formulated the following hypotheses:
H4a:
H4b:
Relationship Between Privacy and Security Attractiveness and Positive and Negative Emotions
Privacy is a complex and multifaceted concept that involves emotions. Emotion plays an important role in explaining the privacy paradox. Users can forget their level of concern for privacy and disclose their personal data when the interactions with the mobile payment service are pleasant (Li et al., 2017). The high concern for user privacy is reflected in the low openness of the user’s personal information (Ortiz et al., 2018). Any doubts about the user’s personal data storage and inadequate security will make customers reluctant to provide personal information on the mobile payment application. In particular, when customers directly feel that their data are insecure and may be accessed by outsiders without their permission, a poor assessment of service providers is created and negative user emotions will form, such as disappointment, worry, and anxiety. Go-Pay application users who lose confidence in the security of their personal data will experience emotions that further reduce their intention to use the mobile payment service (Ortiz et al., 2018). Consequently, we formulated the following hypotheses:
H5a:
H5b:
Relationship Between Convenience and Positive and Negative Emotions
Several studies have identified convenience as one of the important factors affecting the success of a mobile application, including mobile payment. However, studies that examine the effect of convenience on consumers’ emotional responses are still limited. Various Go-Pay m-payment services can be enjoyed anywhere without time and location constraints, with access and updated information in real-time, complete with the history of each transaction, which provides convenience with respect to the user’s cash flow information. The increased convenience offered by the mobile payment can increase positive emotions such as pleasure and satisfaction (Min et al., 2012). Conversely, if the service used becomes a burden for consumers, such as when making transactions, the mobile payment application runs slowly, payment transactions fail, or top-up balances do not enter the system, consumers may be impatient and ultimately get upset, creating negative emotions. For the same service, customers prefer mobile payment that offers increased convenience. Therefore, we propose the following hypotheses:
H6a:
H6b:
Relationship Between Compatibility and Positive and Negative Emotions
Compatibility describes the degree to which Go-Pay users consider payments through Go-Pay to be appropriate for their payment needs. Compatibility reflects consumers’ perceptions of mobile payments that meet their payment needs (Kuan-Yu & Hsi-Peng, 2015; Zahidul et al., 2013). The higher the level of compatibility felt by the user, the more user psychology is affected. This can be reflected in the payment transactions sent to the right target with the right amount, which can be checked through the payment history. The success of the transaction will produce emotional expressions such as relief and satisfaction, which contrasts with the conditions that cause anxiety for consumers, such as recipients of inappropriate transactions, payment failures, difficult to access services, and so on. Diverse customer perceptions and expectations of the services offered by Go-Pay do not always produce the same emotions. Consumers may expect more from the services offered, such as a notification when topping up the balance is successful, and so forth. The expression of consumers’ emotions toward compatibility also depends on the perceptions and payment preferences that are expected of the mobile payment service. Thus, we propose the following hypotheses:
H7a:
H7b:
Relationship Between Ease of Use and Positive and Negative Emotions
Ease of use is the opposite of complexity, which describes the level at which Go-Pay is accepted as a product that is not difficult to use or does not require significant effort when using it. Complex mobile payment services will require more effort to operate. Complexity will make it difficult to understand the services offered and affect the overall user evaluation and user experience. In general, ease of use describes the level of ease of users to run Go-Pay mobile payments. Research to determine the relationship of ease of use with emotions has been performed by Éthier et al. (2008), who demonstrated that the evaluation of usability, including ease of use, has a positive relationship with positive emotions and a negative impact on negative emotions. Therefore, we propose the following hypotheses:
H8a:
H8b:
Relationship Between Relative Advantage and Positive and Negative Emotions
Relative advantage refers to the degree to which users consider a mobile service to be superior to other mobile payments (Fang et al., 2017). In our case, relative advantage is the result of innovations offered by mobile payment Go-Pay. The size of the benefits obtained can be tangible or intangible, such as functional and personalization levels. Personalization of the mobile payment service can increase the curiosity of users, who can further explore the mobile payment service offered, for example, the Go-Pay transfer function through a cellphone number that can be accessed directly through contact, the ticket booking watch function with payment through Go-Pay without having to arrive a few hours before the movie starts to ensure a ticket is available, the top-up credit function through Go-Pay, and others. This situation influences the evaluation of the user experience to be more positive. The existence of service restrictions will trigger negative user emotions, such as the existence of a maximum balance filling and so forth. In this regard, we propose the following hypotheses:
H9a:
H9b:
Relationship Between Service Quality and Positive and Negative Emotions
Service quality is the user perception that distinguishes between user expectations and the actual performance of the system within the scope of service quality. Quality of service is closely related to the performance of mobile payment services (Oghuma et al., 2016). Good service quality can be reflected in the ability of the mobile payment service to deliver the promised service precisely, reliably, quickly, and accurately in real-time, as well as the ability to create customer confidence and confidence when making transactions. Service quality that is judged to be good by consumers will lead to satisfaction because the services provided match with consumer expectations, which gradually creates positive consumer emotions (Liébana-Cabanillas et al., 2019). Therefore, we propose the following hypotheses:
H10a:
H10b:
All proposed hypotheses are portrayed in the context of proposed constructs in the research model illustrated in Figure 1.
Methods
As quantitative research, this study comprised 10 steps, as shown in Figure 2. This section specifically explains the processes of research instrument development, data collection, and data analysis.

Steps followed for completing the study.
Measurement and Data Collection
This research quantitative instrument was developed based on pre-validated constructs from existing literature, comprising two parts. The first part included questions about the personal data and demographics of Go-Pay mobile payment service users (Supplemental Appendix C). The second part contained statements that represented the research instruments obtained from the results of modifications to previous research instruments (Supplemental Appendix D). Each statement provided a choice of answers on a 1 to 5 Likert scale ranging from strongly disagree to strongly agree. This scale was used to facilitate quantitative data processing. The measurement items were then subjected to a readability test by nine respondents. The feedback received from these respondents was subsequently used to improve the measurement items.
The variables used in this study were continuance intention, customer engagement, positive affection, negative affection, user interface attractiveness, privacy and security, convenience, compatibility, ease of use, relative advantage, and service quality. These variables were divided into two groups of variables: exogenous and endogenous variables. The independent variable that influences the dependent variable is called the exogenous variable; the dependent variable that is affected by the independent (exogenous) variable is called the dependent variable (Hair et al., 2014). In this study, endogenous variables were continuance intention, customer engagement, positive affective, and negative effective. The exogenous variables were user interface attractiveness, privacy and security, convenience, compatibility, ease of use, relative advantage, and service quality.
The population in this study was the users of Go-Jek online transportation services (i.e., ride hailing, grocery shopping, parcel delivery, and food delivery) who had used Go-Pay as a payment method. Go-Jek users were chosen as population as the Go-Pay was only available exclusively on the Go-Jek mobile application. The technique used for sample selection in this study was purposive sampling, based on certain considerations such as population characteristics or characteristics that were previously identified. As a representative sample of the population is not expected for purposive sampling, generalizability of this study’s conclusions is limited. This technique was used because the sample criteria were defined previously: Go-Jek online transportation service users who have used Go-Pay as a payment platform. These criteria may introduce bias to the results due to the self-selection nature of the non-probability sampling technique such as purposive sampling.
Data were collected by distributing online survey questionnaires. These questionnaires were completed by the population with predetermined criteria. The questionnaire was distributed online through social networks such as Twitter, LinkedIn, and Facebook, as well as instant messaging such as LINE and WhatsApp. To attract prospective respondents to complete the questionnaire, we offered prizes in the form of a Go-Pay balance or credit, with a total amount of Rp 300,000.00 for 10 winners. We also requested assistance from online influencers who actively use social networks to help distribute the research questionnaires. In addition, we used online advertisements on the LINE instant messaging platform to promote and distribute the questionnaires.
With the help in distributing questionnaires by influencers and advertisements, we managed to obtain respondent data from various ages and diverse occupations in a relatively short time. Data were collected for 4 weeks from March to mid-April 2018. From the data collection, 1,505 responses were collected, which were then subsequently cleaned for duplicates and incomplete responses, resulting in 1,275 responses to be processed and analyzed. Table 1 describes the characteristics of the respondents based on age, sex, occupation, earnings per month, the level of frequency of use, and the duration of use of Go-Pay.
Characteristics of the Sample.
Data Analysis Process
In the process of quantitative data analysis, we used a covariance-based structural equation modeling (CB-SEM) technique using AMOS (version 22.0) software to test and confirm the theory formed in previous studies (Waheed & Khader, 2021). In addition, the model proposed in this study has a type of measurement model that is considered reflective (Reinartz et al., 2009). This is indicated by the one-way variable relationship with the instrument in this study (Byrne, 2013). The use of the CB-SEM approach instead of partial least square structural equation modeling (PLS-SEM) is based on research objectives that we would like to test and confirm the theory based on previous research. In addition, this study used a model that involves reflective constructs; hence it is considered appropriate to use CB-SEM as an approach in processing and analyzing data (Hair et al., 2011).
To analyze data using AMOS, the stages were divided into two major steps: the measurement model and the structural model tests. The measurement model test is a stage to precisely identify how latent variables explain manifest variables. At the measurement model stage, we conducted a construct reliability (CR) test, determined Cronbach’s α (CA), conducted a convergent validity (average variance extracted [AVE]) test, a discriminant validity test (AVE root), and loading factor analysis (see Supplemental Appendices A and B). At this stage, we conducted a goodness-of-fit (GOF) test to explain the relationships between the variables in the study.
After all tests on the measurement model were conducted and validated, we conducted a structural model test. Structural model tests are performed to test the overall model fit and the parameter estimates (the relationship of independent variables contained in the structural model). At this stage, the GOF test was repeated to ensure clear relationships between variables that were modified from the measurement model, followed by a hypothesis test (supported if the
Results
In this section, we evaluate the results of the data analysis—specifically, the measurement model test, structural model test, and hypotheses test.
Measurement Model Results
The measurement model is a part of the SEM, which consists of latent variables (constructs), where each latent variable has several manifest variables that explain the latent variable. The measurement model validity test aims to precisely identify how the manifest variables can explain latent variables. The measurement test is divided into several processes, including the reliability test, convergent validity test, and discriminant validity test.
The reliability is confirmed from the values of construct reliability or composite reliability (CR) of >0.7 and CA values of >0.7 for each variable. The CR and CA test results are provided in Supplemental Appendix B. From the calculation of the reliability value, we found that all variables met the value specified as a minimum value CR and CA based on the loading factor of >0.7. Thus, the research data met the internal consistency reliability test, with CA and CR values >0.7.
The construct validity test (latent variable) consists of two tests: convergent validity and discriminant validity. The convergent validity test is determined from the loading factor value and the average variance extracted (AVE) value. An indicator is considered feasible if it has a loading factor value >0.7. Indicators with loading factor values <0.7 must be eliminated from the research model. The discriminant validity test can be observed from the cross-loading value of each indicator, showing that the load of each indicator of the variable is greater than the cross-loading with other factors. The results of discriminant validity testing are provided in Supplemental Appendix A.
We tested the convergent validity by first calculating the loading factor value and ensuring that all variables had a loading factor >0.7. To obtain an indicator meeting these provisions, we iterated loading factors 21 times. In the 21 iterations, we eliminated indicators with a loading factor <0.7 as many as 18 times; three times, we set the indicator variance error value to 0.01. These steps were applied with reference to the recommendations (Shevlin & Miles, 1998). The indicators that were eliminated were User Interface Attractiveness (UIA) 1, UIA 2, UIA 3, UIA 4, Service Quality (SQ) 3, Positive Affective (PA) 2, PA 7, PA 8, PA 9, Negative Affective (NA) 10, Ease of Use (EOU) 5, Convenience (CON) 3, Continuance Intention (CI) 7, CI 5, CI 2, CE 2, Customer Engagement (CE) 1, and Relative Advantage (ADV) 2. The error variance value of 0.01 was applied to the indicators UIA5, UIA6, and UIA7. A summary of the loading factor iteration is provided in Supplemental Appendix A. The results of the measurement model testing process resulted in a change in the shape of the research model by removing several indicators that did not meet the loading factor requirement of >0.7.
After ensuring all indicators had a loading factor value >0.7, we then calculated the AVE value, which is the average value of the total square of all loading factors for each variable. We then checked that all variables had an AVE value of >0.5. After being fulfilled, we proceeded to discriminant validity testing. The discriminant validity test results are provided in Supplemental Appendix A.
Subsequently, GOF testing was carried out in three parts: absolute fit indices, incremental fit indices, and parsimony fit indices. Following assessment of the measurement model and evaluation of its results, data analysis continued with an assessment of the structural model to evaluate the hypothesized relationships between constructs and the predictive capacities of the conceptual model.
Structural Model Testing
Structural model testing aims to determine how to fit the research model to the data obtained. The level of compatibility of the model with the data is measured by the goodness-of-fit (GOF). GOF is a measure of how well the research model mimics the covariance matrix among existing indicators. A model is said to be valid if it meets the GOF, indicating it can explain the relationships between variables in the research model. GOF consists of several testing criteria: absolute fit indices, incremental fit indices, and parsimony fit indices. The initial GOF test results of this study are provided in Table 2. From these results, we found that the values for the RMR, PRATIO, PNFI, PCFI, RMSEA, and HOETLER criteria met the GOF criteria, whereas the other criteria were in the poor fit range, which indicated that the research model less accurately fit the data obtained. For this reason, we modified the research model to meet the GOF value.
Initial Test of the Research Model’s Goodness-of-Fit (GOF).
Based on Table 2, almost all testing criteria were in the poor fit range. This indicated that the research model was not yet compatible with the data obtained. Hence, we modified the research model in each group of data to reduce the chi-square value, as a smaller chi-square value indicates the model better fits the existing data. Modifications were conducted by combining covariance between factors, indicators with error covariance, and between-covariance errors. The combination was based on the highest modification indices value on the list of combinations recommended by AMOS.
In this study, we modified eight iterations. The results of the modification of the research model are provided in Table 3 and Figure 3. From these results, we found that all measurement criteria were in the range of good fit, so the research model fit the obtained data.
Final Test of the Research Model’s GOF.

Final model for the research regression.
Based on Table 4, the
The Continuance Intention (CI) factor has an
Hypothesis Testing
The hypotheses of the model were tested in two directions (two-tailed) because the influence of one variable on another variable was not positively or negatively directed toward the factors that affected or influenced it. The

Final research model.
Hypotheses Test Results.
The results showed that eight hypotheses were not supported while 11 hypotheses were supported. This suggests that statistically the evidence was not sufficient to support the proposed hypotheses. These findings may be attributed to almost half of the respondents (45.88%) being Go-Pay mobile payment users that rarely use the application, only a few times a month. This could have affected the results of this study because emotions are a medium-term affective state characterized by responses to the experience of using Go-Pay. The experience of using Go-Pay might be influenced by the level of frequency of using Go-Pay. Users who rarely use mobile payment applications will have lesser emotional ties compared to users who often interact with mobile payment applications. This can cause users to lack significant emotional experience related to the use of mobile payment.
Discussion
The results of the data analysis indicate that not all attributes of the mobile application have a significant effect on the user’s positive and negative emotions. With regard to application design, all of the application design sub-attributes (i.e., user interface, privacy and security, convenience) of the Go-Pay mobile payment have no effect on the positive emotions of users when using the mobile payment service. This result is consistent with the results of previous research reporting that the esthetic level of the mobile app design does not impact positive emotions. This finding may be due to the long-term use of the application, where many users have used Go-Pay mobile payments for a long period of time, so the design cannot produce a strong positive response (Bhandari et al., 2017).
Of all the design attributes and application performances tested, compatibility was the sub-attribute found to most strongly influence the user’s positive emotions. This result is line with research conducted by Fang et al. (2017). On the other hand, application design has a significant effect on negative emotions in contrast to application performance, negative emotions are only influenced by the relative advantage attributes of application performance.
Regarding the effect of factors of Go-Pay performance on positive or negative emotions affecting users’ continued use of the application, contrary to application design, almost all application attributes positively affect and support the user. Of the four sub-attributes, compatibility, ease of use, relative superiority, and quality of service, only ease of use does not have significant positive effects. Several studies have compared ease of use during pre- and post-adoption of information systems and found that ease of use results tend to be inconsistent with regard to attitudes on pre-adoption and becomes insignificant in post-adoption installations (Yang & Bahli, 2015).
Other factors tested from this study are the influence of positive affective (positive emotions) and negative affective (negative emotions) on consumer engagement and the continuing intention of using Go-Pay. The test results show that positive emotions and negative emotions affect consumer engagement. Positive emotions have a positive effect on consumer involvement and negative emotions have a negative effect on consumer involvement. In that sense, the higher the positive emotions that are formed, the more consumers will be involved in mobile payment services, and conversely, higher the negative emotions means reducing the consumer’s involvement in mobile payment services.
Every customer has both good and bad experiences in terms of consumer engagement. For example, in an instance where a mobile payment user’s deposit was deducted for an unsuccessful transaction, she or he might respond positively when her or his issue was resolved quickly. This experience would shape the user’s perception from the emotions invoked. Thus, positive emotions from a positive experience will also shape good consumer engagement. From this study, it is revealed that positive emotions have a stronger effect on increasing consumer engagement than negative emotions, which conversely reduce consumer engagement. In this regard, the results of this study are in line with research conducted by Tarute et al. (2017), Martínez-López et al. (2017), and Arguedas et al. (2016).
Furthermore, positive and negative emotions have an influence on the continuance intention. Positive emotions affect consumer engagement positively, negative emotions affect consumer engagement negatively. The results of this study are in accordance with the results of previous studies conducted by Ou and Verhoef (2017), Yi and Hin (2015), and Chaparro-Peláez et al. (2015). Good service can create a positive experience that engages the user’s intention to use a service from the same brand whenever they need a similar service. However, an unpleasant experience while using the service will result in users reluctant to use the service again. A bad experience will get bad feedback as well and have an impact on decreasing loyal customers from mobile payment services. The results also showed that positive emotions had a stronger impact on sustained intent than negative emotions. This is in line with research conducted by Chaparro-Peláez et al. (2015) who proved that positive emotions had a more significant impact than negative emotions on the ongoing intention to continue using mobile payments. In addition, the results show that consumer engagement has a significant effect on intention which indicates that consumer involvement in the use of Go-Pay mobile payments can increase the user’s intention to continue using Go-Pay mobile payments. These results are also in line with research conducted by Tarute et al. (2017).
Research in the field of mobile payment has mostly assessed consumer acceptance of mobile payment technology (Oliveira et al., 2016), whereas research focusing on continuous intention to use mobile payment is still limited (Chiu et al., 2017). Recent existing studies on mobile payment’s intentions focused more on continuance intention’s direct relationships with perceived usefulness (Franque et al., 2021), dissatisfaction, and confirmation constructs (Talwar et al., 2020). To fill the research gap, we examined the research models regarding the continued use of information technology used in previous studies. Our findings provide a new perspective on the loyalty in using mobile payment by examining the part of the user experience represented by the design and performance of applications, which affect positive and negative emotions, and have also been shown to influence user loyalty behavior. Most current research only considered perspectives such as the ECM, which emphasizes satisfaction as the main antecedent (Chea & Luo, 2008; Mouakket, 2015; Oghuma et al., 2016). Therefore, in this study, we focused on the mobile payment domain by investigating user loyalty in terms of application attributes, which are mediated by the negative and positive emotions of users.
In this study, application design is represented by the user interface attractiveness, privacy and security, and convenience sub-attributes; the application performance attributes are represented by the compatibility, ease of use, relative advantage, and service quality sub-attributes. The test results showed that the application design significantly influences negative emotions, and application performance can significantly influence the positive emotions of users. User loyalty, which is represented by the continuous intention and consumer engagement variables, was found to be significantly influenced by both positive and negative emotions. Of the two valence emotions, positive emotions proved to be the most influential in the intention to continuously use Go-Pay mobile payments.
The practical implications of this research for organizations are as follows: Organizations using Go-Pay service providers and other mobile payment service providers can evaluate the attributes of the mobile payment application using the criteria outlined in this study, particularly the criteria that can increase the user’s positive emotions, especially the application performance attributes of mobile payment services. In terms of application performance attributes, for example, service providers can devote resources to improving user perceptions of compatibility, relative advantages, and service quality of mobile payment services offered, such as through marketing campaigns (Fang et al., 2017). Service providers should increase focus on the factors that contribute the most to encouraging the positive user emotions because positive emotions most significantly impact the loyalty of using mobile payments. For this reason, service providers need to pay attention to compatibility factors in mobile payment services, for example, by integrating mobile payment with existing point-of-sales systems to open access to new industries that have not yet been entered by Go-Pay, improving payment efficiency by shortening the payment process, and establishing relationships through engaging content by sharing offers and other benefits through the mobile payment application. These aspects are expected to encourage customers to return to and share their positive experiences with using mobile payment services.
Conclusions
This study was conducted to analyze the factors that influence the intention or desire of users to continue using Go-Pay mobile payment services from the perspective of the positive and negative emotions of users who are driven by the application design and performance attributes. We followed a quantitative approach using online questionnaires administered to Go-Pay mobile payment users. Based on the results of data analysis and research models, we found that the factors influencing user continuance intention of Go-Pay mobile payment services were positive affective, negative affective, and consumer engagement. Of these three factors, positive affective was the construct that most influenced the user’s desire to continue using Go-Pay mobile payments. We also found that application design, consisting of user interface attractiveness, privacy and security, and convenience, did not affect the formation of positive user emotions, but rather affected the negative emotions of users, except for user interface attractiveness. Application performance attributes (i.e., compatibility, ease of use, relative advantage, and service quality) significantly influenced positive emotions, except for ease of use. Of these three attributes that affected positive emotions, we found that compatibility had the strongest influence on positive emotions. However, application performance did not significantly influence negative emotions because only one of the three attributes of application performance affected the user’s negative emotions: relative advantage. It is suggested that mobile payment providers looking to improve customer loyalty consider factors that can increase users’ positive emotions such as compatibility, relative advantage, and service quality.
However, our study has several limitations. First, the distribution of the age and background of respondents did not include users of all ages and backgrounds. Of the respondents, 45.88% were Go-Pay users who did not frequently use Go-Pay (frequency of use: rarely or several times a month), preventing the ability to accurately evaluate the experience with using Go-Pay. In addition, we did not interact directly with the providers and developers of Go-Pay services, so specific information about Go-Pay services was difficult to obtain, such as overall Go-Pay users, the average number of user transactions per month, and so forth. Furthermore, the majority of the respondents were female (60%) whose mobile payment behavioral intentions may be driven by gender differences as suggested by a previous study by Lwoga and Lwoga (2017).
Considering these limitations and research opportunities, we propose a few suggestions for future research on the topic of mobile payments, especially in Indonesia. Specifically, future studies should examine the positive and negative emotions of users using other stimuli in addition to application attributes that can measure emotions more clearly. Subsequent studies should also obtain respondent data with a more even distribution of demographics in the scope of the research object to properly represent mobile payment users.
Supplemental Material
sj-doc-1-sgo-10.1177_21582440231151919 – Supplemental material for The Impact of Mobile Payment Application Design and Performance Attributes on Consumer Emotions and Continuance Intention
Supplemental material, sj-doc-1-sgo-10.1177_21582440231151919 for The Impact of Mobile Payment Application Design and Performance Attributes on Consumer Emotions and Continuance Intention by Untung Rahardja, Claudia Teresa Sigalingging, Panca O. Hadi Putra, Achmad Nizar Hidayanto and Kongkiti Phusavat in SAGE Open
Footnotes
Declaration of Conflicting Interests
Funding
Supplemental Material
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
