Abstract
Keywords
In today’s workplace, service employees increasingly collaborate with technology based on artificial intelligence (AI) to co-produce their work outcomes: recruiters use AI to help screen and decide upon fitting applicants (Marr 2019), analysts need such assistance to decide upon credit loans (Bank, 2021), and consultants develop creative yet viable solutions for their customers using AI (Carl, 2023). Not surprisingly, a recent representative study finds that, across service sectors, 79% of employees were at least somewhat exposed to AI tools, with 22% using them regularly (McKinsey 2023).
While such AI systems generally benefit employees’ service performance (Henkel et al. 2020), they may also have drawbacks, such as inducing threat to meaning of work (Smids, Nyholm, and Berkers 2020), stimulate employees’ overreliance on AI recommendations (Buçinca, Malaya, and Gajos 2021), and decrease the extent to which they want to take responsibility for their service behavior (Santoni de Sio and Mecacci 2021). Clearly, managers need to make decisions on how AI systems should be designed to increase desirable and avoid undesirable employee-AI collaboration outcomes and thereby improve the return on investment in such service technology (Wilson and Daugherty 2018). However, our current understanding of the ideal design and its outcomes is hampered by two shortcomings in the current body of literature.
First, most extant works take a consumer perspective toward human-AI service co-production, rather than an employee perspective, and have consequently focused on outcomes such as service experience (McLeay et al. 2021), use intention (Mende et al. 2019), customer satisfaction (Le et al. 2024), and willingness to share data (Song and Kim 2021). Fine-grained details in the service employees’ outcomes of working with AI (e.g., affect, behavior, and cognition) thus remain underexposed. Second, scholars studying the employee-AI co-production of a service have generally considered the AI system holistically (e.g., Henkel et al. 2020; Mirbabaie et al. 2022; Nazareno and Schiff 2021). For instance, Henkel et al. (2020) focus on emotion regulation of employees using AI versus employees not using AI; the features of the AI system are not manipulated.
A recent study by Le et al. (2024) provides more detail by focusing on cues provided to the customer during collaboration between humans and digital employees. For instance, service providers may explicitly indicate the transfer of the task between AI and human, signal to customers whether the AI or the human is supervising the process, or communicate about a joint team goal (or not). While the authors show that such cues influence customer satisfaction through perceptions of process fluency and team cohesion, the attitudes, behaviors, and cognitions of employees in the collaboration receive little attention. This abstraction may lead to an incomplete picture of reality. To illustrate, employees may be unwilling to take responsibility for the service outcome, or may perceive that their work loses meaning because of AI infusion, which ultimately influences the customer’s experience. In addition, signaling a cue to customers is different from system design. For example, while the cue may be that the employee is supervising the AI, the question remains how this should be designed into the system. Finally, Le et al. (2024) offer limited insights into which cues are most relevant for the optimal outcome, nor do they consider that employees may differ in their AI experience levels.
In sum, thus far there is no comprehensive knowledge on how to design AI systems that optimize the outcomes of service employee-AI co-production. The goal of this study is to address this gap. Specifically, we introduce the concept of a collaborative intelligence (CI) system for service co-production, define its conceptual domain, outline five key features of such a system, and investigate CI’s effects on four essential affective, behavioral, and cognitive responses of employees. Moreover, we spotlight which features of a CI system stimulate favorable and prevent undesirable employee outcomes. We thus answer three central research questions:
1. What are the features that characterize AI systems as CI systems?
2. What effect do CI systems have on different work-related employee outcomes?
3. Which CI system features are most relevant for successful employee-AI service co-production?
In answering these questions, we make the following contributions to literature. First, we contribute to work on human-AI collaboration in service settings by delineating CI’s conceptual domain, identifying the idiosyncratic features of a CI system, and demarcating CI from related constructs and concepts, such as hybrid intelligence (Dellermann et al. 2019, 2021), collective intelligence (Gavriushenko, Kaikova, and Terziyan 2020), and human-AI symbiosis (Jarrahi 2018). By clearly defining CI system features, we provide service scholars and managers with a blueprint for designing and introducing such systems for service co-production.
Second, we conduct a first empirical investigation of our new CI systems concept. We build on the affect-behavior-cognition model (ABC model), which posits that one’s attitude toward an object is expressed as a combination of emotion, behavior, and thought (Breckler 1984). Accordingly, we cover all three ABC outcomes: affect (i.e., threat to meaning of work), behavior (i.e., adherence to the system), and cognition (i.e., perceived service improvement and perceived outcome responsibility). While scholars have outlined human-AI collaboration outcomes on the firm level (e.g., Wilson and Daugherty 2018), the dyadic level (i.e., the quality of the jointly made decisions; Dellermann et al. 2021), and the customer level (e.g., Le et al. 2024), we consider four work-related outcomes on the individual employee level. Substantively, these outcomes likely precede the quality of joint decision-making, which may then influence team, department, or firm-level consequences. By providing insights into CI’s relationship to these individual-level outcomes, we further complete the chain of effects that originates in human-AI collaboration and, ultimately, ends at firm performance.
Third, we empirically investigate which CI system features are most relevant to the four employee outcomes and identify a primary role of the features transparency, process control, and outcome control. While extant work focuses on different system features in isolation (e.g., Westphal et al. 2023) or investigates AI systems holistically without considering the specific contributions of each feature (e.g., Henkel et al. 2020), our research facilitates managers to make more balanced CI design decisions (e.g., to decide on combinations of features). Remarkably, our findings show that reciprocal strength enhancement is not always needed, while the engagement feature did not show any effects in our empirical investigation.
Finally, we add insights to a stream of literature that has concentrated on identifying contingency factors in technology acceptance and use (e.g., Blut, Wang, and Schoefer 2016; Brown, Dennis, and Venkatesh 2010; Park et al. 2014). Specifically, we identify that the effect of CI system features on our employee outcome variables strongly differs between employees who have already used AI systems in their jobs vs. those who did not. This suggests to organizations the need to adapt the CI system’s functionality to their employees’ AI experience.
Our research is organized along two main parts. The first part develops the concept of a multi-feature CI system for service co-production. We conduct a comprehensive literature review and a qualitative study that includes 14 semi-structured interviews. In the second part, we empirically investigate our CI system concept using two studies. In Study 1, we use scenario-based experiments to investigate the effects of CI systems that are weak vs. strong on their identifying features. Gathering data from employees in financial services, we consider their cognitive responses to the system as an initial step toward studying CI’s effects on employees. In Study 2, we flesh out the respective effects of the five identified CI system features on the entire set of ABC outcomes. Specifically, we develop a factorial survey experiment and gather data from HR professionals from different firms. As some participants already use AI systems for recruiting purposes while others do not, this study additionally allows us to contrast employee responses based on their AI experience.
Part 1: Conceptualizing CI Systems for Service Co-Production
Literature Review
Traditionally, service co-production is the production of service outcomes by joint inputs of human frontline employees and customers (Oertzen et al. 2018). With the infusion of AI systems in organizations, another co-production setup is one where employees and AI systems complement each other while working on a joint task (Paschen, Wilson, and Ferreira 2020). We refer to technology in this setup as CI systems. To define the conceptual domain of CI systems, we conducted a comprehensive literature review across scientific fields. Our aim was to gain an overview of how CI and related constructs have been defined and to identify the relevant design features that relate to successful employee-CI co-production (cf. MacKenzie, Podsakoff, and Podsakoff 2011; Walls, Widmeyer, and El Sawy 2004). We aimed to uncover features that are specific to AI technology which differ significantly in their ability to interact and adapt through learning from static service technologies such as self-service terminals (Wirtz et al. 2018).
We systematically searched six research databases (i.e., ACM DL, EconBiz, EbscoHost, IEEE Xplore, Scopus, and a national database on academic publications in social sciences) to identify peer-reviewed articles that were written in English and focused on collaboration of humans and AI. Specifically, we used the following search term:
Collaborative Intelligence Definitions in Literature.
Engagement
For an AI system to be perceived as collaborative, it should actively engage users to co-produce the service (e.g., Epstein 2015). For example, a CI system could proactively ask employees’ opinion on a crossroad in the process toward a task outcome (e.g., via a chat or even speech module). A CI system should also engage employees by actively asking them for feedback on previous task outcomes.
Transparency
As a second feature, a CI system in service co-production should be transparent. A CI system can be described as transparent if it supports the user in understanding the way an advice was conceived. Also, when the system explains the outcomes of its analyses, transparency is enhanced (e.g., Epstein 2015; Lee et al. 2019). For example, a user may be informed about the underlying data and parameters that led to an output.
Process Control
To foster collaboration in service co-production, CI systems should provide users control over the service process. Process control refers to the possibility for the user to influence what evidence or data is considered by the CI system in its process and to decide upon the rules by which the output is generated (e.g., Lee et al. 2019; Lui and Lamb 2018; Paschen, Wilson, and Ferreira 2020). Employees may, for example, include or exclude certain parameters such that the outcomes of an analysis would differ.
Outcome Control
Fourth, outcome control allows employees to appeal or modify the outcome of an analysis from the CI once available (e.g., Epstein 2015; Lee et al. 2019; Lui and Lamb 2018). Thus, the final decision would lie in the human counterpart of a CI system.
Reciprocal Strength Enhancement
Finally, most analyzed articles emphasize that successful human-AI collaboration leverages each actor’s strengths. Thus, when humans and AI systems collaborate, each actor brings in their respective strengths that complement and augment each other (e.g., Huang and Rust 2022; Wilson and Daugherty 2018). For example, an AI system may provide big data analyses, which, on the one hand, supports employees in a decision and, on the other hand, gives them more time for solving problems that require creativity. By feeding decisions or ideas back into the system, the underlying algorithms learn employee preferences and improve their advice and performance over time.
Qualitative Study
We then engaged in a qualitative study by conducting 14 semi-structured interviews with practitioners. The aim of this study was twofold. First, we aimed to corroborate the five features we identified from the literature. We were specifically looking for potential system features that might not have been discussed in literature yet but are considered relevant in practice. Second, the interviews served as a pre-study to set up our scenarios further described in Study 1.
Our sample consisted of practitioners working in different roles in the financial service sector. Note that the financial sector is well suited for our research purposes as employees in this sector are generally used to working with big data and support software (McKinsey 2021). Our interviewees had an average of 10 years of tenure and were highly experienced in working with professional (partly AI-enabled) software, which ensured they would be able to voice their wishes and needs for AI system design. The interview guide was designed based on our aim to uncover CI system features and CI system effects on employees. The guide was slightly adapted after a pretest with one practitioner. The interviews lasted 50 min on average, were recorded, and then transcribed. To analyze the data, we used template analysis, in which primary codes are defined a priori (i.e., CI system features identified in literature) and may be adapted in the process (King 1998). The data was analyzed by two researchers along pre-defined coding rules (Cohen’s Kapa = 0.68; an inter-coder reliability considered satisfactory; Cicchetti 1994). Changes in codes and differences in coding were discussed and resolved by consensus. Web Appendix 2 shows sample characteristics, the interview guide, transcription rules, and example quotes for each feature.
Definition and Design of Collaborative Intelligence System Features.
The five features of collaborative intelligence systems.
Demarcating CI from Related Concepts
To ensure that a CI system—as we have defined it—represents a unique concept, we demarcate it from six related concepts in literature, namely, hybrid intelligence (Dellermann et al. 2019, 2021), human-AI symbiosis (Jarrahi 2018), collective intelligence (Gavriushenko, Kaikova, and Terziyan 2020), intelligence augmentation (Larivière et al. 2017), human-AI teaming (Dubey et al. 2020), and machines-as-teammates (Lyons et al. 2021; Seeber et al. 2020). We extracted these concepts’ definitions, distilled general properties and corresponding features, and outlined the differences to our CI systems concept. Web Appendix 3 illustrates that most related concepts focus on the collaboration of humans and AI systems (e.g., Jarrahi 2018; Lyons et al. 2021). These works do not take a design perspective nor outline CI system features. As notable exceptions, Seeber et al. (2020) and Dellermann et al. (2019; 2021) describe AI meta-design categories (e.g., appearance and conversation) but not their functionality nor how they relate to user outcomes. To better understand the effects of our conceptualized CI system on employee outcomes, we next empirically test these relationships in two scenario-based experimental studies.
Part 2: The Effects of CI System Design on Employee Outcomes
Theoretical Background: Affect-Behavior-Cognition Model
Employee responses to information technology are multifaceted and encompass a range of responses (Hong et al. 2011). The affect-behavior-cognition (ABC) model clusters these responses into affective, behavioral, or cognitive outcomes (Breckler 1984).
Study 1: The Effect of CI Systems on Employee’s Cognitive Outcomes
As an initial perspective, we conceptualize CI systems as a holistic yet multi-dimensional phenomenon, such that the more the five features are pronounced, the more we call the system a CI system rather than “just” an AI system. Accordingly, in our first empirical study, we focus on contrasting the effects of
First, we define
The power of service co-production to induce positive service outcomes lies in the active and smooth collaboration of parties (Bendapudi and Leone 2003). Each party contributes their unique knowledge and their strongest skills to the service encounter. As such, standardized parts of the exchange are conducted more efficiently, while more unstructured or idiosyncratic elements in the service benefit from joint input and deliberation (Dong and Sivakumar 2015). This collaboration runs most smoothly when the parties keep each other informed on their thoughts and intentions in the task fulfillment, such that they can simultaneously adapt to their counterpart (Jarrahi 2018).
Such active coordination and collaboration are more evident in strong than weak CI systems. For example, when CI systems provide process and outcome control, users have a sense of control over the service itself and its outcome, which may increase the employee’s trust and belief in providing a good service. Indeed, Dietvorst, Simmons, and Massey (2016) showed that the option to modify outputs (i.e., perceptions of control) increased reliance on algorithms because users felt that the advice was better. Similarly, when users know how a technology-assisted advice has been derived (i.e., transparency), they perceive the joint collaboration more positively because of an increased understanding of the system or of the service situation (Chen et al. 2018). The associated cognitive learning through engagement and transparency likely makes employees perceive that they have improved their service performance through using the system (Gomez, Unberath, and Huang 2023).
Second,
Generally, in human-to-human service interactions, people are more willing to assume responsibility for an outcome when they have actively contributed to the interaction and when they have been able to influence or control the process (Bendapudi and Leone 2003; Botti and McGill 2011). Similar observations have been made when services are co-produced with self-service technologies (Reinders, Dabholkar, and Frambach 2008) and social robots (Jörling, Böhm, and Paluch 2019). Because a strong CI system fosters service co-production, the presence of engagement, transparency, process control, outcome control, and reciprocal strength enhancement features should positively affect employees’ perceived outcome responsibility. Taken together, we hypothesize:
CI systems that are characterized by engagement, transparency, process control, outcome control, and reciprocal strength enhancement (i.e., strong CI systems) have a positive effect on a) perceived service improvement and b) perceived outcome responsibility for the co-produced service outcome, compared to CI systems that lack these characteristics (i.e., weak CI systems).
Experimental Design and Study Context
To test our hypotheses, we conducted a 2 × 2 between-subjects scenario-based experiment with two employee groups. Participants worked either as analysts or as customer advisors in a corporate bank. The scenarios described a CI system working together with these employees on a typical task in the work context of the respective employee (see Web Appendix 4; Table 4.1 and 4.2). We manipulated all five CI system features (i.e., engagement, transparency, process control, outcome control, and reciprocal strength enhancement) as either strongly or weakly pronounced (i.e., strong CI system vs. weak CI system) [1].
AI is increasingly adopted in the financial services sector because successful service execution relies on big and unstructured data which a single person cannot analyze—think about the complexities in granting credit loans (McKinsey 2021). However, in their decisions, employees still need to rely on their expertise and consider information that cannot be easily formalized in AI processable data (Kokina et al. 2019). Moreover, as stakes are higher in corporate banking than private banking, the context is likely to witness increased employee-AI collaboration in the near future (Bank, 2021; Kokina et al., 2019; McKinsey, 2021).
We followed a comprehensive approach to design robust and realistic scenarios. First, we gathered qualitative data in interviews with three analysts and three customer advisors from our focal financial institution. This ensured that the scenarios reflected a typical work situation and used the correct terminology. Second, to further enhance external and face validity, we discussed the scenarios and manipulations with 14 practitioners from other banking institutions. Finally, we discussed our manipulations of the CI system features with a group of nine researchers to make sure these elements are clearly described and do not conceptually overlap.
Participants and Procedure
Employees of a national bank in Europe were invited to participate in the two surveys via an e-mail newsletter. To encourage participation, each participant was given the opportunity to enter a lottery to win one of 13 Amazon gift vouchers with a total value of 500 Euros. Our final sample consists of 185 analysts (
In the online experiment, participants were asked to carefully read a scenario and picture themselves in this situation. They were then randomly assigned to one of two experimental conditions—a weakly pronounced or strongly pronounced CI system—after which they answered six attention check questions on the scenario content. Participants who did not pass the attention check were sent back to the scenario until they passed the test.
We relied on established scales to measure our two dependent variables. To measure
Following the measures of our focal variables, we included a manipulation check asking participants to rate the extent to which the described AI appeared as
Results
Descriptive statistics, intercorrelations, and psychometric properties of our dependent variables, which demonstrate adequate composite reliability, factor reliability, and discriminant validity, are displayed in Web Appendix 4, Table 4.5. We carried out a two-way analysis of variance (ANOVA) for the CI manipulation controlling for the employee group. Results revealed a significant difference in participants’ perception of the collaborativeness of the system between conditions (
To test the effects of CI on employees’ perceived service improvement we conducted a linear regression analysis controlling for credit loan amount, employee group, age, gender, tenure, and general AI experience. The results of our analysis revealed a significant regression equation (
Regression Analyses Results of Study 1.
Discussion
The results of Study 1 show that, compared with weak CI systems, strong CI systems increase employees’ perceived service improvement and outcome responsibility. While this study provides preliminary insights, we acknowledge some shortcomings. First, this study did not investigate which CI system feature(s) is/are the main driver(s) of the positive effects on employee outcomes. Second, this study considers cognitive employee outcomes but lacks information on affective and behavioral employee responses to CI systems. Third, we focused on a financial services context, but generalization to other contexts is needed. Relatedly, given that we collected data from one organization, we cannot account for different organizational practices (e.g., more or less experience in AI use within the organization). Study 2 attempts to overcome these challenges and dives deeper into the empirical investigation of our CI system concept.
Study 2: The Effects of Individual CI System Features on Employee Outcomes
Additional Employee Outcomes
In this study, we posit that the effects of each CI system feature on work-related employee outcomes may differ. Apart from the cognitive outcomes considered previously, we now also include
Behavioral outcomes such as adhering to or complying with the AI system are commonly investigated in human–AI interaction (Jussupow, Benbasat, and Heinzl 2020). In our context,
In addition, we define threat to meaning of work as an employee emotion in response to the anticipation that co-producing service outcomes with AI systems may harm the meaning of their work (Craig, Thatcher, and Grover 2019; Mirbabaie et al. 2022; Steger, Dik, and Duffy 2012). Compared to working with AI systems, employees perceive working with human colleagues as more meaningful, even independent of task difficulty (Sadeghian and Hassenzahl 2022). The threat to meaning of work may even surpass the job level. For example, metro drivers in Paris were promoted to managerial positions as AI systems took over their driving tasks. While the new job came with less repetitive work, the metro drivers claimed they were deprived of meaning in their new roles as they were not personally responsible for their passengers anymore (Smids, Nyholm, and Berkers 2020).
Hypotheses
Overall, we posit that CI systems characterized by our five CI system features should foster positive employee outcomes and thus increase perceived service improvement, perceived responsibility taking, adherence to the system, and decrease feelings of threat to meaning of work. However, research suggests that the features are differentially related to our employee outcomes, as detailed below.
Engagement
CI systems that are characterized by engagement actively involve users in the service co-production process. This feature is closely related to the process of collaboration in that it pro-actively asks employees’ opinion on a crossroad in the process toward a task outcome (e.g., Lyons et al. 2021). Employees can actively contribute to what they think is the best way forward and, hence, positively influence the perceived service outcome. On the one hand, engaging the user actively in the co-production process signals greater agency through social cues of the system. Such agency perceptions can lead to increased attribution of responsibility to AI systems (e.g., Johnson, Veltri, and Hornik 2008; Zafari and Koeszegi 2021) and might thus diminish perceived outcome responsibility while they increase adherence to the system (e.g., Adam, Wessel, and Benlian 2021). On the other hand, the engagement feature might lead to the feeling of being needed in the co-production process (Arslan et al. 2022) and, therefore, diminishes employees’ perceived threat to meaning of work. We thus hypothesize:
The engagement feature has a positive effect on a) perceived service improvement and b) adherence to the system, while it has a negative effect on c) perceived outcome responsibility and d) threat to meaning of work.
Transparency
Transparent CI systems allow employees to understand the way analytical outcomes and conclusions are obtained. The transparency feature thus provides the employee with additional information on the service process and outcomes, which the employee can then pass on to the internal or external customer. As a result, the typical black box problem of AI systems—which can undermine system acceptance in service co-production—is mitigated (von Eschenbach 2021; Gomez, Unberath, and Huang 2023). Employees should thus feel that their service improves, rather than merely changes, because of the transparency feature. Furthermore, gaining insight into the operational mechanisms of CI systems aids employees in delineating their distinct contributions to the co-production of services, distinguishing them from the contributions made by the AI system (Mirbabaie et al. 2022). It should thus foster perceived outcome responsibility and decrease perceived threat to meaning of work. Understanding how the system draws conclusions should further increase trust in its outputs and, hence, foster adherence to the system (Gomez, Unberath, and Huang 2023). Formally:
The transparency feature has a positive effect on a) perceived service improvement, b) perceived outcome responsibility, and c) adherence to the system. It has a negative effect on d) threat to meaning of work.
Process Control and Outcome Control
Our CI system concept includes two distinct control features. First, process control allows the user to influence what data is considered and the rules by which the output is generated. Second, outcome control allows the human user to appeal or modify final decisions that ultimately result in the jointly produced service outcome. Both types of control enable the user to influence the final service provided to the internal or external customer. As users are usually more satisfied if they can tailor technology interactions at work to their personal preferences (Gasteiger, Hellou, and Ahn 2023), we posit that both control features positively affect perceived service improvement. Moreover, extant research shows that the option to modify functions and outputs of AI systems increases users’ perceived responsibility for service outcomes (Jörling, Böhm, and Paluch 2019) as well as reliance on algorithms (Dietvorst, Simmons, and Massey 2016). We thus hypothesize that process and outcome control increase perceived outcome responsibility and adherence to the system. Finally, feeling in control over one’s work environment is an essential part of developing work satisfaction and meaning (Deci, Connell, and Ryan 1989). In our particular context, the control features signify the user’s indispensable role in generating service outcomes and establishing their dominance over the system. We thus posit that both control features decrease threat to meaning of work. In sum, we hypothesize:
The process control feature has a positive effect on a) perceived service improvement, b) perceived outcome responsibility, and c) adherence to the system. It has a negative effect on d) threat to meaning of work.
The outcome control feature has a positive effect on a) perceived service improvement, b) perceived outcome responsibility, and c) adherence to the system. It has a negative effect on d) threat to meaning of work.
Reciprocal Strength Enhancement
When both the CI system and the employee bring in their unique strengths (e.g., big data analysis vs. creative thinking) into the service co-production process, employees learn that working with the system can enhance their own qualities. In addition, employees experience that the feedback they provide improves the performance of the CI system (Bansal et al. 2021). Hence, if both parties in the co-production effort improve, the employee is likely to perceive that the overall service performance has improved. Moreover, if employees realize that their feedback shows in the system’s output, they may increase their feelings of responsibility for the jointly produced service outcomes. Furthermore, research shows that people are less averse to using an algorithm when they know that the algorithm is capable of performing the task (Bigman and Gray 2018; Castelo, Bos, and Lehmann 2019). We thus posit that the feature of reciprocal strength enhancement should increase adherence to the system. Finally, this feature has the potential to enrich the significance of one’s work as employees incorporate their unique human strengths into the process of service co-production and subsequently nurture these strengths. We thus hypothesize:
The feature of reciprocal strength enhancement has a positive effect on a) perceived service improvement, b) perceived outcome responsibility, and c) adherence to the system. It has a negative effect on d) threat to meaning of work.
Study Design and Context
The study was conducted in the context of human resources (HR) recruitment services. Recruitment services provide a suitable context to investigate the effects of CI system design because the preselection of candidates is one of the major applications of AI systems that promises efficiency gains in the age of labor shortages and the war for talent (Chen 2023). At the same time, since decisions involve future careers and are subject to ethical and legal standards, there will always be humans in the loop (Spring, Faulconbridge, and Sarwar 2022) and employee-AI collaboration is becoming a workplace norm (Black and van Esch 2020).
In this study, we employed a factorial survey (FS) experimental design. A FS combines elements of survey research with those of controlled experiments (Aguinis and Bradley 2014; Auspurg and Hintz 2015; Wallander 2009) and requires participants to rate vignettes that manipulate a set of factors, or dimensions, on their applicable levels. This allows researchers to disentangle which dimension(s) influence(s) participants’ evaluations. In our FS design, we experimentally varied the five CI dimensions on two levels as either weak or strong. We used the existing variations of the CI dimensions from the scenarios in Study 1 and adapted them to the new context. Combining all possible combinations of the five CI dimensions yielded 32 study conditions (i.e., five factors, two levels, 25 = 32 study conditions, cf. Web Appendix 5, Figure 5.1, and Table 5.1).
In order not to overwhelm participants, FS research recommends not presenting one participant with more than ten vignettes (Auspurg and Hintz 2015). Since we included more than two dependent variables which take time to evaluate, we took a conservative approach and randomly presented only four vignettes to each participant. We subjected our research design to a qualitative pretest with an HR professional and ten researchers to ensure the realism of the scenario, ascertain that it offers sufficient information for evaluating the CI system, and confirm that participants find responding to four vignettes a feasible task. Following this pretest, we made several changes in terms of wording and presentation of our FS design. A quantitative pretest with 30 participants revealed no concerns regarding data structure and vignettes’ uniqueness.
Participants and Procedure
We collected data through an online survey from 345 HR professionals in central Europe. Data was collected through the panel provider Respondi/Bilendi and the personal network of the author team. Participants are, on average, 42 years old (
Participants were asked to put themselves into the situation of having to select the top five candidates for a role in their organization. They were told that their organization now provides a new, interactive AI system that aids them in preselecting and ranking the potential candidates. The participants were then presented with four different variants of the system. After reading the vignette description, participants evaluated the presented CI system on our four dependent variables. Following FS standards, all dependent variables were measured using one item, 11-point Likert scales (Auspurg and Hintz 2015). The items were selected based on established scales and their comprehension, which we evaluated during the qualitative pretest. The perceived outcome responsibility and perceived service improvement items were drawn from the Study 1 scales. The item to measure threat to meaning of work was based on the work and meaning inventory (Steger, Dik, and Duffy 2012), and the item for adherence to the system was drawn from Westphal et al.’s (2023) measure of AI compliance. Table 5.2 in Web Appendix 5 lists all items employed. With 345 participants rating four vignettes, this design yielded 1,380 rating observations for each dependent variable. At the end of the survey, participants were asked to indicate information about their work (i.e., company size, industry, tenure, how many positions they fill, and whether they have a leadership position) and demographics (i.e., age and gender).
Results
Regression Analyses Results of Study 2.
We analyzed three regression models for each dependent variable. In the first model, we aimed to understand which CI system feature (Level 1) has an effect on our dependent variables. The second model adds the control variables age, gender, tenure, and whether participants already used AI systems for recruitment tasks (Level 2). In the third model, we explore the interaction effects of the CI system features and AI use, thus analyzing the cross-level interaction effects. Table 4 shows the results of our models. Because main effects are stable across models, we interpret the effect sizes of the most parsimonious model (i.e., Model 1) in our text below.
CI System Feature Effects
Our results reveal that the engagement feature does not significantly affect perceived service improvement (
The transparency feature has a strong effect on all four work-related employee outcomes. We identify a positive effect on perceived service improvement (
For process control, we identify a positive effect on perceived service improvement (
We also identify a strong effect of the outcome control feature on all four work-related employee outcomes. We find a significant positive effect on perceived service improvement (
Finally, we identify a significant positive effect of reciprocal strength enhancement on perceived service improvement (
Exploratory Post-Hoc Analysis: The Effect of AI Use
Regression Analyses Results of Study 2: Comparing Effects of AI Novices vs. AI Users (Twin Sample).
We find a marginally significant positive effect of the
Discussion
The aim of Study 2 was to identify those CI system features that are most relevant for four important employee outcomes. Taken together, the results reveal a strong effect of the features of transparency, process control, and outcome control on the four work-related employee outcomes. We also identify a relevant role of reciprocal strength enhancement while we do not identify any significant effect of the engagement feature on the four investigated employee outcomes. We also find a strong contingency effect of previous AI system use. The effect of CI systems on employee-related outcomes is much stronger for AI non-users than for participants who already use some kind of AI system in recruiting processes. We interpret this pattern in more detail in our general discussion below.
General Discussion
This research systematically develops the concept of CI systems in the context of service co-production and identifies five relevant design features. Two empirical studies show the effect of these features on four important work-related employee outcomes: perceived service improvement, perceived outcome responsibility, threat to meaning of work, and adherence to the system. Below we outline the implications of our findings.
Theoretical Implications
Our research makes four main contributions to literature. First, by delineating the concept for CI system design in service co-production, we conceptually and empirically contribute to the emerging literature on information–systems design for human–AI collaboration in service settings (e.g., Paluch et al. 2022; Sowa, Przegalinska, and Ciechanowski 2021; Walls, Widmeyer, and El Sawy 2004). While previous work defined and discussed the concept of collaborative intelligence (cf. Table 1), few, if any, studies concentrated on the design features of AI systems for service co-production. Rather, extant work defines CI as the combination of human and different types of artificial intelligence (Gill 2012; Martin and Azvine 2018), as the outcome of this combination (Wilson and Daugherty 2018), or as the degree of the collaboration ability of humans and AI systems (Zhong et al. 2015). In a more detailed endeavor, Huang and Rust (2022) develop a conceptual framework for collaborative intelligence in marketing, which helps our understanding of how different levels of human (i.e., contextual, intuitive, and feeling) and artificial (i.e., mechanical, thinking, and feeling) intelligence may be combined in the execution of marketing tasks. In doing so, they delineate the abilities of AI systems that initially augment human intelligence for task delivery, and ultimately may perform the task fully autonomously. However, these authors do not specify the system design features that enable collaboration or that would allow for task division between employees and AI systems. With our work we contribute to this embryonic literature by elaborating on the AI system features that foster successful service co-production.
Second, in contrast to lively discussions on customers and their AI perceptions (e.g., Le et al. 2024; Wirtz et al. 2018) and despite repeated calls to consider the employee perspective (e.g., Ostrom et al. 2021; Xiao and Kumar 2021), insights on how employees deal with AI systems in their daily work routine have so far been scant. We contribute to literature on employee-AI interaction in service contexts (e.g., Epstein 2015; Paschen, Wilson, and Ferreira 2020); building on the ABC model (Breckler 1984) we comprehensively show the impact of our CI system features on four relevant work-related outcomes: threat to meaning of work (affect), adherence to the system (behavior), perceived service improvement, and perceived outcome responsibility (cognition). While existing employee-AI collaboration studies mostly focus on outcomes on the firm level (e.g., Wilson and Daugherty 2018) or the dyadic level (i.e., the quality of the joint decisions made; Dellermann et al. 2021) this work considers individual level, work-related employee outcomes. As individual affect, behavior, and cognitions are important for the productive use of technology, we identify a CI design that may culminate in department- and firm-level consequences. Considering our dependent variables, we also contribute to the emerging discussion of employee engagement (Kumar and Pansari 2016) and meaningful work (Smids, Nyholm, and Berkers 2020) in the age of AI. If AI systems are designed in a collaborative manner, it may be possible to increase firm efficiency while simultaneously maintaining employee well-being and a sense of duty and responsibility.
Third, recent work by Le et al. (2024) focuses on customer perceptions of communicated collaborative cues in human-AI collaboration (e.g., visible handover between AI and employee). Here, the customer is an observer and the recipient of the service. In contrast, we focus on the employee perceptions of design features, which directly affect the employee as the co-producer of the service, their interactions with the CI system, and, ultimately, the service decisions they make. We thus provide more detail on the inner workings of AI-enabled service co-production and unveil a different mechanism of how CI system implementation may influence service outcomes. For example, while Le et al. (2024) show that communicated collaborative cues evoke transparency perceptions of the customer which in turn foster customer satisfaction, we conclude that untransparent CI systems may diminish employees’ sense of meaning of work, reduce their responsibility taking, and thwart perceived service improvement. These employee outcomes may affect the overall service quality delivered, in addition to (merely) the customer’s perceptions of service.
Particularly, Study 2 identifies a primary role of transparency, process control, and outcome control in stimulating desirable employee work-related outcomes. Additionally, our results question the value of the engagement feature. One explanation might be that engagement, for example, through asking for user feedback, is perceived by employees as effortful rather than empowering in the service co-production, and hence, decreases the (perceived) efficiency of the process without improving the outcome. Alternatively, when a CI system frequently asks questions in its decision-making process, the system may come across as incompetent or immature. As an alternative explanation, in our scenario, we manipulated engagement through actively integrating the employee via system-initiated questions. We did not detail how the system interacts though (e.g., voice). With the rise of more interactive systems in private and work domains—think of generative AI like ChatGPT—human-like interactions with AI are likely to become the norm (Cantrell et al. 2022). This suggests that the engagement feature might gain relevance and appreciation in the future. We return to this issue in our Limitations and Future Research section.
Finally, we contribute to a stream of literature that has concentrated on identifying individual differences and task characteristics as contingency factors in technology acceptance and use (e.g., Blut, Wang, and Schoefer 2016; Brown, Dennis, and Venkatesh 2010; Park et al. 2014). Specifically, we reveal strong and positive effects of the CI system features transparency, process control, and outcome control on work-related outcomes for employees who are AI novices but not for employees who have already used AI systems in their job. For inexperienced users, these features might increase trust accompanied by the security of retaining control over task fulfilment (cf. Gomez, Unberath, and Huang 2023). For experienced users, these features might be less relevant as they are already familiar with AI systems that take over tasks independently and thus are less in need of, for example, control over the system. We also identify a clear positive effect of reciprocal strength enhancement on all four dependent variables for AI novices compared to more experienced AI users. It may be that the augmentation of one’s individual qualities is especially salient to novices as they lack such prior experiences. This salience effect might diminish for experienced AI users who know better what CI systems can do.
The observation that experienced AI users generally respond less strongly to CI system design could perhaps also be explained by the fact that most of today’s AI systems are used for automating a part of a service that is somewhat separate from the part that the human would conduct. For instance, AI systems identify critical clauses in a large number of contracts; hereafter lawyers may reconsider the nature and contents of these contracts (Spring, Faulconbridge, and Sarwar 2022). For employees using AI in such an assistive way in the pre-stage of their own activities, a collaborative AI system that takes turns with the user in the central stages of service production may feel futuristic or over-engineered. In any case, the role of employees’ experience with AI seems to be fruitful ground for future research.
Managerial Implications
The cornerstone of the fifth industrial revolution is the collaboration of humans and AI-enabled systems (Noble et al. 2022) and employee-AI collaboration is becoming a workplace norm (Cantrell et al. 2022). Hence, it is essential to understand how successful employee-AI collaboration in service co-production can be managed, including challenges such as employees' taking responsibility for their work outcome and threats to meaning of work. Our research efforts offer two main implications for service managers in this realm.
First, service managers and requirement engineers may use the five CI system features we identified as a blueprint to design internal service processes based on employee-AI collaboration. For example, practitioners could map out the (internal) service encounter, identify at what point the collaboration would benefit from employee input, and program CI systems to allow
Second, we show that the effect of CI system features on all work-related employee outcomes is greater for AI novices than for employees who already work with AI systems in their job. This means that especially organizations introducing their first CI system need to consider the design features carefully. Also, service firms should monitor whether the positive effects of CI system features prevail or whether the design needs to be adapted over time.
Limitations and Future Research
We acknowledge several limitations of our research. First, we conceptualized our CI system with a focus on service co-production; a focus on another domain may identify other features. Still, it is important to note that the features uncovered are specific to AI, rather than technology. For example, strengthening each other’s qualities is not typical in every service technology interface, think about self-service technology. Also, the transparency of a system, pertaining to an algorithm, is typical to AI.
Second, scenario-based experiments are common in research that focuses on cutting-edge technology (e.g., Choi, Mattila, and Bolton 2021; Schepers et al. 2022). However, such setups may also raise external validity concerns. Despite our realistic scenarios in both our studies, it might well be that positive effects were driven by experiences of novelty and diminish over time. In a sense, this is also what our contingency effect of AI use demonstrates. Moreover, in our financial and HR settings, we have been able to clearly separate CI features in our scenario descriptions. For example, engagement meant that the system actively asked the HR professional for feedback during the selection process, with the ability to proactively ask the system questions too. Process control indicated that employees could intervene in the selection process and, for example, adjust the weighting of decision parameters (e.g., years of professional experience). While both dimensions indicate communication between user and system, engagement here is two-way, exploratory or confirmatory, and general in nature. In contrast, process control is one-way, corrective, and task-specific in nature. These distinctions between features may differ, for example, be more or less pronounced, across different contexts. We thus urge scholars to consider long-term field research designs with actual CI systems in various contexts.
Finally, our newly developed CI system concept opens manifold avenues for future research. To move CI systems research in the services field forward, we developed a research agenda according to three foci—see Web Appendix 7 for details. First, future research could further develop our CI system conceptually. For example, depending on the context and with technological advancements, additional features might become relevant. Second, future research could further empirically investigate CI systems. For example, researchers could investigate the downstream consequences of using CI systems and reveal mechanisms and additional contingency factors to further detail the effects of CI systems on various individual-, team-, department-, and organizational-level outcomes. Additionally, future research should further investigate the effect of previous AI use and, for instance, study the nature of the AI systems in which employees are already experienced. Experience in different features, interfaces, and functionalities may affect the evaluation and the effects of CI systems. Finally, future research could focus on ethical considerations when firms introduce CI systems. For example, there might be employees who, due to their dependence on a CI system, interact less with human colleagues. This could negatively affect their need for social belongingness or, ultimately, well-being.
In closing, we feel that CI systems are an intriguing technological development in modern service firms. This development brings with it a host of unanswered research questions, and we hope that our work sparks researchers’ interest to help further develop this area.
Supplemental Material
Supplemental Material - Designing Collaborative Intelligence Systems for Employee-AI Service Co-Production
Supplemental Material for Designing Collaborative Intelligence Systems for Employee-AI Service Co-Production by Marah Blaurock, Marion Büttgen, and Jeroen Schepers in Journal of Service Research
Supplemental Material
Supplemental Material - Designing Collaborative Intelligence Systems for Employee-AI Service Co-Production
Supplemental Material for Designing Collaborative Intelligence Systems for Employee-AI Service Co-Production by Marah Blaurock, Marion Büttgen, and Jeroen Schepers in Journal of Service Research
Footnotes
Acknowledgments
Declaration of Conflicting Interests
Funding
ORCID iDs
Supplemental Material
Note
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
