Sage Journals: Discover world-class research

Abstract

Background

This study introduces an innovative personnel training method for facility management and maintenance of Thermal-Energy-Storage (TES) chiller plants using a serious 3D game. Training games can improve the decision-making of personnel where they can learn to deal with management of TES chiller plants in the context of this study in an active learning approach.

Intervention

The research aims to investigate the effectiveness of an immersive learning experience with computerized simulation and synthetic task environments as a training game for facility managers. The serious 3D game adopts a first-person perspective, allowing players to assume the role of a facility manager and actively learn how to address maintenance issues commonly encountered in chiller plants.

Methods

The study implemented a first-person-based serious 3D game centered around a TES chiller plant. Participants, in the role of facility managers, engaged with the game and followed instructions to gain practical knowledge in managing maintenance issues within a controlled and simulated setting.

Results

The findings demonstrate increased engagement and interest among personnel when learning how to manage chiller plant issues within a serious 3D game environment. Notably, personnel experienced reduced pressure compared to real-life scenarios, as they navigated the challenges without the presence of a supervisor.

Limitations

The study’s limitations include a higher proportion of male participants in the qualitative content analysis, which may affect the generalizability of the findings. Additionally, the absence of a control group limits the ability to make direct comparisons with traditional training methods.

Conclusion

The results suggest that serious 3D games hold potential as an effective training tool for facility management and maintenance personnel. By engaging in active training within simulated and synthetic task environments, personnel can enhance their skills and decision-making capabilities. However, further research with a more diverse participant sample and a control group is warranted to fully evaluate the effectiveness of this innovative personnel training approach.

Keywords

serious games decision-making computerized simulation training games immersive learning synthetic task environments simulation/gaming active learning

Introduction

Facility management and maintenance (FMM) of a building facility is the longest phase of Building Information Modeling (BIM) methodology and the most expensive as well (Ventures, 2021). The part regarding FMM comes about after completing the construction of a building and handing over the site as only after the construction phase, facilities need to be managed and maintained (Coupry, Noblecourt, Richard, Baudry, & Bigaud, 2021). Facility managers are the responsible entity who looks after the building’s daily operations and ensures the comfort of occupants, and in case of any sudden problem that arises in the building facility that may disrupt the daily operations, facility managers have to take action promptly to minimize disruption in activities and discomfort of occupants (Roper & Payant, 2014). The task of facility managers requires constant efficient decision-making and prompt handling of issues that arise and hence, this job requires proper training, so disruption time is minimized, and labor efficiency is optimized (Haines, 2016). Generally, the traditional approach in training new personnel that are hired or transferred to the FMM department is by a verbal dictation from another staff member who works at the same building facility. However, such an approach does not allow the newly hired personnel to be confident in handling the common issues that arise as the newly hired personnel may feel uncomfortable and shy to ask or verify certain information. Other than that, with verbal dictation, there is a high chance of the personnel forgetting the information he/she just learnt, and the person cannot possibly perform the activities unless those issues happen in real-life scenarios.

The issues become a serious management crisis when it is about complicated facilities such as Thermal Energy Storage (TES) chiller plant. Proactive environmentally friendly companies have made a shift to Thermal Energy Storage-Air Conditioning (TES-AC) for cooling or heating their building facilities depending on the weather conditions. However, management and maintenance of a TES-AC system becomes even more tedious when the building facility is a shopping mall, or a serviced apartment and disruption of daily operations or discomfort of occupants will negatively impact the company (Suamir, et al., 2019). To address appropriate training of personnel regarding management of a TES chiller plant, a more immersive and accessible form of training is better suited than the traditional verbal approach (Araszkiewicz, 2017; Coupry, Noblecourt, Richard, Baudry, & Bigaud, 2021; Sullivan, 2016)

Interest in utilizing digital tools for adopting Industry 4.0 such as serious three-dimensional (3D) games is increasing due to the accessibility aspect it provides and such games can be integrated into personnel training for management of common problems of a TES chiller plant (López, Arias-Oliva, Pelegrín-Borondo, & Marín-Vinuesa, 2021; Almeida & Simoes, 2019). The serious 3D games provide remote accessibility where the personnel can easily train themselves whenever they are available in a controlled environment without the pressure of an actual real-life problem where their decisions can have serious consequences (Connolly, Hainey, Boyle, Baxter, & Moreno-Ger, 2013). In comparison to Virtual Reality (VR) games, serious 3D games or Augmented Reality pose to be a preferable option for training for due to the player feeling more grounded whilst being in an immersive active learning environment (Stutzman, Nilsen, Broderick, & Neubert, 2009; Gillis, 2022; Hakkarainen, Woodward, & Billinghurst, 2008). This same immersive aspect can be achieved with a serious game with a computer, keyboard, and mouse, instead of a VR as there is no requirement for additional headset which causes discomfort to many users and not completely being transformed into another space which causes cognitive overload. With realistic graphics and an immersive experience, a serious 3D game has the potential of grasping the player’s attention, so the player is likely to remember the steps taken. In previous research, it also mentioned that such immersive experiences have a positive impact on episodic memory which is why a player has a high knowledge retention in learning with a serious 3D game (Sanzana M. R., Abdulrazic, Wong, Ng, & Ghazy, 2021). Additionally, serious 3D games promote active learning where with a guided instructional game approach, personnel can learn how to manage crises in a controlled setting, so they are more prompt in decision-making when an actual real problem arises (Sharifzadeh, et al., 2020).

Furthermore, a TES chiller plant is generally a noisy environment with various equipment pieces and carrying out a training drill there is tough, and the environment is not very learning friendly (Suamir, et al., 2019; Lu, Yan, & Yang, 2020). During such traditional training drills, the personnel may not be very attentive, or they might want to rehearse the drill at their own time (Moro, Phelps, & Stromberga, 2020). Integrating serious 3D games into personnel training allows such possibilities where newly hired personnel or even any personnel can rehearse training drills in a guided game story which they can access from anywhere with simply a laptop (Almeida & Simoes, 2019). Hence, this research focuses on using a serious 3D game for personnel training for FMM of a TES chiller plant in a guided first-person story mode game approach. First-person is simply the way in which the player views the game world, in this case it indicates looking through the eyes of the main character (Target, 2015; Sanzana M. R., Abdulrazic, Wong, Ng, & Ghazy, 2021). The first-person mode was taken to create an immersive and accessible training tool for all personnel which will also improve labour efficiency as the personnel will be prompt in taking necessary steps when a common problem in real life arises regarding the TES chiller plant. Experts were involved in the research study to understand their interest in implementing such innovative training methods for personnel training and afterward, individuals who may be interested to pursue FMM as a career choice were involved to understand their perspective and interest in benefiting from a serious 3D personnel training game. This research hopes to contribute to finding effective ways in implementing gamification elements into a personnel training game and intends to benefit future researchers and developers to integrate a serious 3D game as a training tool that will improve labour efficiency if applied in industrial cases.

Literature Review

This section gives an overview on the evolution of serious games and how they have been integrated into training. It also considers the other innovative approaches taken to improve the personnel training for facility management and maintenance and outlines the uniqueness of the conducted study.

Serious Games in Training

Serious games have been known to be a potential tool for instructions and Belotti et al. (2013) emphasizes how it is important to provide feedback to the player regarding the player’s activity in the game to improve characterization of the player. With proper feedback, and better assessment integration in games, learning through serious games can be greatly improved. Awareness in many disciplines to integrate serious games have increased over the time and previous research presented factors such as engaging the learner constructively, facilitate the learning task with instructions and leave room for exploitability to maximize the learning effect (Catalano, Luccini, & Mortara, 2014; Yusoff, Crowder, Gilbert, & Wills, 2009). Besides computer games being very popular as a leisure activity, the potential of skill acquisition and how games can assist learning and behaviour change is explored (Boyle, Connolly, & Hainey, 2011).

Effectiveness of learning with serious games in corporate training were evaluated and it was mentioned that designing metrics for learning requires considering the stakeholders such as employers, employees, and the management (Bachvarova, Bocconi, Pols, Popescu, & Roceanu, 2012). With maker-based education like serious games, individuals can participate in active learning which improves learning gain and keep the user more engaged (Abdulrazic, Sanzana, & Ng, 2022). Additionally, to benefit from training games and to acquire skills or knowledge, proper immersion is a necessary game feature, hence a first-person based probably is more appropriate for training because of how the players can see through the eyes of the character (Bachvarova, Bocconi, Pols, Popescu, & Roceanu, 2012; Sanzana M. R., Abdulrazic, Wong, Ng, & Ghazy, 2021). Although serious games use entertainment principles, and creativity to gather training objectives, it does not mean that the intended learning will occur (Tuholski, Engle, & Baylis, 2001). It is important to consider the ingredients that make an engaging synthetic task or learning environment (Greitzer F. L., 2005; Green & Bavelier, 2003). Greitzer et al. (Cognitive science implications for enhancing training effectiveness in a serious gaming context, 2007) mention the importance of incorporating sound cognitive, learning, and pedagogical principles into a training game for it to be effective. Previous research explored how the positive effects on perceptual and motor skills by playing video games have proven to have training-induced increases in performance (Green & Bavelier, 2003). For scenario-based training to train inquiry officials to be able to determine system-induced human errors that lead to security incidents, a web-based interactive training program is described that utilizes media, and proper application of cognitive learning principles (Greitzer, Pond, & Jannotta, Scenario-Based Training on Human Errors, 2004). In a thorough review for training healthcare professionals, it is evident that the number of serious games for health care training continues to be used for its effectiveness which was evaluated in the review (Wang, DeMaria, Goldberg, & Katz, 2016; Haoran, Bazakidi, & Zary, 2019).

Azadegan et al. (2012) discusses the level of awareness in adopting serious games for corporate training in United Kingdom (UK) based companies through survey questionnaires. Serious games are even being considered for children to improve attention and develop self-regulation skills (Hocine, 2019). A study involving a card-based strategy game for United States Air Force to allow active training in multi-domain operations utilizing air, land, and cyber domains revealed serious training games to be interesting, an effective educational and training tool including better self-reported understanding in majority of the participants (Flack, Voltz, Dill, Lin, & Reith, 2020). Recently research has been focusing on actively integrating serious games, and Soft Skills Training Program (SSTP) was proposed to train intrapersonal, interpersonal, personal social responsibility, and organizational sustainability to future employees by combining multiple serious games (Sutil-Martín & Otamendi, 2021). The study conducted at a university regarding soft skill training demonstrated an increased value in soft skill development (Sutil-Martín & Otamendi, 2021). Serious games are already being used for nursing education and after reviewing 7 nursing games suggest better graphics and entertainment for students, using simple serious game technology such as quizzes for learning (Min, Min, & Kim, 2022). Additionally, educators from higher education institutions have conducted studies and included gamified virtual labs in Biology and Chemistry education with gamified lab tasks to allow students to practice hazardous experiments in a risk-free way (Sanzana M. R., Abdulrazic, Wong, & Yip, 2023).

Serious Games in Management and Maintenance

Modern decision-making is characterized by the complexity of project management and subject to decision makers which requires thorough training for appropriate management skills and bad decisions can seriously impact a project (Attri R., 2014; Flyvbjerg, 2014). Universities tend to be a transformative phase for young individuals to hone professional skills about management with improved strategic thinking, self-awareness, critical, and problem-solving thinking, and serious games have a significant effect from a sustainable perspective (Miguel, Lage, & Galindez, 2020). A review of games/simulations for disaster risk management (DRM) involving the relevant stakeholders was conducted and it revealed that DRM related games improve problem-solving skills, especially in identifying hazards, and undertaking preventive actions (Solinska-Nowak, et al., 2018). Utilizing serious games for project management has displayed an improvement in player’s decision-making performance in both less complex and more complex scenarios, and hence should be considered for training (Rumeser & Emsley, 2018). Literature has shown that serious games can fill in the gap of organizational professional skill acquiring as many organizations heavily depend on an acquired experience of management skills (Stettina, Offerman, Mooij, & Sidhu, 2018). Entrepreneurship is a fundamental skill to be learnt by students and a serious game, FLIGBY, demonstrated potentials and possibilities in the development of management, leadership, and entrepreneurship skills in an immersive way based on real challenges that are found in business environments (Buzady & Almeida, 2019). As serious games are being constantly focused on for management skills, Mittal et al. (2022) conducts a review of serious games for urban water management decisions which revealed limited use of story mode, and adaptive gamified elements. Moreover, in current management training based serious games, there is a lack of focus on performing the initial decision-making phase which, instead needs to be properly integrated into a smooth story format (Mittal, Scholten, & Kapelan, 2022). Leitão et al. (2021) emphasizes how more game development is required for business process management as the study could only evaluate 15 serious games categorized to teach process implementation though it is a principle focus lately. According to the literature, not many serious games exist for honing management skills, and more research with serious games that utilize adaptive gamified elements and proper integration of decision-making should be focused on whilst letting the player immerse and explore in an active learning synthetic task environment. With proper immersion, and thoughtful game-design, higher knowledge retention can be achieved with increased edutainment value which can be applied for serious games focusing on facility management and maintenance (Sanzana M. R., Abdulrazic, Wong, Ng, & Ghazy, 2021).

Objectives of the Study

This correlational research explored the association between expert suggestions about the personnel training as a serious 3D game and the usability experience of the developed serious 3D game according to the engineers in the context of facility management and maintenance of TES Chiller Plant. It also investigated potential methods of integrating serious 3D games in industrial training for facility management and maintenance and introduced an approach of personnel training that is more accessible and minimizes risk. The research objectives (O) are listed below:

O1: Explore the association between expert suggestions about the personnel training as a serious 3D game and the usability experience of the developed serious 3D game according to the engineers in the context of facility management and maintenance of TES Chiller Plant.

O2: Investigate potential methods of integrating serious 3D games in industrial training for facility management and maintenance.

O3: Introduce an approach to personnel training that is more accessible and minimizes risk.

Research Questions

The following research questions (RQ) directed this study:

RQ1: What do experts think of implementing a serious 3D game to train personnel in TES-AC chiller plants?

RQ2: What are the key features that should be included in a serious 3D personnel training game?

RQ3: What do engineers think about the usability of the serious 3D personnel training game?

Research Hypothesis

The research questions were restructured as the empirical hypotheses (H) stated below:

H1: The experts gave their feedback about the proposed 3D serious personnel training game.

H2: The experts gave their feedback about certain features that should be included in the proposed 3D serious personnel training game.

H3: The engineers gave their feedback about the usability and usefulness of the serious 3D personnel training game after playing it.

Uniqueness of Study

This research study comprises an innovative approach in personnel training using a serious 3D game to allow an immersive learning of management for common problems in TES chiller plants. The TES-AC system is a crucial building facility component and proper management skill is required to minimize occupant discomfort and daily operation disruption. In a synthetic task environment, the personnel can participate in an active learning where they would follow instructions in a gamified way, by learning to manage issues on their own in a controlled setting without the presence of a supervisor or an actual real-life issue which lessens the pressure on the personnel. With a better understanding of procedures required for management, the personnel can apply their knowledge promptly when an actual issue arises in a TES chiller plant. This study first gathered suggestions to develop a serious 3D training game and then also gathered user satisfaction from the players. Additionally, this study also evaluated the interest in integrating such immersive technologies for training from experts and then evaluated the interest from the players in using training games for learning management skills.

Methodology

This section describes the methodology adopted by this research including mentioning research objectives, research questions and protocol. It also mentions how data was collected for this research and gives an overview of the developed serious 3D game for personnel training for TES chiller plant management of common problems.

Research Protocol

The research study consisted of the methodology shown in Figure 1 where initially a qualitative content analysis consisting of experts were carried out to understand whether they would be interested to integrate a serious 3D game for personnel training and collect design considerations from them. After the development of the game, other participants were asked to interact with the game and fill up a survey which was required to validate and to understand whether there is a correlation of the expert suggestions with the participants who are interested in the field of facility management and maintenance. It was important to understand the feasibility and user satisfaction of implementing serious 3D games for personnel training to improve labour efficiency and build confidence in management of issues. The research protocol consisted of the following to address the objectives of the study:

O1: To address Objective 1, a qualitative content analysis was conducted with experts to understand their interest in integrating a serious 3D game for personnel training and to gather design considerations.

O2: To address Objective 2, a serious 3D game was developed based on the input received from the experts, incorporating key features suggested by them.

O3: To address Objective 3, participants from engineering degrees interested in facility management were asked to interact with the developed game and provide feedback through a quantitative survey.

Figure 1.

Research methodology framework.

The methods adopted are listed below:

• For the first survey involving experts, a qualitative content analysis was conducted to understand their interest in using serious 3D games for personnel training and to gather design considerations. The survey consisted of Multiple-Choice-Questions (MCQs).

• For the second survey involving participants, a Likert scale-based survey was conducted to gather feedback on their experience with the personnel training game and its usability.

Game Process

This section is to address RQ1 and RQ2 . The game was designed to simulate common maintenance issues in a TES chiller plant. Experts were asked about their interest in using the simulated training environment and the serious 3D game format for personnel training (RQ1). Key features of the game, such as the first-person view for immersion, quest-based missions, and interactions with characters, were included based on expert suggestions (RQ2).

The game is designed in the first-person view to give the user the most immersive experience and for the ease of use and to make it easier to navigate around the scene compared to a third- person view. Length of the gameplay is approximately 15-20 minutes depending on the player and it is a story-driven game with quests. The user starts as a newly hired facility manager who is still getting used to the chiller plant facility in the building. He then needs to explore and interact with various characters and computers to learn more about the tasks required. The game will simulate certain maintenance issues, that are common with chiller plants and expect the user to resolve them. Some of the maintenance issues that the user will deal with includes checking the maintenance schedule, dealing with the chiller not turning on, checking the coolant level, and dealing with high and low temperatures in the chiller. The user will experience being with the chiller plant in the basement, having a control office with computers and other staff. The game will require the user to approach characters and interact and talk with them to receive information. It will follow a mission-based style where the user needs to finish one mission before moving on to the next one.

The missions will require the user to explore the scene more, interact with computers to check chiller plant data, answer questions to progress further in the game, and so on. The game assumes the users playing it are somewhat familiar with facility management and chiller plants and not completely new to the topic. However, the game will be fully guided and will provide adequate tips and hints on what to do. It is not a stressful game and will have a relaxed environment but also will include sounds that exist in that environment to make it more realistic. The graphics used in the game were optimized to be able to run on lower-end devices as well to increase the practicality of the game, as this is intended for training purposes and not an AAA i.e., a high-budget game production, usually involving a big team and state of the art technology, the equivalent of a blockbuster movie in the games industry.

Figure 2 shows some screenshots during gameplay, in the first one it shows the first task the player has to do which is to check the maintenance schedule. In the second one they are tasked with inspecting the chiller, to find out if the coolant level is sufficient, as part of a quest. Finally, in the last pictured they are tasked with finding their workstation to fill up a report. The facilitator was present during the gameplay to ensure smooth experience of the participants and solve any technical issues.

Figure 2.

Scenes from the Personnel Training Game of TES Chiller Plant.

Sample

This section is to address RQ3. The sample consisted of 50 adults (18 years or older). Engineers interested in pursuing a career in facility management played the game and provided feedback through a survey, which addresses RQ3. The sample of the research involved 50 adults i.e., 18 years or older for two survey-based questionnaires and involved minimal risk. Out of the two surveys, the first survey involved a Qualitative Content Analysis to collect responses from 15 experts to understand the interest in utilizing serious 3D games for personnel training to learn and practice management and maintenance issues, of a TES chiller plant. After the survey, a serious 3D game was developed and then another quantitative survey was carried out with 35 participants from engineering degrees who were interested in pursuing the career of being a facility manager. The participants first played the game, and then filled up the survey to validate the interest and effectiveness of a personnel training serious 3D game.

Instruments

The surveys were designed to address RQ1, RQ2, and RQ3. The first survey collected expert opinions on the interest in using serious 3D games for personnel training and their preferences regarding game design. The second survey gathered feedback from participants on their experience with the personnel training game and its usability. The surveys for this research study were developed on Google Form for the aspect of accessibility online. There were two surveys, one of which was for experts to understand the suggestions and the interest in using serious 3D games for personnel training rather than traditional verbal dictation and involved Multiple-Choice-Questions (MCQs).

Survey for Experts

This survey aimed to gather expert opinions on the interest in using serious 3D games for personnel training and their preferences regarding game design. The survey consisted of Multiple-Choice-Questions (MCQs). The questions in the survey were designed to address Research Questions 1 and 2 (RQ1 and RQ2). Table 1 shows the questions asked in the survey to the experts.

Table 1.

Questions asked in the first survey to the experts.

No.	Question Description
Q1	Do you think it will be beneficial for your staff to be able to have a simulated training environment?
Q2	A serious 3D game is for educating the players about a certain topic or training in a controlled setting with interaction. Do you want the simulated training in the form of a serious 3D game?
Q3	Do you think managing certain common problems that arise in maintaining and managing a Thermal-Energy-Storage chiller plant in a simulated game would be interesting?
Q4	According to your opinion, do you believe training or managing the common issues faced in a Thermal-Energy-Storage chiller plant would improve labour efficiency?
Q5	Would you want the simulated serious 3D game as a first-person where the player gets a more immersive feeling (he does not see the character, but his eyes are the camera angle) or a third person based where the player gets to see his character?
Q6	Would you want the serious 3D game as a story mode? Story mode means the player will go through the game as a story and complete his training task.
Q7	Are you interested in using such immersive technology for personnel training?

Survey for Participants

This survey aimed to understand the interest of engineers in interacting with a personnel training game for learning how to manage and maintain common chiller plant problems. The survey used a Likert scale from 1 to 5, where 1 denoted the most negative feedback, and 5 denoted the most positive feedback. The questions in this survey were designed to address Research Question 3 (RQ3). The questions in this survey were designed to assess participants’ feedback and experiences with the developed personnel training game, specifically addressing the usability and effectiveness of the game which can be seen in Table 2.

Table 2.

Questions asked in the second survey after interacting with the game.

No.	Question Description
Q1	According to your opinion, did you find the simulated 3D game to be interesting?
Q2	It is supposed to be an educational game, which in other terms is called a serious game. Do you think this game made you understand the concept?
Q3	Could you understand the procedure of maintaining and managing the chiller plant issues?
Q4	Do you think in-game controls were easy to use?
Q5	Did you find the story flow of the game to be smooth and engaging?
Q6	Did you like the use of graphics and the setting of the game?
Q7	Did you get to learn how to manage and maintain common issues at a Thermal-Energy-Storage Chiller plant with this game?
Q8	Do you think this type of game training will be useful for staff training for more efficient management of issues?
Q9	Do you think this form of simulated game for personnel training to be more engaging and active than a senior staff simply sharing or stating what to do?
Q10	Do you think training in a controlled setting to be safer than a new staff handling issues with an order or simply viewing other staff manage it?

Serious 3D Game

The serious 3D personnel training game developed for this research was developed using the Unity 3D game engine, a powerful, multi-platform game engine. Based on Unity’s programming language, C# was used for all the scripts used in the game to make it functional. However, some assets were also purchased from the Unity Asset Store. All 3D characters in the game were acquired from Mixamo, an Adobe online service that provides free 3D characters, rigged and animated.

Validity of the Instruments

The validity of the instruments refers to whether the surveys accurately measure what they were intended to measure. To ensure the validity of the surveys, the questions were carefully crafted to align with the research objectives and research questions. The questions were chosen to capture the key aspects of interest, such as interest in using serious 3D games for training, preferences for game design, and feedback on the training game’s usability and effectiveness. The surveys were also designed to be relevant to the target population, which includes experts in facility management and participants interested in pursuing a career as a facility manager. The average Content Validity Index (CVI) for the Expert Survey is approximately 0.7952, which is slightly below the commonly used threshold of 0.80. As a result, based solely on the average CVI, the content validity can be acceptable. The researchers evaluated the construct validity of the measurement instrument by conducting convergent validity and discriminant validity analyses. The construct "Interest in serious 3D game" from the Expert Survey exhibited a strong positive correlation (r = 0.70, p < 0.01) with the construct "Found the game very interesting" from the Participant Survey, providing robust evidence of convergent validity. Additionally, the construct "Interest in serious 3D game" demonstrated a weak positive correlation (r = 0.20, p > 0.05) with the construct "Easy to understand the concept," supporting the discriminant validity of the measurement instrument. These results indicate that the measurement instrument accurately captures the underlying constructs and aligns with the theoretical framework of the study.

In conclusion, the instruments used in the study were two surveys designed to address the research questions. The questions were carefully chosen to align with the study’s objectives and to gather relevant data from experts and participants. The validity of the surveys was ensured through careful question design, and internal reliability testing would be required to assess the consistency and stability of the survey responses.

Statistical Analysis

This research utilized two main softwares to analyze the data from both surveys and calculate percentages of each answer, addressing RQ1, RQ2, and RQ3. The analysis was simple but sufficient to draw the conclusions from the research. Initially the data from both the surveys, the one intended for the experts and the one intended for the students, were exported into Excel spreadsheets. Each question response was counted and grouped into percentages of each answer using basic Excel formulas. The age data was further analyzed and was exported to IBM SPSS 28.0.0.0 to calculate some extra statistics and check the distribution of the age across the gender. However, since the experts group only had one female, this test was not done based on the experts data, and only on the engineers group. The Mann-Whitney U Test was chosen as a method to test the distribution of age across the same categories of gender.

In conclusion, the methodology implemented various research steps, such as expert input through qualitative content analysis, game development based on expert suggestions, and participant feedback through surveys, to achieve the research objectives and address the research questions related to the association between expert suggestions, game usability, and methods of integrating serious 3D games in industrial training for facility management and maintenance.

Results

Demographics of Participants

The first survey involved 15 experts with an age range of 34-62 years (mean age: 51.8 years) for qualitative content analysis where 14 experts i.e., 93.33% were male, and 1 expert i.e., 6.67% was female. Among these participants, 9 were experts i.e., 60% specifically in facility management and maintenance, and 6 were experts i.e., 40% in civil engineering and architecture.

The second survey involved 35 adults from engineering backgrounds with an age range of 18-32 years (mean age: 24.7 years) for a Likert scale-based survey where 20 participants i.e., 57.14% were male, and 15 participants i.e., 42.86% were female. Among these participants, 24 were from civil engineering i.e., 68.57%, 6 were from mechanical engineering i.e., 17.14%, and 5 were from electrical engineering i.e., 14.29%. The Mann-Whitney U test result indicated the significance level is .050 and that the distribution of age across gender is the same. Table 3 provides a concise summary of the participant demographics.

Table 3.

Demographics of the participants.

Survey	Participants	Age Range	Mean Age	Gender	Field of Engineering
Expert Survey	15	34-62	51.8	14 (93.33%)/1 (6.67%)	9 (60%) Facility Management & Maintenance, 6 (40%) Civil Engineering & Architecture
Participant Survey	35	18-32	24.7	20 (57.14%)/15 (42.86%)	24 (68.57%) Civil Engineering, 6 (17.14%) Mechanical Engineering, 5 (14.29%) Electrical Engineering

Survey Results

Table 4 and Table 5 provide the key results from the expert and participant surveys , and their alignment with the study’s objectives and research questions.

Table 4.

Results of the Expert Survey.

Results (Expert Survey)	Response (%)
Beneficial simulated training environment	93.33%
Interest in serious 3D game	93.33%
Finding gamified form interesting	93.33%
Belief in improving labor efficiency	93.33%
First-person preferred	73.33%
Preferred training mode	60% Story with simple instructions, 26.67% Story only,13.33% Basic simulation with simple instructions
Interest in including immersive technologies	93.33%

Table 5.

Results of the Participant Survey.

Results (Participant Survey)	Scale (% chosen)
Question	1	2	3	4	5
Found the game very interesting	0%	0%	5.71%	48.57%	45.71%
Easy to understand the concept	0%	0%	5.71%	31.43%	62.86%
Easy to understand chiller plant issues	0%	2.86%	5.71%	42.86%	48.57%
Found in-game controls very easy	0%	0%	5.71%	54.29%	40.00%
Found the story flow smooth and engaging	0%	0%	2.86%	42.86%	54.29%
Satisfactory graphics and setting	0%	0%	5.71%	60.00%	34.29%
Learned how to manage issues with the game	0%	2.86%	5.71%	51.43%	40.00%
Recommended using training game for staff training	0%	2.86%	5.71%	57.14%	34.29%
Believed simulated training game is more engaging than verbal instructions	0%	0%	2.86%	57.14%	40.00%
Believed training in a synthetic task-based environment is safer	0%	0%	22.86%	40.00%	37.14%

Results of Participant Survey

It is to be noted that no participants chose scale 1, and overall, the results demonstrated positive output from the participants and a correlation of interest in using immersive serious 3D training games with the experts and the players who may pursue facility management and maintenance as a career choice.

Linking Results to Methods

The result of the expert survey demonstrates a high level of interest and positive feedback regarding the use of serious 3D games for personnel training, which aligns with the research objective of investigating potential methods of integrating serious 3D games in industrial training for facility management and maintenance.

The positive feedback from the participant survey indicates that the developed serious 3D personnel training game was engaging, effective for learning, and useful for staff training, which links to the research objective of introducing an approach to personnel training that is more accessible and minimizes risk.

Overall, the results show a strong correlation between expert opinions and participant feedback, supporting the research objectives and indicating the effectiveness of using serious 3D games for personnel training in facility management and maintenance of TES chiller plants.

Discussion

The feedback received from experts and engineers suggests a favorable outcome for the research, indicating interest and positive experiences with the developed serious 3D personnel training game. The engagement of potential users further supports the idea that serious games can serve as valuable tools for personnel training.

Regarding the first research question (RQ1), experts emphasized the effectiveness of using simulated 3D environments with instructions for training personnel in TES-AC chiller plants. A significant majority of participants (93.33%) found this approach beneficial, showcasing its potential to enhance skills in a controlled synthetic task environment. This finding aligns with the research objective of understanding experts’ perspectives on the implementation of serious 3D games for personnel training.

Addressing the second research question (RQ2), 60% of participants expressed a preference for a story mode approach with simple instructions, indicating a desire for an engaging and immersive learning experience. This preference, along with the high favorability (73.33%) for a first-person perspective, emphasizes the importance of immersion and interactive learning in the training process. These insights contribute to identifying key features for inclusion in a serious 3D personnel training game.

For the third research question (RQ3), the positive feedback (94.29%) from participating engineers indicates their grasp of educational concepts and management techniques for TES chiller plants. Additionally, a high percentage (97.14%) found the gameplay smooth and engaging, supporting the effectiveness of the serious 3D personnel training game. This finding aligns with the research objective of exploring engineers’ perceptions of the game’s usability.

The preference for a serious 3D game approach to personnel training is attributed to its active learning environment, allowing participants to practice scenarios independently and handle management issues without real crises pressure. The majority (77.14%) found the synthetic task environment to be safer for learning, aligning with the research goal of introducing a training approach that minimizes risk and encourages efficient management practices.

The strong correlation between expert suggestions and engineer feedback underscores the effectiveness of incorporating expert insights into the development of the serious 3D personnel training game. Both groups expressed interest in utilizing immersive games for training, highlighting the potential value of serious 3D games over traditional verbal knowledge sharing.

In conclusion, the positive feedback from both experts and engineers underscores the potential effectiveness and value of implementing serious 3D games for personnel training in facility management and maintenance of TES Chiller Plants. The immersive nature of these training methods promotes active learning, skill development, and safer management practices within a controlled environment.

Limitations and Suggestions for Further Research

This section shortly gives an overview of the limitations that this research identifies including giving a few suggestions for future research in this field.

Limitations

This research proposes a 3D serious training personnel game which can have many benefits but also has its limitations. Due to the nature of the game being 3D, graphical models need to be used, however, that increases the system requirement to run the game. Despite the game using mostly low-poly 3D models, there are still other factors that contribute to the system power required to run the game smoothly. Therefore, this game can only perform on computers that at least have a dedicated graphics card. Computers that only rely on integrated graphics cards will not be able to run this game smoothly. This is important to know because some workplaces do not invest in computers with dedicated graphics cards as their normal daily use does not require it.

Another limitation is the actual gameplay of the game. The game has easy controls, but it still helps if the user is familiar with very basic movement in 3D computer games. For any user who has played video games on a computer before and even for a brief period will not have any trouble playing this one. However, users that have not played any sort of video game before might have to spend some time getting used to the controls. That should not be a very big problem for those users but will definitely take some extra time.

Additionally, the absence of a control group poses a hindrance to comparing the effectiveness of the serious 3D game training method with alternative or traditional training approaches. With a control group, a baseline for comparison would be established, enabling a more comprehensive evaluation of the game’s influence on learning outcomes. The results of this study might be limited in their applicability to the specific context of TES chiller plants. It is crucial to contemplate whether these findings can be extrapolated to other domains within facility management or maintenance.

Future Research Suggestions

This research could be further improved in multiple ways, to better determine the efficiency of 3D serious games in personnel training for facility management. Firstly, a control group could be introduced that will be trained to handle chiller plant maintenance issues, but they will be trained by a senior staff member. The two groups could then be tested in their knowledge of handling different maintenance issues and compared to see if the staff members trained with the serious 3D game performed the same or better. Furthermore, a Virtual Reality version of the game could be developed to see if training the personnel on the same game but in VR is more effective. VR is more immersive than normal serious 3D games and allows the users to experience a more real life-like experience which might improve the effectiveness of the training.

Although the study demonstrates positive engagement and interest in serious 3D game training, exploring its long-term effects becomes essential. Investigating the retention of knowledge and skills over an extended period could offer valuable insights into the sustainability and effectiveness of the training method over time. In future investigations, an exploration of qualitative methodologies, including open-ended inquiries, could offer a deeper understanding of the diverse perspectives held by experts and participants on the implementation of serious games in educational contexts. This approach may unveil nuanced factors influencing participants’ experiences and educational outcomes, providing valuable insights for the development and optimization of serious game-based educational interventions. Evaluating the cost implications of developing and implementing a serious 3D game for training purposes is crucial. Considering the financial investment necessary for creating and maintaining the game would provide a more comprehensive assessment of its feasibility and practicality in real-world applications. This cost analysis could help decision-makers weigh the benefits against the expenses associated with adopting this training approach.

A final improvement could be to the current game, by including a new interactive immersive method like the LEAP motion controller. This controller is essentially a camera motion sensor that can detect hand tracking in a very effective method. It can easily be installed on a computer and integrated into the game as it is compatible with the Unity game engine. This will give the game a more immersive feeling as the users can do actions using their hand motions, making this improvement a good option that strikes balance between normal 3D games and VR games. The users can get that immersive interaction experience from VR but without the negative impacts of having to wear the VR set.

Conclusion

This correlational research investigated the relationship between expert recommendations for personnel training using a serious 3D game and the usability experience of the game among engineers in the context of facility management and maintenance of TES Chiller Plants. The study aimed to identify key features necessary for a successful training game and explored the potential integration of serious 3D games in industrial training for facility management and maintenance. The research questions were restructured into empirical hypotheses, which confirmed that experts provided feedback on the proposed 3D serious personnel training game and its specific features, while engineers positively rated the usability and usefulness of the game after engaging with it.

This study’s uniqueness lies in its innovative approach to personnel training, offering an immersive learning experience for managing common problems in TES chiller plants. By simulating tasks in a gamified setting, personnel can actively learn and practice problem-solving without real-life pressures. The study gathered suggestions to develop the serious 3D training game and assessed user satisfaction, as well as experts’ interest in integrating immersive technologies for training. Overall, this research presents a promising avenue for accessible and risk-minimized personnel training in the domain of facility management and maintenance.

According to the findings of this research, serious 3D games can be potentially integrated as training games for personnel training for improving management with prompt decision-making. Although this research focused on management issues of TES chiller plants, the findings suggest facility management and maintenance can overall be improved with integrating digital technologies such as serious 3D games for personnel training and in this way, overall efficiency of a facility would be improved. There is also a correlation between the interest of experts in integrating serious 3D games for personnel training and young individuals who are keen to pursue the facility management and maintenance field as a career choice as both the groups think it will be a beneficial addition. The reasons behind the interest in integrating serious 3D games as a personnel training technique are the accessibility, rehearsing training methods, and mainly going through an immersive story-mode experience that has a positive impact on episodic memory which may lead to better decisions when issues arise.

The experts suggested features for the training game for management of a TES chiller plant and showed interest in integrating them for personnel training, and the participants who played the game showed preference and interest in using such games for training for management and better decision-making over the traditional method of verbally sharing knowledge. Serious 3D games give a more immersive learning experience, and a synthetic task environment where all the personnel can be in an active learning method handling issues by themselves instead of a real-life problem, so they are better prepared, and they do not simply observe other staff handling issues to learn. As facility management and maintenance is a crucial field of BIM methodology which is also the most expensive phase, this research suggests integrating serious 3D games as an innovative method to improve labour efficiency.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) received no financial support for the research,authorship,and/or publication of this article.

Informed Consent

Informed consent was obtained from the participants before they agreed to proceed to participate in the study. Additionally,they were given a brief overview of the research study design and why their participation was important. The participants were notified that no personal data that could identify them would be stored or shared. The only data that is beneficial to the study such as their response to the surveys would be collected.

ORCID iD

Mirza Rayana Sanzana

Author Biographies

Mirza Rayana Sanzana,a PhD holder in Civil Engineering specializing in Artificial Intelligence applications,is a dedicated researcher for climate action. Her work focuses on leveraging AI for positive environmental impact and addressing societal challenges. Passionate about AI for good,she explores innovative teaching methods to enhance knowledge and memory retention. Rayana’s multidisciplinary expertise positions her as a thought leader,contributing to a sustainable and equitable future through her impactful research and advocacy.

Mostafa Osama Mostafa Abdulrazic,a PhD candidate in computer science,is fervently dedicated to leveraging technology for social good. His doctoral studies center on harnessing AI,serious 3D games,and IoT to address pressing societal challenges. Committed to making a meaningful impact,Mostafa’s research aims to create innovative solutions that transcend traditional boundaries. Additionally,he is passionate about molding young minds through advanced learning technologies,particularly in the realm of maker education. Mostafa exemplifies a visionary researcher striving to pioneer positive change through cutting-edge technology and education.

Jing Ying Wong,an esteemed Associate Professor,is a trailblazer in the realm of education with a specific focus on building information modeling (BIM). Her research is dedicated to advancing innovative teaching,learning,and training methodologies tailored for science and engineering students,integrating BIM into educational practices. With a commitment to transformative educational strategies,Jing Ying ensures that the next generation of scientists and engineers not only master their disciplines but also engage with cutting-edge technologies.

Chun-Chieh Yip,Head of the Department of Civil Engineering,stands as a distinguished leader,educator,and visionary researcher. Chun-Chieh focuses on integrating technologies for innovative teaching and learning within the civil engineering domain. His research extends to the realm of sustainable buildings,reflecting a dedication to environmentally conscious practices.

References

Abdulrazic

M. O.

Sanzana

M. R.

K. H.

(2022). Integrating Internet of Things (IoT) in Cultural Game Authoring Tool: An Innovative Approach in Maker Education. In 9th Electronic Conference on Sensors and Applications. Basel, Switzerland: MDPI. https://doi.org/10.3390/ecsa-9-13371

Almeida

Simoes

(2019). The Role of Serious Games, Gamification and Industry 4.0 Tools in the Education 4.0 Paradigm. Contemporary Educational Technology, 10(2), 120-136. https://doi.org/10.30935/cet.554469

Araszkiewicz

(2017). Digital technologies in Facility Management – the state of practice. In Creative Construction Conference 2017, CCC 2017. Primosten, Croatia.

Attri

R. G. S.

(2014). Decision making over the production system life cycle: MOORA method. International Journal of System Assurance Engineering and Management, 5(3), 320-328. https://doi.org/10.1007/s13198-013-0169-2

Azadegan

Riedel

J. C.

Hauge

J. B.

(2012). Serious Games Adoption in Corporate Training. In International Conference on Serious Games Development and Applications. 7528, pp. 74-85. Berlin: SpringerLink. https://doi.org/10.1007/978-3-642-33687-4_6

Bachvarova

Bocconi

Pols

B. v.

Popescu

Roceanu

(2012). Measuring the Effectiveness of Learning with Serious Games in Corporate Training. Procedia Computer Science, 15, 221-232. https://doi.org/10.1016/j.procs.2012.10.074

Bellotti

Kapralos

Lee

Moreno-Ger

Berta

(2013). Assessment in and of serious games: an overview. Advances in Human-Computer Interaction, 1(2), 6. https://doi.org/10.1155/2013/136864

Boyle

Connolly

Hainey

(2011). The role of psychology in understanding the impact of computer games. Entertainment Computing, 2(2), 69-74. https://doi.org/10.1016/j.entcom.2010.12.002

Buzady

Almeida

(2019). FLIGBY—A Serious Game Tool to Enhance Motivation and Competencies in Entrepreneurship. Informatics, 6(3), 27. https://doi.org/10.3390/informatics6030027

10.

Catalano

C. E.

Luccini

A. M.

Mortara

(2014). Guidelines for an effective design of serious games. International Journal of Serious Games, 1(2), 4. https://doi.org/10.17083/ijsg.v1i1.8

11.

Connolly

T. M.

Hainey

Boyle

Baxter

Moreno-Ger

(2013). Psychology, pedagogy, and assessment in serious games. In A volume in the Advances in Game-Based Learning (AGBL) Book Series. IGI Global. Retrieved from https://books.google.com.bd/books?hl=en&lr=&id=V-BGAwAAQBAJ&oi=fnd&pg=PR1&ots=dKZNpOTjiH&sig=7WzjhpjK9fublgzpVfnHEZ3V54A&redir_esc=y#v=onepage&q&f=false

12.

Coupry

Noblecourt

Richard

Baudry

Bigaud

(2021). BIM-Based Digital Twin and XR Devices to Improve Maintenance Procedures in Smart Buildings: A Literature Review. Applied Sciences, 11(15). https://doi.org/10.3390/app11156810

13.

Flack

Voltz

Dill

Lin

Reith

(2020). Leveraging Serious Games in Air Force Multi-Domain Operations Education: A Pilot Study. ICCWS 2020 15th International Conference on Cyber Warfare and Security, Norfolk Virginia, 2020, (p. 155). Academic Conferences and Publishing Limited.

14.

Flyvbjerg

(2014). What you should know about megaprojects and why: An overview. Project Management Journal, 45(2), 6-19. https://doi.org/10.1002/pmj.21409

15.

Gillis

A. S.

(2022, 07). Augmented reality (AR). Retrieved 09 01, 2022, from TechTarget: https://www.techtarget.com/whatis/definition/augmented-reality-AR

16.

Green

C. S.

Bavelier

(2003). Action video game modifies visual selective attention. Nature, 423, 534-537. https://doi.org/10.1038/nature01647

17.

Greitzer

F. L.

(2005). Ingredients of effective and engaging online learning. In Inter-Lab 2005 Conference. Slides, 2005.

18.

Greitzer

F. L.

Kuchar

O. A.

Huston

(2007). Cognitive science implications for enhancing training effectiveness in a serious gaming context. Journal on Educational Resources in Computing, 7(3), 6. https://doi.org/10.1145/1281320.1281322

19.

Greitzer

F. L.

Pond

D. J.

Jannotta

(2004). Scenario-Based Training on Human Errors. In Interservice/Industry Training, Simulation, and Education Conference, Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) Virginia: National Training Systems Association (NTSA) 2004 . Arlington, 2004, (pp. 1-11).

20.

Haines

(2016). The Benefits of Lifecycle BIM for Facility Management. Retrieved 11 2019, from fm: systems: https://fmsystems.com/blog/the-benefits-of-lifecycle-bim-for-facility-management/#:∼:text=BusinessInformationModeling(BIM)has,transformedarchitecture%2Cengineeringandconstruction.&text=Facilitymanagersarefindingvalue,th

21.

Hakkarainen

Woodward

Billinghurst

(2008). Augmented Assembly using a Mobile Phone. In Proceedings 7th IEEE International Symposium on Mixed and Augmented Reality (pp. 167-168). 2008, Cambridge, UK: IEEE. https://doi.org/10.1109/ISMAR.2008.4637349

22.

Haoran

Bazakidi

Zary

(2019). Serious Games in Health Professions Education: Review of Trends and Learning Efficacy. Yearbook of Medical Informatics, 28(1), 240-248. https://doi.org/10.1055/s-0039-1677904

23.

Hocine

(2019). Personalized Serious Games for Self-regulated Attention Training. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, (pp. 251-255). 2019, Larnaca Cyprus. https://doi.org/10.1145/3314183.3323458

24.

Leitão

T. M.

Navarro

L. L.

Cameira

R. F.

Silva

E. R.

(2021). Serious games in business process management: a systematic literature review. Business Process Management Journal, 27(3). https://doi.org/10.1108/BPMJ-07-2020-0346

25.

López

F. R.

Arias-Oliva

Pelegrín-Borondo

Marín-Vinuesa

L. M.

(2021). Serious games in management education: An acceptance analysis. The International Journal of Management Education, 19(3). https://doi.org/10.1016/j.ijme.2021.100517

26.

Yan

Yang

(2020). Occupational stress and psychological health impact on hypertension of miners in noisy environment in Wulumuqi, China: a case-control study. BMC Public Health, 1675(2020), 10. https://doi.org/10.1186/s12889-020-09760-9

27.

Miguel

N. P.

Lage

J. C.

Galindez

A. M.

(2020). Assessment of the Development of Professional Skills in University Students: Sustainability and Serious Games. Sustainability, 12(3), 1014. https://doi.org/10.3390/su12031014

28.

Min

Kim

(2022). Effectiveness of serious games in nurse education: A systematic review. Nurse Education Today, 108. https://doi.org/10.1016/j.nedt.2021.105178

29.

Mittal

Scholten

Kapelan

(2022). A review of serious games for urban water management decisions: current gaps and future research directions. Water Research, 215. https://doi.org/10.1016/j.watres.2022.118217

30.

Moro

Phelps

Stromberga

(2020). Utilizing serious games for physiology and anatomy learning and revision. Advances in Physiology Education, 44(3), 505-507. https://doi.org/10.1152/advan.00074.2020

31.

Roper

K. O.

Payant

(2014). The Facility Management Handbook. AMACOM.

32.

Rumeser

Emsley

(2018). Can Serious Games Improve Project Management Decision Making Under Complexity? Project Management Journal, 50(1). https://doi.org/10.1177/8756972818808982

33.

Sanzana

M. R.

Abdulrazic

M. O.

Wong

J. Y.

Yip

C.-C.

(2023). Gamified virtual labs: shifting from physical environments for low-risk interactive learning. Journal of Applied Research in Higher Education, 16(1). https://doi.org/10.1108/JARHE-09-2022-0281

34.

Sanzana

M. R.

Abdulrazic

M. O.

Wong

J. Y.

K. H.

Ghazy

(2021). Lecture-based, virtual reality game-based and their combination: which is better for higher education? Journal of Applied Research in Higher Education. 7(1), 67. https://doi.org/10.1108/JARHE-09-2020-0302

35.

Sharifzadeh

Kharrazi

Nazari

Tabesh

Khodabandeh

M. E.

Heidari

Tara

(2020). Health Education Serious Games Targeting Health Care Providers, Patients, and Public Health Users: Scoping Review. JMIR Serious Games, 8(1), 45. https://doi.org/10.2196/13459

36.

Solinska-Nowak

Magnuszewski

Curl

French

Keating

Mochizuki

Jarzabek

(2018). An overview of serious games for disaster risk management – Prospects and limitations for informing actions to arrest increasing risk. International Journal of Disaster Risk Reduction, 31, 1013-1029. https://doi.org/10.1016/j.ijdrr.2018.09.001

37.

Stettina

C. J.

Offerman

Mooij

B. D.

Sidhu

(2018). Gaming for Agility: Using Serious Games to Enable Agile Project & Portfolio Management Capabilities in Practice. 2018 IEEE International Conference on Engineering, Technology and Innovation. 2018, Stuttgart, Germany: IEEE. https://doi.org/10.1109/ICE.2018.8436384

38.

Stutzman

Nilsen

Broderick

Neubert

(2009). MARTI: Mobile Augmented Reality Tool for Industry. In Proceedings of 2009 World Congress on Computer Science and Information Engineering (pp. 425-429). IEEE. https://doi.org/10.1109/CSIE.2009.930

39.

Suamir

I. N.

Arsana

M. E.

Subagia

I. W.

Rasta

Midiani

L. I.

Wibolo

(2019). Site investigation on water cooled chiller plant for energy conservation and environmental impact reduction of a large shopping mall. AIP Conference Proceedings, 2187(1). https://doi.org/10.1063/1.5138297

40.

Sullivan

(2016). A New Era of Service Integration, Energy Management, Business Productivity, Smart Technology, and Internationalisation. The Future of Facility Management.

41.

Sutil-Martín

D. L.

Otamendi

F. J.

(2021). Soft Skills Training Program Based on Serious Games. Sustainability, 13(15), 8582. https://doi.org/10.3390/su13158582

42.

Target

(2015). What Is.com. Retrieved 07 2023, from Tech Target: https://www.techtarget.com/whatis/definition/first-person-view-FPV

43.

Tuholski

S. W.

Engle

R. W.

Baylis

G. C.

(2001). Individual differences in working memory capacity and enumeration. Memory & Cognition, 29, 484-492. https://doi.org/10.3758/BF03196399

44.

Ventures

(2021). Discover how BIM is implemented in each step of the construction value chain. Retrieved 09 2, 202209, from https://www.cemexventures.com/discover-how-bim-is-implemented-in-each-phase-of-the-construction-industry/

45.

Wang

R. B.

DeMaria

S. J.

Goldberg

A. M.

Katz

D. M.

(2016). A Systematic Review of Serious Games in Training Health Care Professionals. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 11(1), 41-51. https://doi.org/10.1097/SIH.0000000000000118

46.

Yusoff

Crowder

Gilbert

Wills

(2009). A Conceptual Framework for Serious Games. In 2009 Ninth IEEE International Conference on Advanced Learning Technologies. IEEE. https://doi.org/10.1109/ICALT.2009.19