Sage Journals: Discover world-class research

Abstract

The English version of the Independent Television Commission-Sense of Presence Inventory (ITC-SOPI), which was developed in 2001 to measure how involved or present participants are when experiencing different media, has substantial psychometric evidence. This study was used to translate and validate the ITC-SOPI in interactive virtual environments among the Chinese population. We used the forward-backward translation procedure. An expert panel reviewed the translated ITC-SOPI until the Chinese version of the ITC-SOPI was finalized. A total of 210 participants (133 males and 77 females), with a mean age of 23.05 years (SD = 3.56, range = 17–47), completed the Chinese ITC-SOPI. The following psychometric properties were examined: factor structure, internal consistency, test–retest reliability, and convergent validity. Confirmatory factor analysis (CFA) showed a good fit (χ²/df = 1.70, Tucker–Lewis Index [TLI] = 0.91, comparative fit index [CFI] = 0.92, root mean square error of approximation [RMSEA] = 0.058) of the four-factor model (spatial presence, engagement, ecological validity, and negative effects). For each factor, the Chinese ITC-SOPI had high internal consistency (Cronbach’s α ranging from 0.75 to 0.87) and test–retest reliability (intraclass correlation coefficient ranging from 0.82 to 0.91). Significant correlations were identified between all factors and the Interpersonal Reactivity Index-C (IRI-C) and the Generalized Anxiety Disorder-7 (GAD-7). The Chinese ITC-SOPI had good psychometric properties, suggesting that it is a reliable and valid tool for evaluating media users’ sense of presence in a Chinese-speaking context.

Keywords

sense of presence virtual reality reliability validity Chinese population

Introduction

The concept of presence has commanded much research. Scholars have argued about its different definitions (Ghani et al., 2016). Sheridan (1992) described presence as “feeling like you are present in the environment generated by the computer” (p. 120). Parsons et al. (2017) divided presence into two main perspectives: one refers to an interactive experience with the media and the other, a phenomenon linked to the “organization and control of action” (p. 3). Waterworth et al. (2015) defined presence as “being physically present somewhere” (p. 36), whereas Jacobson (2002) described it as “engagement in a virtual world” (p. 1). In general, presence reflects how real one’s feeling is about the virtual environment depending on individual characters and media qualities.

Sense of presence is an intermediate mechanism of media technology, which helps us evaluate the media and understand user experience (Lessiter et al., 2001). The need to explore ways of creating a higher sense of presence is increasing. However, there is still no clear consensus on how to measure sense of presence, as reflected in existing self-report measures. Witmer and Singer (1998, 2005) developed the Presence Questionnaire (PQ) in 1998 and then re-validated it in 2005. They found that four factors (i.e., involvement, adaptation/immersion, sensory fidelity, and interaction quality) affect one’s involvement and immersion, which measure one’s sense of presence. Schubert et al. (2001) developed the Igroup Presence Questionnaire (IPQ) based on the embodied cognition framework and identified three factors (i.e., spatial presence, involvement, and judgment of reality) to measure sense of presence. Lombard et al. (2000) developed the Temple Presence Inventory (TPI), which includes eight factors measuring sense of presence: spatial presence, parasocial interaction, passive interpersonal presence, active interpersonal presence, engagement (mental immersion), social richness, social realism, and perceptual realism. Among these questionnaires, Witmer and Singer (1998, 2005) focused on environment interaction, whereas Lombard et al. (2009) on social presence. Scales that focus on spatial presence, involvement, and immersion cannot be used for measuring cross-media presence.

Based on previous research and existing questionnaires, Lessiter et al. (2001) developed the Independent Television Commission-Sense of Presence Inventory (ITC-SOPI). This study validated the ITC-SOPI because it can be used in measuring cross-media presence in subjective perception settings (Lessiter et al., 2001). This 44-item scale was originally developed in English. A 5-point Likert-type scale (1 = strongly disagree; 5 = strongly agree) was used as the response format for all items. The scale included four factors: Sense of Physical Space, Engagement, Ecological Validity, and Negative Effects. Sense of physical space indicates how individuals are physically involved in a virtual environment. Engagement refers to how individuals are psychologically involved in a virtual environment. Ecological validity measures how real a virtual environment is. Negative effects indicate adverse physiological reactions. In Lessiter et al. (2001), the internal consistencies of all factors were high (spatial presence: 0.94, engagement: 0.89, ecological validity: 0.76, negative effects: 0.77), which means that the ITC-SOPI is a reliable tool to measure sense of presence. José Vasconcelos-Raposo et al. (2019) adapted and validated the ITC-SOPI in a Portuguese population. They used virtual reality (VR) as a medium to collect data on sense of presence from 459 participants. The internal consistencies of all factors ranged from medium to high: 0.89 for spatial presence, 0.67 for engagement, 0.79 for ecological validity, and 0.78 for negative effects.

The ITC-SOPI has not yet been validated in other countries, including China. Therefore, we translated the 44 items of the ITC-SOPI into simplified Chinese. Then we examined factor structure, internal consistency, and test–retest reliability. Wallach et al. (2010) believed that sense of presence, like empathy, is an emotional projection. In this sense, it can be understood as an emotional connection with a place and, therefore, is related to empathy. Stavropoulos et al. (2017) found that there is a strong correlation between anxiety and sense of presence. Based on their finding, we tested the correlations between the Chinese version of the ITC-SOPI and measures of empathy and generalized anxiety to evaluate the former’s convergent validity as the final purpose. Sense of presence was expected to be significantly related to high empathy and severe symptoms of anxiety.

We used VR as the medium for evaluating the Chinese ITC-SOPI. VR is a three-dimensional virtual situation generated by computer simulation. Under a laboratory environment, researchers can study complex human behaviors using VR to simulate a complex reality (Julia et al., 2015). The word “VR” was put forward by Jaron Lanier in the 1980s (Blanchard et al., 1990). Recently, with the growing popularity of commercial VR products, VR technology has been increasingly applied in social well-being fields, such as nursing, mental health, and learning promotion (Chirico et al., 2016). VR can provide high fidelity and elicit “presence” (Chirico et al., 2018; Steed et al., 2016), which means users experience increased presence, engagement, and usability (McMahan et al., 2012). Here, presence refers to an individual psychological response to a VR system (Bowman & McMahan, 2007). It is the intermediary mechanism in which the VR technology works (Price et al., 2011; Wiederhold & Wiederhold, 2005).

Method

Participants

This study was approved by the Ethics Committee of the Department of Psychology of the Tsinghua University (IRB NO. 2019THUPsy56). Of the 226 Chinese-speaking participants from the University Town of Shenzhen who volunteered to participate, 210 (133 males and 77 females) completed the Chinese ITC-SOPI, with a response rate of 92.9%. Participants’ average age was 23.05 years (SD = 3.56, range = 17–47). Test–retest reliability was evaluated using 20 participants (13 males and 7 females, with an average age of 22.4 [range: 18–29; SD = 1.93]) who completed the Chinese ITC-SOPI twice (2 weeks between two occasions). All participants received CNY 100 as incentives.

Translation and Adaptation

We employed the forward-backward translation procedure (Beaton et al., 2000). First, the original English version of the ITC-SOPI was translated into simplified Chinese separately by two translators who are bilingual in their native language Chinese and second language English. One translator was a psychologist and the other, a graduate student of the department of Chinese language and literature. Then, members of an expert panel discussed the two preliminary translations until they reached an agreement with the first synthesized version. Next, two fluent English speakers, a graduate student of a master’s in psychology and an undergraduate from the Department of English, translated the synthesized version back into English. Afterward, members of the expert panel compared the re-translated English version to the original. This process continued until the re-translated and original English versions were consistent. Finally, the expert panel, consisting of the two original translators and one professor with experience in scale development and translation, assessed the final version of the Chinese ITC-SOPI. We randomly selected five postgraduates majoring in psychology and data sciences in the Graduate School at Shenzhen, Tsinghua University to check the scale’s acceptability. After this process, the Chinese ITC-SOPI was finalized for data collection.

Process

Data were collected in November and December 2017. Participants were recruited from the University Town of Shenzhen by posting an announcement on campus and social media like WeChat. Potential participants were added in a WeChat group in which a detailed information on this experiment was provided. The VR staff members completed a 2-hr training that emphasized on standardizing the administration procedures to improve data quality. The experiment was held at the VR lab of the Graduate School at Shenzhen, Tsinghua University (Figure 1). First, study purpose and questionnaire characteristics were described to participants. Informed consent was obtained verbally. Participants were assured of the anonymity and confidentiality of their answers. Second, participants wore the VR headset, assisted by an experiment staff member. Third, participants played the four chosen games in sequence. Finally, participants were asked to fill out a written questionnaire independently. The total time of completing the experiment was approximately 45 min: 30 min to experience VR games and 15 min to complete the questionnaire.

Figure 1.

Participants undertaking the experiment at the VR lab (participants provided consent).

The questions were reordered in the second data collection to obtain test–retest reliability. Participants were not informed that they would be completing the same questionnaire. They all confirmed that they did not have any VR experience between the two tests.

Instruments

Virtual reality

Data were obtained using the Chinese version of the ITC-SOPI. A detailed information on ITC-SOPI was provided in the introduction. This section discusses a more detailed information on VR games. The current VR headset that provides the best experience is the external Head Mount Display, such as Oculus Rift, HTC Vive, and Sony PS VR. Among these headsets, HTC Vive can provide a more fully immersive first-person experience and more realistic interaction. That is why, we selected HTC Vive as the media display. Participants’ experiences were based on the HTC Vive headset, STEAM, and VIVEPORT APP market. The PC specifications are as follows: Lenovo Y720, CPU Intel® Core™ i7-7,700, Operating System Windows 10, GPU NVIDIA® GeForce® GTX 1070. Considering the item content in ITC-SOPI, we selected the following four games with a realistic environment and interaction for participants to experience: theBlu, Desert Ride, Fist to Legend, and Fruit Ninja (Figure 2). theBlu provides players with an undersea world experience in which they can interact with the surrounding fish. Desert Ride is a roller coaster game. Fist to Legend is a boxing game. Fruit Ninja is a VR version of Fruit Slice. Data were also obtained from two other scales, in addition to the Chinese version of the ITC-SOPI, for criterion validity evidence.

Figure 2.

Games (theBlu, Desert Ride, Fist to Legend, and Fruit Ninja) used in the experiment.

Interpersonal Reactivity Index-C

The Interpersonal Reactivity Index-C (IRI-C) is a Chinese version of the Interpersonal Reactivity Index. Empathy includes emotional and cognitive empathy. The IRI-C evaluates empathy from a cognitive and emotional perspective, making the assessment of empathy more comprehensive (Davis, 1980, 1983). The IRI-C consists of 22 items scored on a 5-point Likert-type scale, ranging from 1 (do not agree) to 5 (agree very much). It contains four factors, namely, fantasy, perspective-taking, empathic concern, and personal distress. In previous studies, the IRI-C’s Cronbach’s α was .75 and split-half reliability was 0.73 (Rong et al., 2010). In this study, its Cronbach’s α was .76.

Generalized Anxiety Disorder-7

The Generalized Anxiety Disorder-7 (GAD-7) is a seven-item scale measuring generalized anxiety (Spitzer et al., 2006). It measures participants’ level of anxiety, such as whether they are nervous, worried, or annoyed, in the past 2 weeks. Response options include “not at all,” “several days,” “more than half of the days,” and “nearly every day,” which are scored as 0, 1, 2, and 3, respectively. In a previous study, Cronbach’s α of GAD-7 was .89 and the correlation coefficient between items and total score was higher than .63 (Löwe et al., 2008). In this study, Cronbach’s α of GAD-7 was .84.

Data Analysis

In our dataset, 11 of 210 completed responses were missing completely at random (MCAR). Therefore, we used mean imputation for missing data throughout the analyses (missing processing for the factor analysis).

Mplus version 7.2 was used for factor analysis and IBM SPSS Statistics version 20.0, for other analyses. Before conducting factor analysis, item-total correlations were examined to detect low-performing items. Based on a presence theory’s suggestion on the use of a four-factor structure and the previous development of the scale, we employed the confirmatory factor analysis (CFA) to validate the factor structure of the Chinese ITC-SOPI. Data were not normally distributed. The kurtosis and skewness values were 1.12 and −0.17 for spatial presence, 0.51 and −0.26 for engagement, 0.01 and −0.20 for ecological validity, and 0.38 and 0.58 for negative effects. Therefore, the maximum likelihood (ML) estimation was not appropriate for analysis (Finney & DiStefano, 2013). We used the weighted least square with mean and variance correction (WLSMV), which is the default estimation method for ordinal data, to evaluate the fitness of the original four-factor model. These fit indices were considered. Generally, chi-square/degrees of freedom <3, Tucker–Lewis Index (TLI) ≥0.900, comparative fit index (CFI) ≥0.900, and root mean square error of approximation (RMSEA) ≤0.080 were considered acceptable (Bentler & Bonett, 1980; Browne & Cudeck, 1992; Hu & Bentler, 1999). After the factor structure was confirmed, internal consistency (i.e., Cronbach’s α) and test–retest reliability were examined through factor analyses.

The scale requires summation in four sub-dimensions (Lessiter et al., 2001). Therefore, each of the four-factor scores (each generated by calculating a mean of all completed items contributing to each factor) and all of these four factors (spatial presence, engagement, ecological validity, and negative effects) were analyzed individually. These scores were used to investigate the criterion validity (i.e., relationships between factor scores and other scales).

Results

Factor Analysis

The Kaiser–Meyer–Olkin score was 0.881 (0.50 < KMO < 1) and Bartlett’s test of sphericity was (χ² = 3,718.305, p < .001) significant (Bartlett, 1954; Kaiser, 1970). These indices indicated that the factor analysis was appropriate. The chi-square/degree of freedom was 1.70; TLI was 0.91; CFI was 0.92; and RMSEA ranged from 0.052 to 0.063, with a 90% confidence interval. These results showed goodness of fit of the original four-factor model (spatial presence, engagement, ecological validity, and negative effects) in the Chinese context.

Item Analysis and Reliability

The item-total correlations for the 44 items of the Chinese ITC-SOPI ranged from 0.17 to 0.68. Items A1 (“I felt sad that my experience was over”), A2 (“I felt disorientated”), and B17 (“I paid more attention to the displayed environment than to my own thoughts”) had relatively low item-total correlations (r < 0.3), suggesting that these items failed to measure the same construct as the other items. The deletion of these items can significantly promote relevant internal consistency (Table 1). Consequently, we decided to remove these three items, assuming that it would make the scale more internally consistent.

Table 1.

Means, Standard Deviations, Corrected Item-Total Correlation, and Cronbach’s α of the Deleted A1, A2, and B17 Items.

Item	M	SD	Corrected item-total correlation	Cronbach’s α
A1	2.67	1.18	0.24	.82
A2	2.27	0.95	0.27	.83
B17	3.72	0.83	0.17	.82

The mean, standard deviation, Cronbach’s α, Cronbach’s α with a 95% confidence interval, and test–retest reliability of each factor are presented in Table 2. Internal consistency was examined using Cronbach’s α. Cronbach’s α ranged from 0.75 to 0.87, suggesting good levels of internal consistency for the four factors (Mellinger & Hanson, 2017). Test–retest reliability is an indicator of the stability of ITC-SOPI scores over time. A subset of participants completed the ITC-SOPI for 2 weeks after the initial examination. The intraclass correlation coefficient (ICC) between the two sessions was 0.82 for spatial presence, 0.84 for engagement, 0.90 for ecological validity, and 0.91 for negative effects, which demonstrated high test–retest reliability (Shrout, 1998).

Table 2.

Means, Standard Deviations, Cronbach’s α, and Intraclass Correlation Coefficient (ICC) of the Questionnaires.

Indexes	Spatial presence	Engagement	Ecological validity	Negative effects
Mean	3.75	3.99	3.39	2.34
SD	0.44	0.48	0.64	0.69
Cronbach’s α	0.87	0.84	0.75	0.83
Cronbach’s α [95% CI]	[0.84, 0.90]	[0.81, 0.87]	[0.69, 0.80]	[0.79, 0.86]
ICC	0.82	0.84	0.90	0.91

Convergent Validity

Convergent validity indicates the validity of an instrument relative to other standards. We computed Pearson correlations between the total scores of the four factors of the Chinese ITC-SOPI and IRI-C and these scores and GAD-7 to assess concurrent validity. As presented in Table 3, all the four factors of the Chinese ITC-SOPI showed significantly positive correlations with IRI-C and GAD-7. Specifically, spatial presence and engagement were significantly correlated with IRI-C; ecological validity was significantly correlated with both IRI-C and GAD-7; and negative effect was significantly correlated with GAD-7.

Table 3.

Pearson Correlations Between Scores of Spatial Presence, Engagement, Ecological Validity, Negative Effects, and IRI-C and GAD-7.

Questionnaires	Spatial presence	Engagement	Ecological validity	Negative effects
IRI-C	0.194**	0.216**	0.185**	0.145*
GAD-7	0.138*	0.158*	0.185**	0.350**

Note. IRIC = Interpersonal Reactivity Index-C; GAD = Generalized Anxiety Disorder-7.

p < .05. **p < .01.

Discussion

This study validated the Chinese version of the ITC-SOPI among a convenience sample of 210 participants from the University Town of Shenzhen. We used a standard translation process, which involved forward-backward translation, an expert panel, and pilot testing. Our study provided Chinese-speaking researchers with a self-report measure of sense of presence in cross-media. The issue of how to accurately and conveniently evaluate sense of presence is necessary as it is an important intermediary variable in psychological studies, when researchers apply media, especially immersive media like VR, to their experiments (Price et al., 2001; Wiederhold & Wiederhold, 2005). The Chinese ITC-SOPI was validated with the following psychometric evidence: factor structure, internal consistency, test–retest reliability, and convergent validity. This is a pioneering study on measuring sense of presence in mainland China using the Chinese ITC-SOPI.

The various fit indices indicated that the four-factor model was a good fit in the Chinese version of ITC-SOPI. The Chinese ITC-SOPI consisted of four factors: spatial presence, engagement, ecological validity, and negative effects. This study’s factor analysis results are similar to those in the previous research (Lessiter et al., 2001), indicating that the scale had good structural validity in the Chinese context.

The item-level analysis indicated that items A1 (“I felt sad that my experience was over”), A2 (“I felt disorientated”), and B17 (“I paid more attention to the displayed environment than to my own thoughts”) had relatively low item-total correlations; therefore, deleting these items can significantly promote relevant internal consistency. Especially, in the Chinese context, A1 lacks the reason of “disorientated” and can be understood as “sad” because the experience was over or terrible; A2 also lacks the reason of “paying more attention to the displayed environment” and can be understood as disorienting because the experience was so real or bad. Hence, these three items, especially A1 and A2, can be ambiguous in the Chinese-speaking context. A different understanding of A1 and A2 may lead to an opposite result.

The Chinese ITC-SOPI had high internal consistency. Its Cronbach’s α was .87 for spatial presence, .81 for engagement, .75 for ecological validity, and .80 for negative effects. The results were similar to the original study conducted by Lessiter et al. (2001). Similarly, an acceptable test–retest reliability was obtained. Both separate ICC factor scores between the two sessions were highly correlated: 0.82 for spatial presence, 0.84 for engagement, 0.90 for ecological validity, and 0.91 for negative effects, which demonstrated that the results in the Chinese ITC-SOPI were stable over time.

Significant relationships between the factor scores in the Chinese ITC-SOPI and measures of empathy and anxiety were highlighted. High empathy anxiety levels were associated with a higher score of sense of presence. This result indicated that the Chinese ITC-SOPI had good criterion validity. In other words, people with high empathy and high anxiety may also have high sense of presence. More specifically, spatial presence and engagement were closely correlated with IRI-C; ecological validity was closely correlated with IRI-C and GAD-7; and negative effect was closely correlated with GAD-7. These findings are consistent with the results of other previous studies (Browne & Cudeck, 1992; Wallach et al., 2010).

Limitations

Although our findings showed good reliability and validity of the Chinese ITC-SOPI, study limitations should be considered. First, participants were recruited only from the University Town of Shenzhen. These young and highly educated people may not represent all people in China. Moreover, the sample size of 210 is relatively small compared with the original study (Lessiter et al., 2001). Given these limitations, our research results may reflect only Chinese university students. To improve the generalization of this scale, further studies should be replicated in different settings and populations, including those with low education and older adults who lack technological experiences in general, to verify whether the same results are obtained. Second, we only employed two scales (IRI-C and GAD-7) for criterion validity. More variety in measures of personality trait should be included to examine convergent validity. Third, although standardized translating procedures were used, the translation may have deviations from the original. Therefore, in the future, researchers should verify the accuracy of translated items. Finally, we did not analyze whether each item functioned appropriately. Therefore, a multidimensional Rasch analysis is necessary to examine item fit (i.e., infit and outfit).

Conclusion

The Chinese version of ITC-SOPI was validated with high internal consistency, good test–retest reliability, acceptable convergent validity, and a four-factor model, suggesting that the scale can be used for researchers to evaluate sense of presence in a Chinese-speaking context.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This study was funded by the Humanities and Social Sciences Foundation of China Ministry of Education (18YJAZH065),the RD program of Shenzhen (JCYJ20170307153032483,JCYJ20170817161546744),Shenzhen Key Research Base of Humanities and Social Sciences,and the Interdisciplinary Research Project of Graduate School of Shenzhen of Tsinghua University (JC2017005).

ORCID iD

Bangyi Yan

References

Bartlett

M. S.

(1954). A note on the multiplying factors for various χ2 approximations. Journal of the Royal Statistical Society. Series B (Methodological), 16(2), 296–298.

Beaton

D. E.

Bombardier

Guillemin

Ferraz

M. B.

(2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191.

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.

Blanchard

Burgess

Harvill

Lanier

Lasko

Oberman

Teitel

(1990). Reality built for two: A virtual reality tool. ACM SIGGRAPH Computer Graphics, 24(2), 35–36.

Bowman

D. A.

McMahan

R. P.

(2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36–43.

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258.

Chirico

Ferrise

Cordella

Gaggioli

(2018). Designing awe in virtual reality: An experimental study. Frontiers in Psychology, 8, Article 2351. https://doi.org/10.3389/fpsyg.2017.02351

Chirico

Yaden

D. B.

Riva

Gaggioli

(2016). The potential of virtual reality for the investigation of awe. Frontiers in Psychology, 7, Article 1766. https://doi.org/10.3389/fpsyg.2016.01766

Davis

M. H.

(1980). A multidimensional approach to individual differences in empathy. Journal of Personality and Social Psychology, 10, 85–101.

10.

Davis

M. H.

(1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44(1), 113–126.

11.

Finney

S. J.

DiStefano

(2013). Nonnormal and categorical data in structural equation models. In Hancock

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (2nd ed., pp. 439–492). Information Age.

12.

Ghani

Rafi

Woods

(2016, October). Sense of place in immersive architectural virtual heritage environment [Conference session]. International Conference on Virtual System & Multimedia, Kuala Lumpur, Malaysia. https://doi.org/10.1109/vsmm.2016.7863169

13.

L.-t.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

14.

Jacobson

(2002). On theorizing presence. Journal of Virtual Environments, 6, 1–11.

15.

Julia

Alpers

G. W.

Peperkorn

H. M.

Youssef

Andreas

(2015). The impact of perception and presence on emotional reactions: A review of research in virtual reality. Frontiers in Psychology, 6, Article 26. https://doi.org/10.3389/fpsyg.2015.00026

16.

Kaiser

H. F.

(1970). A second generation little jiffy. Psychometrika, 35(4), 401–415.

17.

Lessiter

Freeman

Keogh

Davidoff

(2001). A Cross-Media Presence Questionnaire: The ITC-sense of presence inventory. Presence: Teleoperators and Virtual Environments, 10(3), 282–297.

18.

Lombard

Ditton

T. B.

Crane

Davis

Gilegui

Horvath

(2000, March). Measuring presence: A literature-based approach to the development of a standardized paper-and-pencil instrument [Conference session]. Third International Workshop on Presence, Delft, The Netherlands.

19.

Lombard

Ditton

T. B.

Weinstein

(2009, November). Measuring presence: The temple presence inventory [Conference session]. Proceedings of the 12th Annual International Workshop on Presence, Los Angeles, CA, United States.

20.

Löwe

Decker

Müller

Brähler

Schellberg

Herzog

(2008). Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Medical Care, 46(3), 266–274.

21.

McMahan

R. P.

Bowman

D. A.

Zielinski

D. J.

Brady

R. B.

(2012). Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Transactions on Visualization and Computer Graphics, 18(4), 626–633.

22.

Mellinger

C. D.

Hanson

T. A.

(2017). Quantitative research methods in translation and interpreting studies. Routledge.

23.

Parsons

Gaggioli

Riva

(2017). Virtual reality for research in social neuroscience. Brain Sciences, 7(42), 1–21.

24.

Price

Mehta

Tone

E. B.

Anderson

P. L.

(2011). Does engagement with exposure yield better outcomes? Components of presence as a predictor of treatment response for virtual reality exposure therapy for social phobia. Journal of Anxiety Disorders, 25(6), 763–770.

25.

Rong

Sun

Huang

Cai

(2010). The reliability and validity of the Interpersonal Reactivity Index-C (IRI-C). Chinese Journal of Clinical Psychology, 18(2), 158–160.

26.

Schubert

Friedmann

Regenbrecht

(2001). The experience of presence: Factor analytic insights. Presence Teleoperators & Virtual Environments, 10(3), 266–281.

27.

Sheridan

T. B.

(1992). Musings on telepresence and virtual presence. Presence: Teleoperators and Virtual Environments, 1(1), 120–126.

28.

Shrout

P. E.

(1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317.

29.

Spitzer

R. L.

Kroenke

Williams

J. B.

Löwe

(2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097.

30.

Stavropoulos

Wilson

Kuss

Griffiths

Gentile

(2017). A multilevel longitudinal study of experiencing virtual presence in adolescence: The role of anxiety and openness to experience in the classroom. Behaviour & Information Technology, 36(5), 524–539.

31.

Steed

Friston

Lopez

M. M.

Drummond

Swapp

(2016). An “in the wild” experiment on presence and embodiment using consumer virtual reality equipment. IEEE Transactions on Visualization and Computer Graphics, 22(4), 1406–1414.

32.

Vasconcelos-Raposo

Melo

Teixeira

Cabral

Bessa

(2019). Adaptation and validation of the ITC-sense of presence inventory for the Portuguese language. International Journal of Human-Computer Studies, 125, 1–6.

33.

Wallach

H. S.

Safir

M. P.

Samana

(2010). Personality variables and presence. Virtual Reality, 14(1), 3–13.

34.

Waterworth

J. A.

Waterworth

E. L.

Riva

Mantovani

(2015). Presence: Form, content and consciousness. In Lombard

Biocca

Freeman

IJsselsteijn

Schaevitz

R. J.

(Eds.), Immersed in media: Telepresence theory, measurement & technology (pp. 35–58). Springer.

35.

Wiederhold

B. K.

Wiederhold

M. D.

(2005). The effect of presence on virtual reality treatment. In Wiederhold

B. K.

Wiederhold

M. D.

(Eds.), Virtual reality therapy for anxiety disorders: Advances in evaluation and treatment (pp. 77–86). American Psychological Association.

36.

Witmer

B. G.

Jerome

C. J.

Singer

M. J.

(2005). The factor structure of the Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 14(3), 298–312.

37.

Witmer

B. G.

Singer

M. J.

(1998). Measuring presence in virtual environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 7(3), 225–240.