Sage Journals: Discover world-class research

Abstract

Mobile phone usage is typically measured via self-reporting. However, scholars have questioned the validity of self-reported data, which may lead to Type I or Type II errors. Using an online survey (n = 777), this study compared self-reported and log mobile phone usage data using a simplified version of the mobile data donation method. The results showed that people generally underreported their mobile phone usage in terms of time duration, the number of notifications, and apps used. Moreover, self-reported data may either have no additional effect on or overestimate the communication findings depending on the outcome variables. This challenges the Type II error explanation and suggests that the effect sizes of self-reported data might not be underestimated after all. Instead, past research examining mobile use and pertinent outcomes may have false-positive findings and Type I errors. Given the potential inaccuracies of self-reported data, future research on mobile media and communications should go beyond self-reported data to enhance the validity of findings.

Keywords

mobile communication measurement error self-report log data mobile phone data donation

Get full access to this article

View all access options for this article.

References

Bartels

L. M.

(1993). Messages received: The political impact of media exposure. American Political Science Review, 87(2), 267–285. https://doi.org/10.2307/2939040

Boase

Humphreys

(2018). Mobile methods: Explorations, innovations, and reflections. Mobile Media & Communication, 6(2), 153–162. https://doi.org/10.1177/2050157918764215

Boase

Ling

(2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18(4), 508–519. https://doi.org/10.1111/jcc4.12021

Burnell

George

M. J.

Kurup

A. R.

Underwood

M. K.

Ackerman

R. A.

(2021). Associations between self-reports and device-reports of social networking site use: An application of the truth and bias model. Communication Methods and Measures, 15(2), 156–163. https://doi.org/10.1080/19312458.2021.1918654

Chan

(2020). Smartphones and psychological well-being in China: Examining direct and indirect relationships through social support and relationship satisfaction. Telematics and Informatics, 54, 1–11. https://doi.org/10.1016/j.tele.2020.101469

Cicchetti

D. V.

(1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284

Cohen

A. A.

Lemish

(2003). Real time and recall measures of mobile phone use: Some methodological concerns and empirical applications. New Media & Society, 5(2), 167–183. https://doi.org/10.1177/1461444803005002002

Cohen

(2013). Statistical power analysis for the behavioral sciences. Routledge.

Cumming

(2009). Inference by eye: Reading the overlap of independent confidence intervals. Statistics in Medicine, 28(2), 205–220. https://doi.org/10.1002/sim.3471

10.

Deng

Kanthawala

Meng

J. B.

Peng

Kononova

Hao

Zhang

Q. H.

David

(2019). Measuring smartphone usage and task switching with log tracking and self-reports. Mobile Media & Communication, 7(1), 3–23. https://doi.org/10.1177/2050157918761491

11.

Dong

Peng

C.-Y. J.

(2013). Principled missing data methods for researchers. SpringerPlus, 2(1), 222–222. https://doi.org/10.1186/2193-1801-2-222

12.

Elhai

J. D.

Levine

J. C.

Dvorak

R. D.

Hall

B. J.

(2016). Fear of missing out, need for touch, anxiety and depression are related to problematic smartphone use. Computers in Human Behavior, 63, 509–516. https://doi.org/10.1016/j.chb.2016.05.079

13.

Ellis

D. A.

Davidson

B. I.

Shaw

Geyer

(2019). Do smartphone usage scales predict behavior? International Journal of Human-Computer Studies, 130, 86–92. https://doi.org/10.1016/j.ijhcs.2019.05.004

14.

Götz

F. M.

Stieger

Reips

U.-D.

(2017). Users of the main smartphone operating systems (iOS, Android) differ only little in personality. PLoS ONE, 12(5), e0176921. https://doi.org/10.1371/journal.pone.0176921

15.

Hodes

L. N.

Thomas

K. G. F.

(2021). Smartphone screen time: Inaccuracy of self-reports and influence of psychological and contextual factors. Computers in Human Behavior, 115, 106616. https://doi.org/10.1016/j.chb.2020.106616

16.

Huang

Y.-H.

(2001). OPRA: A cross-cultural, multiple-item scale for measuring organization-public relationships. Journal of Public Relations Research, 13(1), 61–90. https://doi.org/10.1207/S1532754XJPRR1301_4

17.

Hughes

M. E.

Waite

L. J.

Hawkley

L. C.

Cacioppo

J. T.

(2004). A short scale for measuring loneliness in large surveys: Results from two population-based studies. Research on Aging, 26(6), 655–672. https://doi.org/10.1177/0164027504268574

18.

Jones-Jang

S. M.

Heo

Y. J.

McKeever

Kim

J. H.

Moscowitz

(2020). Good news! Communication findings may be underestimated: Comparing effect sizes with self-reported and logged smartphone use data. Journal of Computer-Mediated Communication, 25(5), 346–363. https://doi.org/10.1093/jcmc/zmaa009

19.

Kahn

A. S.

Ratan

(2014). Why we distort in self-report: Predictors of self-report errors in video game play. Journal of Computer-Mediated Communication, 19(4), 1010–1023. https://doi.org/10.1111/jcc4.12056

20.

Kobayashi

Boase

(2012). No such effect? The implications of measurement error in self-report measures of mobile communication used. Communication Methods and Measures, 6(2), 126–143. https://doi.org/10.1080/19312458.2012.679243

21.

Kwon

Lee

J. Y.

Won

W. Y.

Park

J. W.

Min

J. A.

Hahn

Choi

J. H.

Kim

D. J.

(2013). Development and validation of a smartphone addiction scale (SAS). PLoS ONE, 8(2), e56936. https://doi.org/10.1371/journal.pone.0056936

22.

Lee

Ahn

Nguyen

T. G.

Choi

S. W.

Kim

D. J.

(2017). Comparing the self-report and measured smartphone usage of college students: A pilot study. Psychiatry Investigation, 14(2), 198–204. https://doi.org/10.4306/pi.2017.14.2.198

23.

Lichtenstein

Slovic

Fischhoff

Layman

Combs

(1978). Judged frequency of lethal events. Journal of Experimental Psychology: Human Learning and Memory, 4(6), 551–578. https://doi.org/10.1037/0278-7393.4.6.551

24.

Lin

Y. H.

Lin

P. H.

Chiang

C. L.

Lee

Y. H.

Yang

C. C. H.

Kuo

T. B. J.

Lin

S. H.

(2017). Incorporation of Mobile application (app) measures into the diagnosis of smartphone addiction. Journal of Clinical Psychiatry, 78(7), 866–872. https://doi.org/10.4088/JCP.15m10310

25.

Lin

Y. H.

Lin

Y. C.

Lee

Y. H.

Lin

P. H.

Lin

S. H.

Chang

L. R.

Tseng

H. W.

Yen

L. Y.

Yang

C. C. H.

Kuo

T. B. J.

(2015). Time distortion associated with smartphone addiction: Identifying smartphone addiction via a mobile application (app). Journal of Psychiatric Research, 65, 139–145. https://doi.org/10.1016/j.jpsychires.2015.04.003

26.

Loid

Taht

Rozgonjuk

(2020). Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. Computers in Human Behavior, 102, 22–30. https://doi.org/10.1016/j.chb.2019.08.007

27.

Ohme

Araujo

de Vreese

C. H.

Piotrowski

J. T.

(2020). Mobile data donations: Assessing self-report accuracy and sample biases with the iOS screen time function. Mobile Media & Communication, 9(2), 293–313. https://doi.org/10.1177/2050157920959106

28.

Parry

D. A.

Davidson

B. I.

Sewall

C. J. R.

Fisher

J. T.

Mieczkowski

Quintana

D. S.

(2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature Human Behaviour, 5(11), 1535–1547. https://doi.org/10.1038/s41562-021-01117-5

29.

Radloff

L. S.

(1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. https://doi.org/10.1177/014662167700100306

30.

Ryding

F. C.

Kuss

D. J.

(2020). Passive objective measures in the assessment of problematic smartphone use: A systematic review. Addictive Behaviors Reports, 11, 100257. https://doi.org/10.1016/j.abrep.2020.100257

31.

Scharkow

(2016). The accuracy of self-reported internet us—A validation study using client log data. Communication Methods and Measures, 10(1), 13–27. https://doi.org/10.1080/19312458.2015.1118446

32.

Scharkow

(2019). The reliability and temporal stability of self-reported media exposure: A meta-analysis. Communication Methods and Measures, 13(3), 198–211. https://doi.org/10.1080/19312458.2019.1594742

33.

Statista (2020). Percentage of adults owning a smartphone in China 2016, by age group. https://www.statista.com/statistics/896512/china-share-of-adults-owning-a-smartphone-by-age-group/

34.

Tai

K. W.

(2020). Specificity, conflict, and focal point: A systematic investigation into social media censorship in China. Journal of Communication, 70(6), 842–867. https://doi.org/10.1093/joc/jqaa032

35.

Tourangeau

Rips

L. J.

Rasinski

(2000). The psychology of survey response. Cambridge University Press, pp. 100–135.

36.

Tversky

Kahneman

(1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683

37.

Vanden Abeele

Beullens

Roe

(2013). Measuring mobile phone use: Gender, age and real usage level in relation to the accuracy and validity of self-reported mobile phone use. Mobile Media & Communication, 1(2), 213–236. https://doi.org/10.1177/2050157913477095

38.

Vanden Abeele

M. M. P.

Hendrickson

A. T.

Pollmann

M. M. H.

Ling

(2019). Phubbing behavior in conversations and its relation to perceived conversation intimacy and distraction: An exploratory observation study. Computers in Human Behavior, 100, 35–47. https://doi.org/10.1016/j.chb.2019.06.004

39.

Vigil-Colet

Morales-Vives

Lorenzo-Seva

(2013). How social desirability and acquiescence affect the age–personality relationship. Psicothema, 25(3), 342–348. https://doi.org/10.7334/psicothema2012.297

40.

Vrijheid

Deltour

Krewski

Sanchez

Cardis

(2006). The effects of recall errors and of selection bias in epidemiologic studies of mobile phone use and cancer risk. Journal of Exposure Science & Environmental Epidemiology, 16(4), 371–384. https://doi.org/10.1038/sj.jes.7500509

41.

Wooldridge

J. M.

(2015). Introductory econometrics: A modern approach. Cengage learning.

42.

Wu-Ouyang

(2022). Are smartphones addictive? Examining the cognitive-behavior model of motivations, leisure boredom, extended self, and fear of missing out on possible smartphone addiction. Telematics and Informatics, 71, 101834. https://doi.org/10.1016/j.tele.2022.101834

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.61 MB