Restricted accessResearch articleFirst published online 2025-10
Improving the Quality of Individual-Level Web Tracking: Challenges of Existing Approaches and Introduction of a New Content and Long-Tail Sensitive Academic Solution
This article evaluates the quality of data collection in individual-level desktop web tracking used in the social sciences and shows that the existing approaches face sampling issues, validity issues due to the lack of content-level data and their disregard for the variety of devices and long-tail consumption patterns as well as transparency and privacy issues. To overcome some of these problems, the article introduces a new academic web tracking solution, WebTrack, an open-source tracking tool maintained by a major European research institution, GESIS. The design logic, the interfaces, and the backend requirements for WebTrack are discussed, followed by a detailed examination of the strengths and weaknesses of the tool. Finally, using data from 1,185 participants, the article empirically illustrates how an improvement in data collection through WebTrack leads to innovative shifts in the use of tracking data. As WebTrack allows for collecting the content people are exposed to beyond the classical news platforms, it can greatly improve the detection of politics-related information consumption in tracking data through automated content analysis compared to traditional approaches that rely on the source-level analysis.
AraujoT.WonnebergerA.NeijensP.de VreeseC. (2017). How much time do you spend online? Understanding and improving the accuracy of self-reported measures of internet use. Communication Methods and Measures, 11(3), 173–190. https://doi.org/10.1080/19312458.2017.1317337
2.
BachR. L.KernC.AmayaA.KeuschF.KreuterF.HechtJ.HeinemannJ. (2021). Predicting voting behavior using digital trace data. Social Science Computer Review, 39(5), 862–883. https://doi.org/10.1177/0894439319882896
3.
BarthelM.MitchellA.Asare-MarfoD.KennedyC.WordenK. (2020). Measuring news consumption in a digital era. Pew Research Center.
4.
BeamM. A.KosickiG. M. (2014). Personalized news portals: Filtering systems and increased news exposure. Journalism & Mass Communication Quarterly, 91(1), 59–77. https://doi.org/10.1177/1077699013514411
5.
BevanS. (2019). Gone fishing: The creation of the comparative Agendas project master codebook. In BaumgartnerF.BreuningC.GrossmanE. (Eds.), Comparative policy agendas: Theory, tools, data (pp. 17–34). Oxford University Press.
6.
BiegaA. J.PotashP.DaumeH.DiazF.FinckM. (2020). Operationalizing the legal principle of data minimization for personalization.
7.
BodoB.HelbergerN.IrionK.ZuiderveenB. F.MollerJ.van de VeldeB.BolN.van EsB.de VreeseC. (2017). Tackling the algorithmic control crisis – the technical, legal, and ethical challenges of research into algorithmic agents. Yale Journal of Law and Technology, 19(1), 133–180.
8.
BoschO. J.RevillaM. (2022). When survey science met web tracking: Presenting an error framework for metered data. Journal of the Royal Statistical Society - Series A: Statistics in Society, 185(Suppl 2), 408–436. https://doi.org/10.1111/rssa.12956
9.
ChristnerC.UrmanA.AdamS.MaierM. (2022). Automated tracking approaches for studying online media use: A critical review and recommendations. Communication Methods and Measures, 16(2), 79–95. https://doi.org/10.1080/19312458.2021.1907841
de LeónE.MakhortykhM.AdamS. (2024). Hyperpartisan, alternative, and conspiracy media users: An anti-establishment portrait. Political Communication, 42(1), 1–26. https://doi.org/10.1080/10584609.2024.2325426
12.
Dvir-GvirsmanS.TsfatiY.Menchen-TrevinoE. (2016). The extent and nature of ideological selective exposure online: Combining survey responses with actual web log data from the 2013 Israeli elections. New Media & Society, 18(5), 857–877. https://doi.org/10.1177/1461444814549041
13.
EUR-Lex. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). https://eur-lex.europa.eu/eli/reg/2016/679/oj
14.
FesticN.BüchiM.LatzerM. (2021). How long and what for? Tracking a nationally representative sample to quantify internet use. Journal of Quantitative Descriptions: Digital Media, 1(1), 23–35. https://doi.org/10.51685/jqd.2021.018
15.
FlaxmanS.GoelS.RaoJ. M. (2016). Filter bubbles, echo chambers & online news consumption. Public Opinion Quarterly, 80(1), 298–320. https://doi.org/10.1093/poq/nfw006
16.
FletcherR.KalogeropoulosA.NielsenR. K. (2021). More diverse, more politically varied: How social media, search engines and aggregators shape news repertoires in the United Kingdom. New Media & Society, 25(8), 2118–2139. https://doi.org/10.1177/14614448211027393
17.
GentzkowM.ShapiroJ. M. (2011). Ideological segregation online and offline. Quarterly Journal of Economics, 126(4), 1799–1839. https://doi.org/10.1093/qje/qjr044
18.
Gil-LópezT.ChristnerC.de LeónE.MakhortykhM.UrmanA.MaierM.AdamS. (2023). Do (not!) track me: Relationship between willingness to participate and sample composition in online information behavior tracking research. Social Science Computer Review, 41(6), 2274–2292. https://doi.org/10.1177/08944393231156634
19.
González-BailónS.XenosM. A. (2022). The blind spots of measuring online news exposure: A comparison of self-reported and observational data in nine countries. Information, Communication & Society, 26(10), 2088–2106. https://doi.org/10.1080/1369118X.2022.2072750
20.
GrovesR.LybergL. (2010). Total survey error. Past, present and future. Public Opinion Quarterly, 74(5), 849–879. https://doi.org/10.1093/poq/nfq065
GuessA. M. (2021). (Almost) everything in moderation: New evidence on Americans’ online media diets. American Journal of Political Science, 65(4), 1007–1022. https://doi.org/10.1111/ajps.12589
23.
GuessA. M.BarberáP.MunzertS.YangJ. (2021). The consequences of online partisan media. Proceedings of the National Academy of Sciences of the United States of America, 118(14), Article e2013464118. https://doi.org/10.1073/pnas.2013464118
24.
GuessA. M.NyhanB.O’KeeffeZ.ReiflerJ. (2020). The sources and correlates of exposure to vaccine-related (mis)information online. Vaccine, 38(49), 7799–7805. https://doi.org/10.1016/j.vaccine.2020.10.018
25.
HaimM.NienierzaA. (2019). Computational observation: Challenges and opportunities of automated observation within algorithmically curated media environments using a browser plug-in. Computational Communication Research, 1(1), 79–102. https://doi.org/10.5117/ccr2019.1.004.haim
26.
JürgensP.StarkB.MaginM. (2019). Two half-truths make a whole? On bias in self-reports and tracking data. Social Science Computer Review, 38(5), 600–615. https://doi.org/10.1177/0894439319831643
27.
KeuschF.StruminskayaB.AntounC.CouperM. P.KreuterF. (2019). Willingness to participate in passive mobile data collection. Public Opinion Quarterly, 83(Suppl 1), 210–235. https://doi.org/10.1093/poq/nfz007
KonitzerT.AllenJ.EckmanS.HowlandB.MobiusM.RothschildD.WattsD. J. (2021). Comparing estimates of news consumption from survey and passively collected behavioral data. Public Opinion Quarterly, 85(S1), 347–370. https://doi.org/10.2139/ssrn.3548690
30.
KrieterP. (2019). Can I record your screen? Mobile screen recordings as a long-term data source for user studiesProceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (pp. 1–10). ACM Press.
31.
KrieterP.BreiterA. (2018). Analyzing mobile application usage: Generating log files from mobile screen recordingsProceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1–10). ACM Press.
32.
LiuB. (2008). Web data mining. Exploring hyperlinks, contents, and usage data. Springer.
33.
LoecherbachF.TrillingD. (2020). 3bij3 – developing a framework for researching recommender systems and their effects. Computational Communication Research, 2(1), 53–79. https://doi.org/10.5117/ccr2020.1.003.loec
34.
LoecherbachF.Van AtteveldtW. (2022). Social media data donation and digital tracking.
35.
MakhortykhM.de LeónE.UrmanA.ChristnerC.SydorovaM.AdamS.MaierM.Gil-LopezT. (2022a). Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data.
36.
MakhortykhM.UrmanA.Gil-LopezT.UlloaR. (2022b). To track or not to track: Examining perceptions of online tracking for information behavior research. Internet Research, 32(7), 260–279. https://doi.org/10.1108/intr-01-2021-0074
37.
MasurP. K.DiFranzoD.BazarovaN. N. (2021). Behavioral contagion on social media: Effects of social norms, design interventions, and critical media literacy on self-disclosure. PLoS One, 16(7), Article e0254670. https://doi.org/10.1371/journal.pone.0254670
38.
Menchen-TrevinoE.KarrC. (2012). Researching real-world web use with Roxy: Collecting observational web data with informed consent. Journal of Information Technology & Politics, 9(3), 254–268. https://doi.org/10.1080/19331681.2012.664966
39.
MöllerJ.HameleersM.FerreauF. (2020a). Typen von Desinformation und Misinformation. Die Medienanstalten – ALM GbR.
40.
MöllerJ.van de VeldeR. N.MertenL.PuschmannC. (2020b). Explaining online news engagement based on browsing behavior: Creatures of habit?Social Science Computer Review, 38(5), 616–632. https://doi.org/10.1177/0894439319828012
41.
NewmanN. (2022). Reuters digital news report. University of Oxford.
42.
ParryD. A.DavidsonB. I.SewallC. J. R.FisherJ. T.MieczkowskiH.QuintanaD. S. (2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature Human Behaviour, 5(11), 1535–1547. https://doi.org/10.1038/s41562-021-01117-5
43.
PetersonE.DammE. (2019). A window to the world: Americans’ exposure to political news from foreign media outlets. https://doi.org/10.31235/osf.io/er48b
44.
PetersonE.GoelS.IyengarS. (2019). Echo chambers and partisan polarization: Evidence from the 2016 presidential campaign. Stanford PACS.
45.
ReevesB.RamN.RobinsonT. N.CummingsJ. J.GilesC. L.PanJ.ChiattiA.ChoM. J.RoehrickK.YangX.GagnejaA.BrinbergM.MuiseD.LuY.LuoM.FitzgeraldA.YeykelisL.YeykelisL. (2021). Screenomics: A framework to capture and analyze personal life experiences and the ways that technology shapes them. Human-Computer Interaction, 36(2), 150–201. https://doi.org/10.1080/07370024.2019.1578652
46.
RevillaM.OchoaC.LoeweG. (2017). Using passive data from a meter to complement survey data in order to study online behavior. Social Science Computer Review, 35(4), 521–536. https://doi.org/10.1177/0894439316638457
47.
RichterS.StierS. (2022). Learning about the unknown Spitzenkandidaten: The role of media exposure during the 2019 European Parliament elections. European Union Politics, 23(2), 309–329. https://doi.org/10.1177/14651165211051171
48.
SannaL.RomanoS.CoronaG.AgostiC. (2021). Yttrex: Crowdsourced analysis of YouTube’s recommender system during COVID-19 pandemic. Information Management and Big Data, 1410(1), 107–121.
49.
ScharkowM. (2016). The accuracy of self-reported Internet use – a validation study using client log data. Communication Methods and Measures, 10(1), 13–27. https://doi.org/10.1080/19312458.2015.1118446
50.
ScharkowM.MangoldF.StierS.BreuerJ. (2020). How social network sites and other online intermediaries increase exposure to news. Proceedings of the National Academy of Sciences of the United States of America, 117(6), 2761–2763. https://doi.org/10.1073/pnas.1918279117
51.
SenI.FlöckF.WellerK.WeißB.WagnerC. (2021). A total error framework for digital traces of human behavior on online platforms. Public Opinion Quarterly, 85(S1), 399–422. https://doi.org/10.1093/poq/nfab018
StierS.BreuerJ.SiegersP.ThorsonK. (2020a). Integrating survey data and digital trace data: Key issues in developing an emerging field. Social Science Computer Review, 38(5), 503–516. https://doi.org/10.1177/0894439319843669
54.
StierS.KirkizhN.FroioC.SchroederR. (2020b). Populist attitudes and selective exposure to online news: A cross-country analysis combining web tracking and surveys. The International Journal of Press/Politics, 25(3), 426–446. https://doi.org/10.1177/1940161220907018
55.
StierS.MangoldF.ScharkowM.BreuerJ. (2021). Post-broadcast democracy? News exposure in the age of online intermediaries. American Political Science Review, 116(2), 768–774. https://doi.org/10.1017/S0003055421001222
56.
TorcalM.CartyE.ComellasJ. M.BoschO. J.ThomsonZ. A.SeraniD. (2023). The dynamics of political polarisation: A dataset for Spain, Portugal, Italy, Argentina, and Chile (2019–2022). SSRN. https://doi.org/10.2139/ssrn.4377503
57.
UnangstJ.AmayaA. E.SandersH. L.HowardJ.FerrellA.KaronS.DeverJ. A. (2019). A process for decomposing total survey error in probability and nonprobability surveys: A case study comparing health statistics in us internet panels. Journal of Survey Statistics and Methodology, 8(1), 62–88. https://doi.org/10.1093/jssam/smz040
58.
WojcieszakM.de LeeuwS.Menchen-TrevinoE.LeeS.Huang-IsherwoodK. M.WeeksB. (2021). No polarization from partisan news: Over-time evidence from trace data. The International Journal of Press/Politics, 28(3), 601–626. https://doi.org/10.1177/19401612211047194
59.
ZuboffS. (2019). The age of surveillance capitalism. The fight for a human future at the new frontier of power. Public Affairs.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
