We used an unobtrusive approach, keystroke logging, to examine students’ cognitive states during essay writing. Based on data contained in the logs, we classified writing process data into three states: text production, long pause, and editing. We used semi-Markov processes to model the sequences of writing states and compared the state transition time and probability for demographic subgroups that were matched on writing proficiency. Results suggested that the subgroups employed different processes in essay writing.
AllenL.JacovinaM.DascaluM.RoscoeR.KentK.LikensA.McNamaraD. (2016). Entering the time series space: Uncovering the writing process through keystroke analysis. In BarnesT.ChiM.FengM. (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 22–29). Raleigh, NC: International Educational Data Mining Society.
2.
BaaijenV.GalbraithD.de GlopperK. (2012). Keystroke analysis: Reflections on procedures and measures. Written Communication, 19, 246–277. doi:10.1177/0741088312451108
3.
BerkR.BrownL.ZhaoL. (2010). Statistical inference after model selection. Journal of Quantitative Criminology, 26, 217–236.
4.
BreetveltI.van den BerghH.RijlaarsdamG. (1994). Relations between writing processes and text quality: When and how?Cognition and Instruction, 12, 103–123. doi:10.1207/s1532690xci1202
5.
CoxD. R. (1972). Regression models and life-tables. Journal of Royal Statistic Society B, 34, 187–220.
6.
DeaneP.ZhangM. (2015). Exploring the feasibility of using writing process features to assess text production skills(Research Report No. RR-15-26).Princeton, NJ: Educational Testing Service.
7.
GuoH.DeaneP.van RijnP.ZhangM.BennettR. (2018). Exploring the heavy-tailed features of keystroke logs in writing processes. Journal of Educational Measurement, 55, 194–216.
8.
HabermanS. (2015). Pseudo-equivalent groups and linking. Journal of Educational and Behavioral Statistics, 40, 254–273. doi:10.3102/1076998615574772
9.
HayesJ. (2012). Modeling and remodeling writing. Written Communication, 29, 369–388. doi:10.1177/0741088312451260
10.
HoD.ImaiK.KingG.StuartE. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15, 199–236. doi:10.1093/pan/mpl013
11.
JacksonC. (2011). Multi-state models for panel data: the msm package for R. Journal of Statistical Software, 38, 1–28. doi:10.18637/jss.v038.i08.
12.
KalbfleischJ.PrenticeR. (1980). The statistical analysis of failure time data. New York, NY: Wiley.
13.
KauferD.HayesJ.FlowerL. (1986). Composing written sentences. Research in the Teaching of English, 20, 121–140.
14.
KelloggR. (2001). Competition for working memory among writing processes. The American Journal of Psychology, 114, 175–191. doi:10.2307/1423513
15.
KrolA.Saint-PierreP. (2015). SemiMarkov: An R package for parametric estimation in multi-state semi-Markov models. Journal of Statistical Software, 66, 1–16. doi:10.18637/jss.v066.i06
16.
LeijtenM.van WaesL. (2013). Keystroke logging in writing research: Using input log to analyze and visualize writing processes. Written Communication, 30, 358–392.
17.
LockwoodJ. R.McCaffreyD. F. (2016). Matching and wighting with functions of error-prone covariates for causal inference. Journal of American Statistical Association, 111, 1831–1839. doi:10.1080/01621459.2015.1122601
18.
MatsuhashiA. (1981). Pausing and planning: The tempo of written discourse production. Research in the Teaching of English, 15, 113–134.
19.
McCutchenD. (1996). A capacity theory of writing: Working memory in composition. Educational Psychology Review, 8, 299–325. doi:10.1007/BF01464076
20.
MedhiJ. (1982). Stochastic processes. New York, NY: Wiley. ISBN 978-0-470-27000-4.
21.
MislevyR. J.AlmondR. G.YanD.SteinbergL. S. (1999). Bayes nets in educational assessment: Where do the numbers come from? In LaskeyK. B.PradeH. (Eds.), Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (pp. 437–446). San Francisco, CA: Morgan Kaufmann.
National Center for Education Statistics. (2011). Writing 2011: The National Assessment of Educational Progress at grades 8 and 11(NCES-2012-470). Washington, DC: Institute for Education Sciences, U.S. Department of Education.
24.
R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved fromhttp://www.R-project.org/.
25.
RosenbaumP.RubinD. (1983). The central role of the propensity score in observational studies for causal effecs. Biometrika, 70, 41–55.
26.
SchilperoordJ. (2002). On the cognitive status of pauses in discourse production. In OliveT.LevyM. C. (Eds.), Contemporary tools and techniques for studying writing (pp. 61–87). Dordrecht, the Netherlands: Kluwer Academic. doi:10.1007/978-94-010-0468-8
27.
ZenoS.IvensS. M.KoslinS. H.ZenoB. L. (1995). The educator’s word frequency guide. Brewster, NY: Touchstone Applied Science Associates.
28.
ZhangM.BennettE. R.DeaneP.van RijnP. (2019). Are there gender differences in how students write their essays? An analysis of writing processes. Educational Measurement: Issues and Practice. doi:10.1111/emip.12249.
29.
ZhangM.DeaneP. (2015). Process features in writing: internal structure and incremental value over product features(ETS RR-15-27). Princeton, NJ: Educational Testing Service.
30.
ZhangM.van RijnP.DeaneP.BennettE. R. (2019). Scenario-based assessments in writing: An experimental study. Educational Measurement: Issues and Practice. doi:10.1080/10627197.2018.1557515
31.
ZiliakS.McCloskeyD. (2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. Ann Arbor: University of Michigan Press. doi:10.3998/mpub.186351
