Restricted accessResearch articleFirst published online 2021-10
Dimensionality of speech fluency: Examining the relationships among complexity,accuracy,and fluency (CAF) features of speaking performances on the Aptis test
Speech fluency has been extensively researched as a core construct for second language (L2) speaking assessment. Despite the broad consensus on its multifaceted nature, few researchers have empirically explored the dimensionality of this construct. Operationalizations of fluency vary across research and practice, using both holistic and fine-grained features. To address the dimensionality of speech fluency, in this study we examined an array of fluency features of speaking performances on the Aptis test. We conducted both univariate and multivariate statistical analyses to investigate the relationship between individual fluency features and language proficiency, as well as the relationships among fluency, complexity, and accuracy features. We found differences in the holistic and fine-grained fluency features, suggesting that they might reflect different dimensions of speech fluency and be associated with different components of language proficiency. Based on the findings, we labeled these two types of fluency features as macro and micro fluencies. Whereas macro fluency features tend to entail a holistic representation of fluency, micro fluency features tend to be more closely related to the automatic processing of lexico-grammar, constituting a more direct reflection of the cognitive processes in speech production. The findings support the multidimensionality of speech fluency and the need to include both macro and micro fluency features in the scoring, scale development, and validation of L2 speaking assessment.
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. American Educational Research Association.
2.
Audacity Team. (2018). Audacity(R): Free Audio Editor and Recorder (Version 2.3.0) [Computer application]. https://audacityteam.org/
3.
Bardovi-HarligK.BofmanT. (1989). Attainment of syntactic and morphological accuracy by advanced language learners. Studies in Second Language Acquisition, 11(1), 17–34. https://doi.org/10.1017/S0272263100007816
4.
BaxS. (2013). The cognitive processing of candidates during reading tests: Evidence from eye-tracking. Language Testing, 30(4), 441–465. https://doi.org/10.1177/0265532212473244
BiberD.GrayB.PoonponK. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development?TESOL Quarterly, 45(1), 5–35. https://doi.org/10.5054/tq.2011.244483
7.
BiberD.GrayB.StaplesS. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37(5), 639–668. https://doi.org/10.1093/applin/amu059
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Routledge.
13.
CorleyM.StewartO. W. (2008). Hesitation disfluencies in spontaneous speech: The meaning of um. Language and Linguistics Compass, 2(4), 589–602. https://doi.org/10.1111/j.1749-818X.2008.00068.x
14.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press / Council of Europe.
15.
CrowtherD.TrofimovichP.SaitoK.IsaacsT. (2018). Linguistic dimensions of L2 accentedness and comprehensibility vary across speaking tasks. Studies in Second Language Acquisition, 40(2), 443–457. https://doi.org/10.1017/S027226311700016X
16.
CummingA.KantorR.BabaK.ErdosyU.EouanzouiK.JamesM. (2006). Analysis of discourse features and verification of scoring levels for independent and integrated tasks for the new TOEFL. (TOEFL Monograph No.MS-30 RM 05–13). Educational Testing Service. https://www.ets.org/Media/Research/pdf/RR-05-13.pdf
17.
de JongN. H. (2016). Predicting pauses in L1 and L2 speech: The effects of utterance boundaries and word frequency. International Review of Applied Linguistics in Language Teaching, 54(2), 113–132. https://doi.org/10.1515/iral-2016-9993
18.
de JongN. H. (2018). Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly, 15(3), 237–254. https://doi.org/10.1080/15434303.2018.1477780
19.
de JongN. H.SteinelM. P.FlorijnA.SchoonenR.HulstijnJ. H. (2013). Linguistic skills and speaking fluency in a second language. Applied Psycholinguistics, 34(5), 893–916. https://doi.org/10.1017/S0142716412000069
20.
de JongN. H.WempeT. (2009). Praat script to detect syllable nuclei and measure speech rate automatically. Behavior Research Methods, 41(2), 385–390. https://doi.org/10.3758/BRM.41.2.385
21.
DörnyeiZ.KormosJ. (1998). Problem-solving mechanisms in L2 communication: A psycholinguistic perspective. Studies in Second Language Acquisition, 20(3), 349–385. https://doi.org/10.1017/S0272263198003039
FosterP.TonkynA.WigglesworthG. (2000). Measuring spoken language: A unit for all reasons. Applied linguistics, 21(3), 354–375. https://doi.org/10.1093/applin/21.3.354
24.
Fox TreeJ. E. (1995). The effects of false starts and repetitions on the processing of subsequent words in spontaneous speech. Journal of Memory and Language, 34(6), 709–738. https://doi.org/10.1006/jmla.1995.1032
25.
FreedB. (1995). What makes us think that students who study abroad become fluent? In FreedB. (ed.), Second language acquisition in a study abroad context (pp. 123-–148). John Benjamins. https://doi.org/10.1075/sibil.9.09fre
GebrilA.PlakansL. (2013). Toward a transparent construct of reading-to-write tasks: The interface between discourse features and proficiency. Language Assessment Quarterly, 10(1), 9–27. https://doi.org/10.1080/15434303.2011.642040
28.
GintherA.DimovaS.YangR. (2010). Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing, 27(3), 379–399. https://doi.org/10.1177/0265532210364407
29.
GraesserA. C.McNamaraD. S.LouwerseM. M.CaiZ. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202. https://doi.org/10.3758/BF03195564
HousenA.KuikenF. (2009). Complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 461–473. https://doi.org/10.1093/applin/amp048
32.
IwashitaN.BrownA.McNamaraT.O’HaganS. (2008). Assessed levels of second language speaking proficiency: How distinct?Applied Linguistics, 29(1), 24–49. https://doi.org/10.1093/applin/amm017
33.
IwashitaN.McNamaraT.ElderC. (2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51, 401–436. https://doi.org/10.1111/0023-8333.00160
34.
JangE. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26(1), 31–73. https://doi.org/10.1177/0265532208097336
35.
KahngJ. (2014). Exploring utterance and cognitive fluency of L1 and L2 English speakers: Temporal measures and stimulated recall. Language Learning, 64(4), 809–854. https://doi.org/10.1111/lang.12084
36.
KangO.RubinD.PickeringL. (2010). Suprasegmental measures of accentedness and judgments of language learner proficiency in oral English. The Modern Language Journal, 94(4), 554–566. https://doi.org/10.1111/j.1540-4781.2010.01091.x
KormosJ.DénesM. (2004). Exploring measures and perceptions of fluency in the speech of second language learners. System, 32(2), 145–164. https://doi.org/10.1016/j.system.2004.01.001
LaFlairG. T.StaplesS. (2017). Using corpus linguistics to examine the extrapolation inference in the validity argument for a high-stakes speaking assessment. Language Testing, 34(4), 451–475. https://doi.org/10.1177/0265532217713951
41.
Larsen-FreemanD. (2009). Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30, 579–589. https://doi.org/10.1093/applin/amp043
LeveltW. I. M. (1989). Speaking: From intention to articulation. MIT Press.
45.
McCarthyP. M.JarvisS. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392. https://doi.org/10.3758/BRM.42.2.381
46.
McNamaraD. S.GraesserA. C.McCarthyP. M.CaiZ. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press. https://doi.org/10.1017/CBO9780511894664
47.
NeumannH. (2014). Teacher assessment of grammatical ability in second language academic writing: A case study. Journal of Second Language Writing, 24, 83–107. https://doi.org/10.1016/j.jslw.2014.04.002
48.
NorrisJ. M.OrtegaL. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied linguistics, 30(4), 555–578. https://doi.org/10.1093/applin/amp044
ParkS. (2016). Measuring fluency: Temporal variables and pausing patterns in L2 English speech [Unpublished doctoral dissertation]. Purdue University.
51.
PlakansL.GebrilA.BilkiZ. (2019). Shaping a score: Complexity, accuracy, and fluency in integrated writing performances. Language Testing, 36(2), 161–179. https://doi.org/10.1177/0265532216669537
52.
PlakansL.LiaoJ.WangF. (2019). “I should summarize this whole paragraph”: Shared processes of reading and writing in iterative integrated tasks. Assessing Writing, 40, 14–26. https://doi.org/10.1016/j.asw.2019.03.003
53.
RévészA.EkiertM.TorgersenE. N. (2016). The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37(6), 828–848. https://doi.org/10.1093/applin/amu069
54.
RiazantsevaA. (2001). Second language proficiency and pausing: A study of Russian speakers of English. Studies in Second Language Acquisition, 23(4), 497–526. https://doi.org/10.1017/S027226310100403X
55.
RiggenbachH. (1991). Toward an understanding of fluency: A microanalysis of nonnative speaker conversations. Discourse Processes, 14(4), 423–441. https://doi.org/10.1080/01638539109544795
56.
RobinsonP. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22(1), 27–57. https://doi.org/10.1093/applin/22.1.27
57.
RStudio Team. (2016). RStudio: Integrated development for R. RStudio, Inc. http://www.rstudio.com
58.
SchmidtR. (1992). Psychological mechanisms underlying second language fluency. Studies in Second Language Acquisition, 14(4), 357–385. https://doi.org/10.1017/S0272263100011189
59.
SegalowitzN. (2007). Access fluidity, attention control, and the acquisition of fluency in a second language. TESOL Quarterly, 41, 181–186. https://www.jstor.org/stable/40264338
SegalowitzN. (2016). Second language fluency and its underlying cognitive and social determinants. International Review of Applied Linguistics in Language Teaching, 54(2), 79–95. https://doi.org/10.1515/iral-2016-9991
62.
SegalowitzN.HulstijnJ. (2005). Automaticity in bilingualism and second language learning. In KrollJ. F.de GrootA. M. B. (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 371–388). Oxford University Press.
63.
SkehanP. (1998). A cognitive approach to language learning. Oxford University Press.
64.
SkehanP. (2001). Tasks and language performance assessment. In BygateM.SkehanP.SwainM. (Eds.), Research pedagogic tasks: Second language learning, teaching, and testing (pp. 167–185). Longman.
SkehanP. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30, 510–532. https://doi.org/10.1093/applin/amp047
67.
SkehanP.FosterP. (1997). Task type and task processing conditions as influences on foreign language performance. Language Teaching Research, 1, 185–211. https://doi.org/10.1177/136216889700100302
68.
SkehanP.FosterP. (2008). Complexity, accuracy, fluency and lexis in task-based performance: A metaanalysis of the Ealing research. In Van DaeleS.HousenA.KuikenF.PierrardM.VedderI. (Eds.), Complexity, accuracy, and fluency in second language use, learning, and teaching (pp. 207–226). Contactforum.
69.
SloetjesH.WittenburgP. (2008). Annotation by category – ELAN and ISO DCR. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08) (pp. 816–820). European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2008/pdf/208_paper.pdf
70.
TavakoliP. (2011). Pausing patterns: Differences between L2 learners and native speakers. ELT Journal, 65(1), 71–79. https://doi.org/10.1093/elt/ccq020
TowellR.HawkinsR.BazerguiN. (1996). The development of fluency in advanced learners of French. Applied Linguistics, 17(1), 84–119. https://doi.org/10.1093/applin/17.1.84
74.
WieseR. (1984). Language production in foreign and native languages: same or different? In DechertH. W.MöhleD.RaupachM. (Eds.), Second language productions (pp. 11–25). Gunter Narr Verlag.
75.
YanX.StaplesS. (2020). Fitting MD analysis in an argument-based validity framework for writing assessment: Explanation and generalization inferences for the ECPE. Language Testing, 37(2), 189–214. https://doi.org/10.1177/0265532219876226
76.
YounS. J.BiN. Z. (2019). Investigating test-takers’ strategy use in task-based L2 pragmatic speaking assessment. Intercultural Pragmatics, 16(2), 185–218. https://doi.org/10.1515/ip-2019-0009
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
