Google Translate (GT) has become a popular machine translation (MT) tool among language learners, received by instructors with excitement over its pedagogical potential and concerns about its possible misuse in the classroom, particularly when this misuse goes undetected. This study investigated the suitability of natural language processing (NLP) software for the automated detection of MT use in second language (L2) writing, examining a dataset composed of written samples generated by GT and direct L2 writing produced by intermediate-level postsecondary learners of Spanish. NLP-powered analyses found significant lexical and sentential-level differences, as well as estimated proficiency-level differences across text types. Automated judgments based on lexical diversity and amount of coordination yielded detection accuracy rates of 73.08% each, whereas proficiency estimates informed correct automated judgments with an overall accuracy rate of 86.54%. An automated reverse-translation protocol using probability estimates was capable of differentiating between direct L2 writing and MT-assisted texts 98% of the time, far surpassing human detection rates (73%) found in a previous study for the same dataset. These findings argue strongly for the potential of NLP-driven textual analysis as a reliable tool to assist instructors in detecting unauthorized uses of MT in L2 writing.
AharoniR.KoppM.GoldbergY. (2015). Automatic detection of machine-translated text and translation quality estimation. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA, 23–25 June, 2014, pp. 289–295.
2.
AhnS.ChungE.S. (2020). Students’ perception on the use of online machine translation in L2 writing. Multimedia-Assisted Language Learning, 23(2), 10–35.
3.
AlilicheK.YakoubiI. (2020). EFL students’ attitudes towards the impact of Google Translate on their writing quantity and quality: The case of students at the Department of English at Mohammed Seddik Ben Yahiya University. Unpublished thesis, University of Mohammed Seddik Ben Yahia.
4.
AndersonD. (1995). Machine translation as a tool in second language learning. CALICO Journal, 13(1), 68–97.
5.
AraseY.ZhouM. (2013). Machine translation detection from monolingual web-text. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria, 4–9 August, 2013, pp. 1597–1607.
6.
Asención-DelaneyY.CollentineJ. (2011). A multidimensional analysis of a written L2 Spanish corpus. Applied linguistics, 32(3), 299–322.
7.
AtaM.DebreliE. (2021). Machine translation in the language classroom: Turkish EFL learners’ and instructors’ perceptions and use. IAFOR Journal of Education: Technology in Education, 9(4), 103–121.
8.
BakerC.L. (2013). Student and instructor perceptions of the use of online translators in English composition. Unpublished thesis, Mississippi State University.
9.
BakerM. (1993). Corpus linguistics and translation studies — implications and applications. In BakerM.FrancisG.Tognini-BonelliE., Text and technology: In honour of John Sinclair (pp. 232–252). John Benjamins.
10.
Bardovi-HarligK. (1992). A second look at T-unit analysis: Reconsidering the sentence. TESOL Quarterly, 26(2), 390–395.
11.
BehrD. (2017). Assessing the use of back translation: The shortcomings of back translation as a quality testing method. International Journal of Social Research Methodology, 20(6), 573–584. https://doi.org/10.1080/13645579.2016.1252188
12.
BirdS.LoperE.KleinE. (2009). Natural language processing with Python. O’Reilly Media Inc. http://www.nltk.org/
13.
BowkerL. (2020a). Chinese speakers’ use of machine translation as an aid for scholarly writing in English: A review of the literature and a report on a pilot workshop on machine translation literacy. Asia Pacific Translation and Intercultural Studies, 7(3), 288–298. https://doi.org/10.1080/23306343.2020.1805843
14.
BowkerL. (2020b). Machine translation literacy instruction for international business students and business English instructors. Journal of Business & Finance Librarianship, 25(1–2), 25–43.
15.
BriggsN. (2018). Neural machine translation tools in the language learning classroom: Students’ use, perceptions, and analyses. JALT CALL Journal, 14(1), 3–24.
16.
BrownA.BennettC.BulmanG.GianniniS.HabibR.Ticio QuesadaM.E. (2022). Machine translation: An enduring chasm between language students and teachers. Center for Applied CALR Linguistics Research Journal, 12, article 3.
17.
BultéB.HousenA. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65.
18.
Campillos LlanosL.RossetS.ZweigenbaumP. (2017). Automatic classification of doctor-patient questions for a virtual patient record query task. In Cohen, K. B., Demner-Fushman, D., Ananiadou, S., & J. Tsujii (Eds.), BioNLP 2017 Workshop Proceedings (pp. 333–341). Association for Computational Linguistics. https://aclanthology.org/W17-2343/
19.
CañeteJ.ChaperonG.FuentesR.HoJ.H.KangH.PérezJ. (2020). Spanish pre-trained BERT model and evaluation data. Practical Machine Learning for Developing Countries (PML4DC) Workshop. Eighth International Conference on Learning Representations (ICLR), pp. 1–10. https://users.dcc.uchile.cl/~jperez/papers/pml4dc2020.pdf
20.
CarterD.InkpenD. (2012). Searching for poor quality machine-translated text: Learning the difference between human writing and machine translations. In KosseimL.InkpenD. (Eds.), Advances in artificial intelligence, Canadian Conference on AI, vol. 7310 of Lecture Notes in Computer Science (pp. 49–60). Springer.
21.
CaseM. (2015). Machine translation and the disruption of foreign language learning activities. eLearning Papers, 45, 4–16.
22.
ChandraS.O.YuyunI. (2018). The use of Google Translate in INEFL essay writing. LLT Journal, 21(2), 228–238. https:///10.24071/llt.2018.210212
23.
ChenC.W-Y. (2020). Using Google Translate in an authentic translation task: The process, refinement efforts, and students’ perceptions. Current Trends in Translation Teaching and Learning E, 7, 213–238.
24.
ChonY.V.ChinD. (2020). Direct writing, translated writing, and machine-translated writing: A text level analysis with Coh-Metrix. English Teaching, 75(1), 25–48.
25.
ChristenP.HandD.J.KirielleN. (2023). A review of the F-measures: Its history, properties, criticisms, and alternatives. ACM Computing Surveys, 56(3), 1–24. https://doi.org/10.1145/3606367
26.
CliffordJ.MerschelL.MunnéJ. (2013). Surveying the landscape: What is the role of machine translation in language learning?@tic Revista d’Innovació Educativa, 10, 108–121.
27.
CobbT. (2002). Web Vocabprofile, https://www.lextutor.ca/vp/, an adaptation of Heatley, Nation & Coxhead’s (2002) Range [Web tool].
28.
ÇorbacıoğluŞ.K.AkselG. (2023). Receiver operating characteristic curve analysis in diagnostic accuracy studies: A guide to interpreting the area under the curve value. Turkish Journal of Emergency Medicine, 23(4), 195–198.
29.
CorreaM. (2011). Academic dishonesty in the second language classroom: Instructors’ perspectives. Modern Journal of Language Teaching Methods, 1, 65–79.
30.
CorreaM. (2014). Leaving the “peer” out of peer-editing: Online translators as a pedagogical tool in the Spanish as a second language classroom. Latin American Journal of Content and Language Integrated Learning, 7, 1–20.
31.
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Council for Cultural Co-operation, Education Committee, Modern Languages Division/Cambridge University Press.
32.
CrossleyS.A. (2013). Advancing research in second language writing through computational tools and machine learning techniques: A research agenda. Language Teaching, 46(2), 256–271.
33.
CrossleyS.A.AllenL.K.KyleK.McNamaraD.S. (2014). Analyzing discourse processing using a simple natural language processing tool (SiNLP). Discourse Processes, 51(5–6), 511–534. https://doi.org/10.1080/0163853X.2014.910723
34.
CrossleyS.A.KyleK.DascaluM. (2019). The tool for the automatic analysis of cohesion 2.0: Integrating semantic similarity and text overlap. Behavioral Research Methods, 51(1), pp. 14–27. https://doi.org/10.3758/s13428-018-1142-4
35.
CrossleyS.A.McNamaraD.S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18, 119–135.
36.
CrossleyS.A.McNamaraD.S. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66–79.
37.
CuetosF.Glez-NostiM.BarbónA.BrysbaertM. (2012). SUBTLEX-ESP: Spanish word frequencies based on film subtitles. Psicológica, 33(2), 133–143.
38.
DavidsonS.YamadaA.MiraP.F.CarandoA.GutierrezC.H.S.SagaeK. (2020). Developing NLP tools with a new corpus of learner Spanish. In Proceedings of the 12th Language Resources and Evaluation Conference (pp. 7238–7243). European Language Resources Association. https://aclanthology.org/2020.lrec-1.892
39.
Delorm-BenitesA.D.KurethS.C.LehrC.SteeleE. (2021). Machine translation literacy: A panorama of practices at Swiss universities and implications for language teaching. In ZoghlamiN.BrudermannC.SarréC.GrosboisM.BradleyL.ThouësnyS. (Eds), CALL and professionalisation: short papers from EUROCALL 2021 (pp. 80–87). Research-publishing.net.
40.
Díez-OrtegaM.KyleK. (2024). Measuring the development of lexical richness of L2 Spanish: A longitudinal learner corpus study. Studies in Second Language Acquisition, 46, 169–199. https://10.1017/S0272263123000384
41.
DikliS.BleyleS. (2014). Automated essay scoring feedback for second language writers: How does it compare to instructor feedback?Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006
42.
DucarC.SchocketD.H. (2018). Machine translation and the L2 classroom: Pedagogical solutions for making peace with Google Translate. Foreign Language Annals, 51(4), 779–795.
43.
El EbyaryK. (2023). The impact of online machine translation (OMT) on vocabulary learning and translation ability. CDELT Occasional Papers in the Development of English Education, 84(1), 281–315. https://10.21608/opde.2023.337479
44.
Enríquez RaídoV.Sánchez TorrónM. (2020). Machine translation, language learning and the ‘knowledge economy,’ In FilimowiczM.TzankovaV. (Eds.), Reimagining communication: Action (pp.155–171). Taylor and Francis/ Routledge.
FarziR. (2016). Taming translation technology for L2 writing: Documenting the use of free online translation tools by ESL Students in a writing course. Unpublished doctoral dissertation. University of Ottawa.
Ferrer i CanchoR. (2004). Euclidean distance between syntactically linked words. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics, 70(5), 056135. https://10.1103/PhysRevE.70.056135
49.
FredholmK. (2015). Online translation use in Spanish as a foreign language essay writing: Effects on fluency, complexity, and accuracy. Revista Nebrija de Lingüística Aplicada, 9, 1–18.
50.
GaspariF.SomersH. (2007). Making a sow’s ear out of a silk purse: (Mis)using online MT services as bilingual dictionaries. Proceedings of Translating and the Computer (Vol. 29), London, UK, Aslib, pp. 1–15.
51.
Godwin-JonesR. (2015). Contributing, creating, curating: Digital literacies for language learners. Language Learning & Technology, 19(3), 8–20.
52.
GorrostietaJ.M.G.LópezS.G.López-LópezA. (2012). Assessing and advising on lexical richness in an intelligent tutoring system. Research in Computing Science, 56, 29–36.
53.
GraesserA.C.McNamaraD.S.LouwerseM.M.CaiZ. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments & Computers, 36(2), 193–202. https://10.3758/BF03195564
54.
GrangerS.BestgenY. (2014). The use of collocations by intermediate vs. advanced non-native writers: A bigram-based study. International Review of Applied Linguistics in Language Teaching, 52(3), 229–252.
55.
GrantL.GintherA. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123–145. https://doi.org/10.1016/s1060-3743(00)00019-9
GunawanD.SembiringC.A.BudimanM.A. (2018). The implementation of cosine similarity to calculate text relevance between two documents. IOP Publishing, Conference Series 2nd International Conference on Computing and Applied Informatics 2017, 28–30 November 2017, Medan, Indonesia.
58.
HanJ.KamberM.PeiJ. (2012). Data mining: Concepts and techniques. A volume in the Morgan Kaufmann Series in data management systems. Science Direct.
HarrisH. (2010). Machine translations revisited: Issues and treatment protocol. Language Teacher, 34(4), 25–29.
61.
HutchinsW. J.SomersH. L. (1992). An introduction to machine translation. Academic Press.
62.
InnesA. (2019). Differentiating between machine translation and student translation: Red flags and salient lexicogrammatical features. Lublin Studies in Modern Languages and Literature, 43(4), 1–13. https://doi.org/10.17951/lsmll.2019.43.4.1-13
63.
JarvisS. (2013). Capturing the diversity in lexical diversity. Language Learning, 63, 87–106.
JinL.DeifellE. (2013). Foreign language learners’ use and perception of online dictionaries: A survey study. MERLOT Journal of Online Learning and Teaching, 9(4), 515–532.
66.
JolleyJ.R.MaimoneL. (2015). Free online machine translation: Use and perceptions by Spanish students and instructors. In MoellerA. J. (Ed.), Learn Languages, Explore Cultures, Transform Lives, (pp. 181-200). Central States Conference on the Teaching of Foreign Languages.
67.
JolleyJ.R.MaimoneL. (2022). Thirty years of machine translation in language teaching and learning: A review of the literature. L2 Journal, 14(1), 26–44. https://doi.org/10.5070/L214151760
68.
JonesK.S. (1994). Natural language processing: A historical review. In ZampolliA.CalzolariN.PalmerM. (Eds.), Current issues in computational linguistics: In honour of Don Walker. Linguistica Computazionale (Vol. 9, pp. 3–16). Pisa.
69.
JuutiM.SunB.MoriT.AsokanN. (2018). Stay on-topic: Generating context-specific fake restaurant reviews. In AtluriV.VaidyaJ. (Eds.), Computer Security – ESORICS 2018: 23rd European Symposium on Research in Computer Security, Proceedings, Part I (pp. 132–151). Barcelona, Spain, September 3–7, 2018, Springer. https://doi.org/10.1007/978-3-319-99073-6_7
70.
KerzE.WiechmannD.QiaoY.TsengE.StröbelM. (2021). Automated classification of written proficiency levels on the CEFR‑scale through complexity contours and RNNs. In BursteinJ.HorbachA.KochmarE.Laarmann‑QuanteR.LeacockC.MadnaniN.PilánI.YannakoudakisH.ZeschT. (Eds.), Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications (pp. 199–209). Association for Computational Linguistics. https://aclanthology.org/2021.bea-1.21/
71.
KnospeY.SullivanK.MalmqvistA.ValfridssonI. (2019). Observing writing and website browsing: Swedish students write L3 German. In LindgrenE.SullivanK. (Eds.), Observing writing: Insights from keystroke logging and handwriting (pp. 258–284). Brill.
72.
KnowlesC.L. (2016). Investigating instructor perceptions of online machine translation and second language acquisition within most commonly taught language courses. Unpublished doctoral dissertation, University of Memphis. https://digitalcommons.memphis.edu/etd/1369/
73.
KnowlesC.L. (2022). Using an ADAPT approach to integrate Google Translate into the second language classroom. L2 Journal, 14(1), 195–236. https://doi.org/10.5070/L214151690
74.
KolS.SchcolnikM.Spector-CohenE. (2018). Google Translate in academic writing courses?The EuroCALL Review (Tel Aviv University, Israel), 26(2), 50–57. https://doi.org/10.4995/eurocall.2018.10140
75.
KornilovV.S.GlushanV.M.YuL.A. (2021). Metric for evaluation of machine translation quality on the basis of edit distances and reverse translation. IEEE 15th International Conference on Application of Information and Communication Technologies (AICT), Baku, Azerbaijan, pp. 1–6.
76.
KhuranaD.KoliA.KhatterK.SinghS. (2023). Natural language processing: State of the art, current trends and challenges. Multimedia Tools and Applications, 82, 3713–3744. https://doi.org/10.1007/s11042-022-13428-4
77.
KurokawaD.GoutteC.IsabelleP. (2009). Automated detection of translated text and its impact on machine translation. Proceedings of Machine Translation Summit XII, Ottawa, Canada, 26–30 August. https://aclanthology.org/2009.mtsummit-papers.9.pdf
78.
LabbéC.LabbéD. (2013). Duplicate and fake publications in the scientific literature: How many SCIgen papers in computer science?Scientometrics, 94(1), 379–396.
79.
Larson-GuenetteJ. (2013). “It’s just reflex now”: German language learners’ use of online resources. Die Unterrichtspraxis / Teaching German, 46(1), 62–74.
80.
LauferB.NationP. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307–322. https://doi.org/10.1093/applin/16.3.307
81.
LeeS-M. (2020). The impact of using machine translation on EFL students’ writing. Computer Assisted Language Learning, 33(3), 157–175. https://doi.org/10.1080/09588221.2018.1553186
82.
LiptonZ.C.ElkanC.NaryanaswamyB. (2014). Thresholding classifiers to maximize F1 score. ArXiv: Statistics and Machine Learning. https://doi.org/10.48550/arXiv.1402.1892
LozanoC. (2009). CEDEL2: Corpus escrito del Español L2. In Bretones CallejasC.M.Fernández SánchezJ.F.Ibáñez IbáñezJ.R.García SánchezM.E.de los RíosE.C.Salaberri RamiroS.Cruz MartínezS.HoneymanN.P.MárquezB.C. (Eds), Applied linguistics now: Understanding language and mind (pp. 197–212). Universidad de Almería. http://hdl.handle.net/10481/22177
86.
LuX.AiH. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29, 16–27.
LuoJ.CherryC.FosterG. (2024). To diverge or not diverge: A morphosyntactic perspective on machine translation vs human translation. ArXive: Computation and Language. https://doi.org/10.48550/arXiv.2401.01419
89.
LutonL. (2003). If the computer did my homework, how come I didn’t get an “A”?The French Review, 76(4), 766–770.
90.
MaimoneL.JolleyJ. (2023). Looks like Google to me: Instructor ability to detect machine translation in L2 Spanish writing. Foreign Language Annals, 56(3), 627–644. https://doi.org/10.1111/flan.12690
91.
McCarthyB. (2004). Does online machine translation spell the end of take-home translation assignments?CALL-EJ Online, 6(1), 26–39.
92.
McGinnisW.D.SiuC.AndreS.HuangH. (2018). Category encoders: A scikit-learn-contrib package of transformers for encoding categorical data. Journal of Open Source Software, 3(21), 501.
MitchellR.DomínguezL.ArcheM.J.MylesF.MarsdenE. (2008). SPLLOC: A new database for Spanish second language acquisition research. EuroSLA Yearbook, 8(1), 287–304.
95.
MundtK.GrovesM. (2016). A double-edged sword: The merits and the policy implications of Google Translate in higher education. European Journal of Higher Education, 6(4), 387–401. https://doi.org/10.1080/21568235.2016.1172248
96.
Murphy OdoD. (2020). Supporting pre-service English teachers’ academic reading and writing with online machine translation. STEM Journal, 21(2), 123–143. https://doi.org/10.16875/stem.2020.21.2.123
97.
MurtisariE.T.WidiningrumR.BranataJ.SusantoR.D. (2019). Google Translate in language learning: Indonesian EFL students’ attitudes. The Journal of Asia TEFL, 16(3), 978–986.
98.
Nguyen-SonH-Q.ThaoT.P.HidanoS.KiyomotoS. (2019). Detecting machine-translated text using back translation. ArXiv: Computation and Language. https://doi.org/10.48550/arXiv.1910.06558
99.
NiñoA. (2009). Machine translation in foreign language learning: language learners’ and tutors’ perceptions of its advantages and disadvantages. ReCALL, 21(2), 241–258.
100.
NiñoA. (2022). Online translators in online language assessments. Computer Assisted Language Learning Electronic Journal (CALL-EJ), 23(3), 115–135.
101.
Niño-AlonsoA. (2008). Free online machine translation as a new form of cheating in foreign language written production. The EuroCALL Review, 14, 18. https://doi.org/10.4995/eurocall.2008.16349
102.
NivreJ.De MarneffeM.C.GinterF.GoldbergY.HajicJ.ManningC.D.McDonaldR.PetrovS.PyysaloS.SilveiraN.TsarfatyR.ZemanD. (2016). Universal dependencies v1: A multilingual treebank collection. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), Portorož, Slovenia, pp. 1659–1666.
103.
OlivaJ.M.BlancoA. (2023). Rasch analysis and validity of the construct understanding of the nature of models in Spanish-speaking students. European Journal of Science and Mathematics Education, 11(2), 344–359. https://doi.org/10.30935/scimath/12651
104.
O’NeillE.M. (2013). Online translator usage in foreign language writing. Dimension, 11, 74–88.
105.
O’NeillE.M. (2019). Online translator, dictionary, and search engine use among L2 students. CALL-EJ: Computer-Assisted Language Learning–Electronic Journal, 20(1), 154–177.
106.
OrtegaL. (2015). Syntactic complexity in L2 writing: Progress and expansion. Journal of Second Language Writing, 29, 82–94.
PecorariD.E. (2002). Original reproductions: An investigation of the source use of postgraduate second language writers. The University of Birmingham.
109.
PedregosaF.VaroquauxG.GramfortA.MichelV.ThirionB.GriselO.BlondelM.PrettenhoferP.WeissR.DubourgV.VanderplasJ.PassosA.CournapeauD.BrucherM.PerrotM.DuchesnayE. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
110.
PetersM.FrankoffM. (2014). New literacy practices and plagiarism: A study of strategies for digital scrapbooking. In GuikemaJ.P.WilliamsL. (Eds.), Digital literacies in foreign and second language education (pp. 245–264). CALICO Monographs Series.
111.
PilánI.VajjalaS.VolodinaE. (2015) A readable read: Automatic assessment of language learning materials based on linguistic complexity. International Journal of Computational Linguistics and Applications (IJLCA), 6(2), 143–159.
112.
PlonskyL.OswaldF.L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. https://doi.org/10.1111/lang.12079
113.
QianM.WuH.Q.YangL.WanA. (2023). Augmented machine translation enabled by GPT4: Performance evaluation on human-machine teaming approaches. In DegenH.NtoaS. (Eds), Artificial Intelligence. Proceedings of the 26th HCI International Conference, HCII 2024, Washington, DC, USA, 29 June–4 July, 2024. Springer, Cham.
114.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
115.
Restrepo-RamosF. (2021). Writing to increase complexity: Spanish L2 learners’ complexity gains in a college composition class. Hispania, 104(4), 671–690.
116.
Restrepo-RamosF.ArrónizS. (2022). SEÑAL: A program for the analysis and assessment of Spanish L2 writing [Online]. https://senal.app/
117.
ScheffelD.LeflyD.HouserJ. (2012). The predictive utility of DIBELS reading assessment for reading comprehension among third-grade English language learners and English-speaking children. Reading Improvement, 49(3), 75–92.
118.
SchnurE.RubioF. (2021). Lexical complexity, writing proficiency, and task effects in Spanish Dual Language Immersion. Language Learning & Technology, 25(1), 53–72.
119.
SkachkovN.A.VorontsovK.V. (2022). Improving the quality of machine translation using the reverse model. Automation and Remote Control, 83, 1897–1907. https://doi.org/10.1134/S00051179220120049
120.
SomersH.GaspariF.NiñoA. (2006). Detecting inappropriate use of free online machine-translation by language students: A special case of plagiarism detection. 11th Annual Conference of the European Association for Machine Translation, Oslo, pp. 41–48.
121.
StapletonP.Ka KinB.L. (2019). Assessing the accuracy and teacher’s impressions of Google Translate: A study of primary L2 writers in Hong Kong. English For Specific Purposes, 56, 18–34. https://doi.org/10.1016/j.esp.2019.07.001
122.
StedingS. (2009). Machine translation in the German classroom: Detection, reaction, prevention. Teaching German, 42(2), 178–189.
123.
StrakaM.StrakováJ. (2017). Tokenizing, pos tagging, lemmatizing and parsing UD 2.0 with UDPipe. Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, Vancouver, Canada, pp. 88–99.
124.
SunY-C.YangF-Y.LiuH-J. (2022). Exploring Google Translate‑friendly strategies for optimizing the quality of Google Translate in academic writing contexts. SN Social Sciences, 2(147), 1–18.
125.
TauléM.MartíM.A.RecasensM. (2008). Ancora: Multilevel annotated corpora for Catalan and Spanish. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08) (pp. 96–101). European Language Resources Association (ELRA). Retrieved from https://aclanthology.org/L08-1222/
126.
TorresL.S.AluísioS.M. (2011). Using machine learning methods to avoid the pitfall of cognates and false friends in Spanish-Portuguese word pairs. Proceedings of the 8th Brazilian Symposium in Information and Human Language Technology, Cuiabá, MT, Brazil, 24–26 October, 2011, pp. 67–76. https://aclanthology.org/W11-4508/
127.
TourmenC.HoffmanD. (2022). A “hands-on” approach to raise awareness of technologies: A pilot class and its lessons. L2 Journal, 14(1), 237–257. https://doi.org/10.5070/L214151734
128.
TsaiS-C. (2019). Using Google Translate in EFL drafts: A preliminary investigation. Computer Assisted Language Learning, 32(5–6), 510–526. https://doi.org/10.1080/09588221.2018.1527361
129.
TsaiS.-C. (2020). Chinese students’ perceptions of using Google Translate as a translingual CALL tool in EFL writing. Computer Assisted Language Learning, 35, 1250–1272. https://doi.org/10.1080/09588221.2020.1799412
130.
TsaiS.-C. (2023). Interactive academic EFL writing assisted by GT for Chinese non-English major students. In QinJ.StapletonP. (Eds.), Technology in second language writing: Advances in composing, translation, writing pedagogy and data-driven learning (pp. 2–27). Taylor & Francis.
UnalI. (2017). Defining an optimal cut-point value in ROC analysis: An alternative approach. Computational and Mathematical Methods in Medicine, 2017(1), 1–14. https://doi.org/10.1155/2017/3762651
133.
VermaV.AggarwalR.K. (2020). A comparative analysis of similarity measures akin to the Jaccard index in collaborative recommendations: Empirical and theoretical perspective. Social Network Analysis and Mining, 10, 43. https://doi.org/10.1007/s13278-020-00660-9
134.
VincentP.LarochelleH.BengioY.ManzagolP.A. (2008). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning (ICML 2008) (pp. 1096–1103). ACM. https://doi.org/10.1145/1390156.1390294
135.
WangB.LiC.PavluV.AslamJ. (2018). A pipeline for optimizing F1-measure in multi-label text classification. Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL.
136.
WarschauerM.WareP. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10, 157–180. https://doi.org/10.1191/1362168806lr190oa
137.
WeigleS.C. (2013). English language learners and automated scoring of essays: Critical considerations. Assessing Writing, 18(1), 85–99. https://doi.org/10.1016/j.asw.2012.10.006
138.
WhiteK.D.HeidrichE. (2013). Our policies, their text: German language students’ strategies with and beliefs about web-based machine translation. Teaching German, 46(2), 230–250.
139.
WilliamsL. (2006). Web-based machine translation as a tool for promoting electronic literacy and language awareness. Foreign Language Annals, 39(4), 565–578.
140.
WolfT.DebutL.SanhV.ChaumondJ.DelangueC.MoiA.CistacP.RaultT.LoufR.FuntowiczM.DavisonJ.ShleiferS.von PlatenP.MaC.JerniteY.PluJ.XuC.Le ScaoT.GuggerS.. . . RushA.M. (2019). Huggingface’s transformers: State-of-the-art natural language processing. ArXiv: Computation and Language. https://doi.org/10.48550/arXiv.1910.03771
141.
WoollsD.CoulthardR. M. (1998). Tools for the trade. Forensic Linguistics: The International Journal of Speech, Language and the Law, 5(1), 33–57.
142.
WuY.SchusterM.ChenZ.LeQ.V.NorouziM.MachereyW.KrikunM.CaoY.GaoQ.MachereyK.KlingnerJ.ShahA.JohnsonM.LiuX.KaiserL.GouwsS.KatoY.KudoT.KazawaH.. . . DeanJ. (2016). Google’s neural machine translation system: Bridging the gap between human and machine translation. ArXiv: Computation and Language. https://doi.org/10.48550/arXiv.1609.08144
143.
XiX. (2008). Investigating the criterion-related validity of the TOEFL® Speaking Scores for ITA screening and setting standards for ITAs. TOEFL iBT Research Report; TOEFLiBT-03. ETS. https://files.eric.ed.gov/fulltext/EJ1111304.pdf
144.
YamadaA.DavidsonS.Fernández-MiraP.CarandoA.SagaeK.Sánchez-GutiérrezC. (2020). COWS-L2H: A corpus of Spanish learner writing. Research in Corpus Linguistics, 8(1), 17–32.
145.
YangY.WangX. (2019). Modeling the intention to use machine translation for student translators: An extension of Technology Acceptance Model. Computer & Education, 133, 116–126.
