Sage Journals: Discover world-class research

Abstract

For institutions where English is the primary language of instruction, English assessments for admissions such as the Test of English as a Foreign Language (TOEFL) and International English Language Testing System (IELTS) give admissions decision-makers a sense of a student’s skills in academic English. Despite this explicit purpose, these exams have also been used for the practice of predicting academic success. In this study, we meta-analytically synthesized 132 effect sizes from 32 studies containing validity evidence of academic English assessments to determine whether different assessments (a) predicted academic success (as measured by grade point average [GPA]) and (b) did so comparably. Overall, assessments had a weak positive correlation with academic achievement (r = .231, p < .001). Additionally, no significant differences were found in the predictive power of the IELTS and TOEFL exams. No moderators were significant, indicating that these findings held true across school type, school level, and publication type. Although significant, the overall correlation was low; thus, practitioners are cautioned from using standardized English-language proficiency test scores in isolation in lieu of a holistic application review during the admissions process.

Keywords

Academic success IELTS meta-analysis predictive validity TOEFL iBT

Get full access to this article

View all access options for this article.

References

Abunawas

(2014). A meta-analytic investigation of the predictive validity of the Test of English as a Foreign Language (TOEFL) scores on GPA [Unpublished doctoral dissertation]. Texas A&M. https://oaktrust.library.tamu.edu/bitstream/handle/1969.1/154156/ABUNAWAS-DISSERTATION-2014.pdf?sequence=1

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. American Educational Research Association.

Arcuino

C. L. T.

(2013). The relationship between the Test of English as a Foreign Language (TOEFL), the International English Language Testing System (IELTS) scores and academic success of international Master’s students [Unpublished doctoral dissertation]. Colorado State University. https://www.proquest.com/dissertations-theses/relationship-between-test-english-as-foreign/docview/1413309058/se-2

Arrigoni

Clark

(2015). Investigating the appropriateness of IELTS cut-off scores for admissions and placement decisions at an English-medium university in Egypt. (IELTS Research Reports Online Series No. 3). IELTS. https://www.ielts.org/en-us/for-researchers/research-reports/online-series-2015-3

Avdi

(2011). IELTS as a predictor of academic achievement in a Master’s Program. English Australia Journal, 26(2), 42–49.

Baer

Martel

(2020). Fall 2020 international student enrollment snapshot. Institute of International Education. https://www.iie.org/Research-and-Insights/Open-Doors/Fall-International-Enrollments-Snapshot-Reports

Becker

B. J.

(2000). Multivariate meta-analysis. In Brown

S. D.

Tinsley

H. E. A.

(Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 499–525). Academic Press. https://doi.org/10.1016/B978-0-12-691360-6.X5000-9

Bodycott

(2009). Choosing a higher education study abroad destination: What mainland Chinese parents and students rate as important. Journal of Research in International Education, 8(3), 349–373. https://doi.org/10.1177/1475240909345818

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2009). Introduction to meta-analysis. Wiley. https://doi.org/10.1002/9780470743386

10.

Bridgeman

Cho

DiPietro

(2016). Predicting grades from an English language assessment: The importance of peeling the onion. Language Testing, 33(3), 307–318. https://doi.org/10.1177/0265532215583066

11.

Bright

(2020, January 15). IELTS vs. TOEFL: What’s the difference? StudyinCanada.com https://www.studyincanada.com/Discover/Article/1/5116/IELTS-vs-TOEFL:-What’s-the-Difference?

12.

British Council. (2017, September 8). IELTS numbers rise to three million a year. https://www.britishcouncil.org/organisation/press/ielts-numbers-rise-three-million-year

13.

Cho

Bridgeman

(2012). Relationship of TOEFL iBT® scores to academic performance: Some evidence from American universities. Language Testing, 29(3), 421–442. https://doi.org/10.1177/0265532211430368

14.

Cohen

A. C.

(1959). Simplified estimators for the normal distribution when samples are singly censored or truncated. Technometrics, 1(3), 217–237. https://doi.org/10.1080/00401706.1959.10489859

15.

Cohen

(1988). Set correlation and contingency tables. Applied Psychological Measurement, 12(4), 425–434. https://doi.org/10.1177/014662168801200410

16.

Cotton

Conrow

(1998). An investigation of the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania. International English Language Testing System (IELTS) Research Reports, 1, 72–115. https://www.ielts.org/-/media/research-reports/ielts_rr_volume01_report4.ashx

17.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957

18.

Daller

M. H.

Phelan

(2013). Predicting international student study success. Applied Linguistics Review, 4(1), 173–193. https://doi.org/10.1515/applirev-2013-0008

19.

Dooey

Oliver

(2002). An investigation into the predictive validity of the IELTS test as an indicator of future academic success. Prospect, 17, 36–54.

20.

Educational Testing Service. (2018). How TOEFL® test scores are used. https://www.ets.org/toefl/institutions/scores/use/

21.

Educational Testing Service. (2020). TOEFL® program history [TOEFL® Research Insight Series, 6]. https://www.ets.org/s/toefl/pdf/toefl_ibt_insight_s1v6.pdf

22.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple, graphical test. The BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

23.

Fisher

R. A.

(1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507–521. https://doi.org/10.2307/2331838

24.

Fisher

Tipton

(2015). Robumeta: An R-package for robust variance estimation in meta-analysis (Version 2.0) [Computer software].

25.

(2012). The effectiveness of traditional admissions criteria in predicting college and graduate success for American and international students [Unpublished doctoral dissertation]. The University of Arizona. https://www.proquest.com/docview/917742486

26.

Gagen

(2019). The predictive validity of IELTS scores: A meta-analysis [Unpublished doctoral dissertation]. The University of Western Ontario. https://ir.lib.uwo.ca/cgi/viewcontent.cgi?article=8762&context=etd

27.

Gershenfeld

Ward Hood

Zhan

(2016). The role of first-semester GPA in predicting graduation rates of underrepresented students. Journal of College Student Retention: Research, Theory & Practice, 17(4), 469–488. https://doi.org/10.1177/1521025115579251

28.

Gochev

(2013). Criterion-related validity of strong and weak second language performance assessments in a university pathway programme [Unpublished doctoral dissertation]. Lancaster University. https://www.baleap.org/wp-content/uploads/2016/03/Dissertation_Nikolay_Gochev.pdf

29.

Graham

J. G.

(1987). English language proficiency and the prediction of academic success. TESOL Quarterly, 21(3), 505–521. https://doi.org/10.2307/3586500

30.

Green

(2007). IELTS washback in context: Preparation for academic writing in higher education (Vol. 25). Cambridge University Press.

31.

Hedges

L. V.

Tipton

Johnson

M. C.

(2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5

32.

Hegarty

(2014). Where we are now—The presence and importance of international students to universities in the United States. Journal of International Students, 4(3), 223–235. https://files.eric.ed.gov/fulltext/EJ1054975.pdf

33.

Higgins

J. P.

Thompson

S. G.

(2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558.

34.

Hill

Storch

Lynch

(1999). A comparison of IELTS and TOEFL as predictors of academic success. International English Language Testing System (IELTS) Research Reports, 2, 62–73. https://www.ielts.org/-/media/research-reports/ielts_rr_volume02_report3.ashx

35.

Honaker

King

Blackwell

(2011). Amelia II: A program for missing data. Journal of Statistical Software, 45(1), 1–47. https://doi.org/10.18637/jss.v045.i07

36.

IELTS (n.d.). IELTS test score guidance. https://www.ielts.org/-/media/pdfs/ielts-test-score-guidance.ashx

37.

Institute of International Education. (2018). A world on the move: Trends in global student mobility. https://p.widencdn.net/w9bjls/A-World-On-The-Move

38.

Israel

Batalova

(2021, January 14). International students in the United States. Migration Policy Institute. https://www.migrationpolicy.org/article/international-students-united-states-2020

39.

Itaya

L. E.

Chambers

D. W.

King

P. A.

(2008). Analyzing the influence of admissions criteria and cultural norms on success in an international dental studies program. Journal of Dental Education, 72(3), 317–328. https://doi.org/10.1002/j.0022-0337.2008.72.3.tb04498.x

40.

Johnson

R. C.

Tweedie

M. G.

(2017). A comparison of IELTS, TOEFL, and EAP course results as predictors of English language learning success in an undergraduate nursing program. In Coombe

Davidson

Gebril

Boraie

Hidri

(Eds.), Language assessment in the Middle East and North Africa: Theory, practice and future trends (pp. 36–53). TESOL Arabia.

41.

Johnson

R. C.

Tweedie

M. G.

(2021). “IELTS-out/TOEFL-out”: Is the end of general English for academic purposes near? Tertiary student achievement across standardized tests and general EAP. Interchange, 52(1), 101–113. https://doi.org/10.1007/s10780-021-09416-6

42.

Koys

(2010). GMAT versus alternatives: Predictive validity evidence from Central Europe and the Middle East. Journal of Education for Business, 85, 180–185. https://doi.org/10.1080/08832320903258618

43.

Kuncel

N. R.

Credé

Thomas

L. L.

(2007). A meta-analysis of the predictive validity of the Graduate Management Admission Test (GMAT) and undergraduate grade point average (UGPA) for graduate student academic performance. Academy of Management Learning & Education, 6(1), 51–68. https://doi.org/10.5465/amle.2007.24401702

44.

Kuncel

N. R.

Wee

Serafin

Hezlett

S. A.

(2010). The validity of the Graduate Record Examination for master’s and doctoral programs: A meta-analytic investigation. Educational and Psychological Measurement, 70(2), 340–352. https://doi.org/10.1177/0013164409344508

45.

Kwai

C. K.

(2009). Model of international student persistence: Factors influencing retention of international undergraduate students at two public statewide four-year university systems [Unpublished doctoral dissertation]. University of Minnesota, Minneapolis. https://search-proquest-com.ezp2.lib.umn.edu/dissertations/docview/305213878/fulltextPDF/66262DE91244947PQ/1?accountid=14586

46.

Lahib

(2016). Testing tensions: The use of English Language Proficiency tests for the admission of Ontario high school applicants to one Ontario university [Unpublished doctoral dissertation]. The University of Western Ontario. https://ir.lib.uwo.ca/cgi/viewcontent.cgi?article=5646&context=etd

47.

Lee

Y. J.

Greene

(2007). The predictive validity of an ESL placement test: A mixed methods approach. Journal of Mixed Methods Research, 1(4), 366–389. https://doi.org/10.1177/1558689807306148

48.

(2018). A comparison of TOEFL iBT and IELTS reading tests. Open Journal of Social Sciences, 6(8), 283–309. https://doi.org/10.4236/jss.2018.68023

49.

Mazzarol

Soutar

(2002). “Push-Pull” factors influencing international student destination choice. International Journal of Educational Management, 16(2), 82–90. https://doi.org/10.1108/09513540210418403

50.

Moher

Liberati

Tetzlaff

Altman

D. G.

(2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), Article e1000097. https://doi.org/10.1371/journal.pmed.1000097

51.

Müller

Daller

(2019). Predicting international students’ clinical and academic grades using two language tests (IELTS and C-test): A correlational research study. Nurse Education Today, 72, 6–11. https://doi.org/10.1016/j.nedt.2018.10.007

52.

O’Dwyer

Kantarcıoğlu

Thomas

(2018). An Investigation of the Predictive Validity of the TOEFL iBT® Test at an English-medium university in Turkey. ETS Research Report Series, 2018(1), 1–13. https://doi.org/10.1002/ets2.12230

53.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

(2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5, Article 210. https://doi.org/10.1186/s13643-016-0384-4

54.

Pearson

W. S.

(2021). The predictive validity of the Academic IELTS test: A methodological synthesis. International Journal of Applied Linguistics, 172(1), 85–120. https://doi.org/10.1186/s13643-016-0384-4

55.

Person

N. E.

(2002). Assessment of TOEFL scores and ESL classes as criteria for admission to career & technical education and other selected Marshall University graduate programs [Unpublished master’s thesis]. Marshall University. https://files.eric.ed.gov/fulltext/ED473756.pdf

56.

Plonsky

Oswald

F. L.

(2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. https://doi.org/10.1111/lang.12079

57.

Rice

Higgins

J. P.

Lumley

(2018). A re-evaluation of fixed effect (s) meta-analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 181(1), 205–227. https://doi.org/10.1111/j.1467-985X.2008.00552.x

58.

Sá

C. M.

Sabzalieva

(2018). The politics of the great brain race: Public policy and international student recruitment in Australia, Canada, England and the USA. Higher Education, 75(2), 231–253. https://doi.org/10.1007/s10734-017-0133-1

59.

Scott

Bailey

Kienzl

(2006). Relative success? Determinants of college graduation rates in public and private colleges in the US. Research in Higher Education, 47(3), 249–279. https://doi.org/10.1007/s11162-005-9388-y

60.

Shbeeb

(2019). The relationship of English language scores on international students’ academic success [Unpublished doctoral dissertation]. University of Central Florida. https://stars.library.ucf.edu/cgi/viewcontent.cgi?article=7577&context=etd

61.

Westrick

P. A.

Robbins

S. B.

Radunzel

J. M.

Schmidt

F. L.

(2015). College performance and retention: A meta-analysis of the predictive validities of ACT® scores, high school grades, and SES. Educational Assessment, 20(1), 23–45. https://doi.org/10.1080/10627197.2015.997614

62.

Wongtrirat

(2010). English language proficiency and academic achievement of international students: A meta-analysis [Unpublished doctoral dissertation]. Old Dominion University. https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1183&context=efl_etds

63.

Wood

(2022, July 8). The complete guide to the TOEFL test. U.S. News. https://www.usnews.com/education/best-colleges/articles/the-complete-guide-to-the-toefl-test

64.

Woodrow

(2006). Academic success of international postgraduate education students and the role of English proficiency. University of Sydney Papers in TESOL, 1, 51–70. https://www.sydney.edu.au/content/dam/corporate/documents/faculty-of-arts-and-social-sciences/research/research-centres-institutes-groups/uos-papers-in-tesol/volume-1/article03.pdf

65.

York

T. T.

Gibson

Rankin

(2015). Defining and measuring academic success. Practical Assessment, Research & Evaluation, 20, Article 5. https://doi.org/10.7275/hz5x-tx03

66.

Young

J. W.

(1990). Adjusting the cumulative GPA using item response theory. Journal of Educational Measurement, 27(2), 175–186. https://doi.org/10.1111/j.1745-3984.1990.tb00741.x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.56 MB

0.00 MB

A meta-analysis on the predictive validity of English language proficiency assessments for college admissions

Abstract

Keywords

Get full access to this article

References

Supplementary Material