Teacher data-driven decision making (DDDM) is a professional practice of great prominence in the current K-12 education system. Moreover, teacher self-efficacy and anxiety around DDDM represent important measurement targets in both research and practice. This study consequently examined the validity, reliability, and invariance of data collected via the Data-Driven Decision-Making Efficacy and Anxiety (3D-MEA) Inventory among U.S. in-service (N = 365) and pre-service (N = 457) teachers. The 3D-MEA is intended to measure four dimensions of self-efficacy related to DDDM, as well as anxiety related to DDDM. Multi-group confirmatory factor analyses established structural measurement invariance, which supports meaningful and effectively interchangeable interpretation of 3D-MEA scores with both in-service and pre-service teachers. Reliability estimates were also high in both populations. Limitations and future directions are discussed.
American Educational Research AssociationAmerican Psychological Association& National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing, American Educational Research Association.
2.
American Recovery and Reinvestment Act of 2009. (2009), P.L. 111-5, 111th Cong. (2009).
BanduraA. (1994). Self-efficacy. In RamachaudranV. S. (Ed.), Encyclopedia of human behavior (Vol. 4, pp. 71–81). Academic Press.
9.
Brosseau-LiardP. E.SavaleiV. (2014). Adjusting incremental fit indices for nonnormality. Multivariate Behavioral Research, 49(5), 460–470. https://doi.org/10.1080/00273171.2014.933697
10.
Brosseau-LiardP. E.SavaleiV.LiL. (2012). An investigation of the sample performance of two nonnormality corrections for RMSEA. Multivariate Behavioral Research, 47(6), 904–930. https://doi.org/10.1080/00273171.2012.715252
11.
BrowneM. W.CudeckR. (1993). Alternative ways of assessing model fit. In BollenK. A.LongJ. S. (Eds.), Testing structural equation models (pp. 136–162). Sage.
12.
ByrneB. M.StewartS. M. (2006). Teacher’s corner: The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling: A Multidisciplinary Journal, 13(2), 287–321. https://dx.doi.org/10.1207/s15328007sem1302_7
13.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
14.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/s15328007sem0902_5
15.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555
16.
CurranP. J.WestS. G.FinchJ. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. https://doi.org/10.1037/1082-989x.1.1.16
17.
DimitrovD. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. https://doi.org/10.1177/0748175610373459
DunnK. E.AirolaD. T.HayawakaT. (2020). Pre-service teachers’ efficacy, anxiety, and concerns about data and the new idea of anchored judgment. Current Issues in Education, 21(1), 1–25.
20.
DunnK. E.AirolaD. T.LoW.-J.GarrisonM. (2013). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81(2), 222–241. https://doi.org/10.1080/00220973.2012.699899
Every Student Succeeds Act of 2015. (2015). P. L. 114-95, S.1177, 114th Cong. (2015).
23.
GreenJ. L.Schmitt-WilsonS.VerslandT. M. (2016). Teachers and data literacy: Ablueprint for professional development to foster data driven decision making. Journal of Continuing Education and Professional Development, 3(1), 14–32. https://doi.org/10.7726/jcepd.2016.1002
24.
GuskeyT. R. (1988). Teacher efficacy, self-concept, and attitudes toward the implementation of instructional innovation. Teaching and Teacher Education, 4(1), 63–69. https://doi.org/10.1016/0742-051x(88)90025-x
25.
HirschfeldG.BrachelR.V (2014). Improving Multiple-Group confirmatory factor analysis in R–A tutorial in measurement invariance with continuous and ordinal indicators. Practical Assessment, Research, and Evaluation, 19(7), 1–12. https://doi.org/10.7275/qazy-2946
26.
HuL.-T.BentlerP. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. https://doi.org/10.1037/1082-989x.3.4.424
27.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
KaffenbergerM. (2021) Modelling the long-run learning impact of the Covid-19 learning shock: actions to (more than) mitigate loss. International Journal of Educational Development, 81, 102326. https://doi.org/10.1016/j.ijedudev.2020.102326
32.
LittleR. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. https://doi.org/10.1080/01621459.1988.10478722
MandinachE. B.GummerE. S. (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60, 366–376. https://doi.org/0.1016/j.tate.2016.07.011
McDonaldR. P. (2013). Test theory: A unified treatment. Psychology Press.
37.
MeadeA. W.BauerD. J. (2007). Power and precision in confirmatory factor analytic tests of measurement invariance. Structural Equation Modeling, 14(4), 611–635. https://doi.org/10.1080/10705510701575461
MeansB.PadillaC.GallagherL. (2010). Use of education data at the local level: From accountability to instructional improvement. U.S. Department of Education.
No Child Left Behind Act of 2001. (2002), P.L. 107-110, 20 U.S.C. § 6319
42.
NunnallyJ. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
43.
ReevesT. D.ChiangJ. L. (2017). Building pre-service teacher capacity to use external assessment data: An intervention study. The Teacher Educator, 52(2), 155–172.
44.
ReevesT. D.OnderY.AbdiB. (2020). Validation of the Data-Driven Decision-Making Efficacy and Anxiety Inventory (3D-MEA) with U.S. pre-service teachers. Mid-Western Educational Researcher, 32(4), 286–303.
45.
ReevesT. D.SummersK. H.GroveE. (2016). Examining the landscape of teacher learning for data use: The case of Illinois. Cogent Education, 3(1).
46.
RosseelY. (2012). Lavaan: An R package for structural equation modeling and more (Version 0.5–12). Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
47.
SatorraA.BentlerP. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/10.1007/bf02296192
48.
SchumackerR. E.LomaxR. G. (1996). A beginner’s guide to structural equation modeling. Lawrence Erlbaum.
49.
TaieS.GoldringR. (2020). Characteristics of public and private elementary and secondary school teachers in the United States: Results From the 2017–18 National Teacher and Principal Survey first look (NCES 2020- 142). National Center for Education Statistics, U.S. Department of Education.
WalkerD. A.ReevesT. D.SmithT. J. (2018). Confirmation of the Data Driven Decision-Making Efficacy and Anxiety Inventory’s (3D-MEA) score factor structure among teachers. Journal of Psychoeducational Assessment, 36(5), 477–491.
