This article details the design of a mixed-methods evaluation for a new equity program serving underrepresented and racially minoritized students at a selective-admissions, gifted public secondary school in the U.S. Rooted in culturally responsive and values-engaged approaches, we explain how principles and practices drawn from these approaches informed our decision-making on the evaluation's mixed-methods design and implementation over two years (two cycles of evaluation). These approaches emphasize ensuring inclusivity in participation; calling attention to culture, power, and equity in all evaluation aspects; and fostering dialogue on conceptualizing and operationalizing values like equity. It is not simple to implement these principles and practices in school-based evaluation. In this article, we share a constellation of issues we navigated throughout the evaluation process and our lessons learned.
AbmaT. A.SchrijverJ. (2020). ‘Are we famous or something?’ Participatory health research with children using photovoice. Educational Action Research, 28(3), 405–426. https://doi.org/10.1080/09650792.2019.1627229
2.
AklE. A.KhabsaJ.PetkovicJ.MagwoodO.LytvynL.MotilallA.CampbellP.Todhunter-BrownA.SchünemannH. J.WelchV.TugwellP.ConcannonT. W. (2024). “Interest-holders”: A new term to replace “stakeholders” in the context of health research and policy. Cochrane Evidence Synthesis and Methods, 2(11), e70007. https://doi.org/10.1002/cesm.70007
3.
Aponte-SotoL.MaldonadoS. I.Valentín CortésM. A.NeubauerL. C.HernandezA. (2024). La sazón de liderazgo among Latine evaluators to advance LatCREE. New Directions for Evaluation, 2024(181), 27–38. https://doi.org/10.1002/ev.20594
4.
AventC. M. (2025). The criticality of language: Exploring STEM education evaluators conceptualizations of equity, diversity, and inclusion and the influence on their roles and practice. Evaluation and Program Planning, 108, 102511. https://doi.org/10.1016/j.evalprogplan.2024.102511
5.
AzzamT.SzanyiM. (2011). Designing evaluations: A study examining preferred evaluation designs of educational evaluators. Studies in Educational Evaluation, 37(2), 134–143. https://doi.org/10.1016/j.stueduc.2011.04.006
6.
BarrowL.SartainL.de la TorreM. (2018). Selective enrollment high schools in Chicago: Admission and impacts. University of Chicago Consortium on School Research.
7.
BoyceA. S. (2017). Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation. Evaluation and Program Planning, 64, 33–43. https://doi.org/10.1016/j.evalprogplan.2017.05.018
8.
CaracelliV. J.GreeneJ. C. (1997). Crafting mixed-method evaluation designs. New Directions for Evaluation, 1997(74), 19–32. https://doi.org/10.1002/ev.1069
9.
ChouinardJ. A. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237–253. https://doi.org/10.1177/1098214013478142
10.
ChouinardJ. A. (2014). Understanding relationships in culturally complex evaluation contexts. Evaluation, 20(3), 332–347. https://doi.org/10.1177/1356389014539428
11.
ChristieC. A.AlkinM. (2013). An evaluation theory tree. In AlkinM. (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (2nd ed., pp. 11–57). Sage.
CollinsP. H. (2019). Intersectionality as critical social theory. Duke University Press.
14.
CronbachL. J.AmbronS. R.DornbuschS. M.HessR. D.HornikR. C.PhillipsD. C.WalkerD. F.WeinerS. S. (1980). Toward reform of program evaluation. Jossey-Bass.
15.
FinnC. E.HockettJ. A. (2012). Exam schools: Inside America's most selective public high schools. Princeton University Press.
16.
FordD. Y.GranthamT. C.WhitingG. W. (2008). Culturally and linguistically diverse students in gifted education: Recruitment and retention issues. Exceptional Children, 74(3), 289–306. https://doi.org/10.1177/001440290807400302
17.
GoodnightM. (2025). Ethnography and evaluation possibilities: Fostering transformative, intersectional, and comparative work. In GoodnightM.HopsonR. (Eds.), Theories bridging ethnography and evaluation: Making transformative, intersectional, and comparative connections (Vol. 1, pp. 1–35). Studies in Educational Ethnography. Emerald Insight Limited.
18.
GoodnightM.HanafyM.RegelsonK. (2025). Supporting learning and decision-making in high school bridge programs with formative evaluation. Manuscript submitted for publication.
GreeneJ. C. (2004). Challenges in practicing deliberative democratic evaluation. New Directions for Evaluation, 2000(85), 13–26. https://doi.org/10.1002/ev.1158
21.
GreeneJ. C. (2012). Engaging critical issues in social inquiry by mixing methods. American Behavioral Scientist, 56(6), 755–773. https://doi.org/10.1177/0002764211433794
22.
GreeneJ. C. (2016). Advancing equity: Cultivating an evaluation habit. In DonaldsonS.PicciottoR. (Eds.), Evaluation for an equitable society (pp. 49–66). Evaluation and Society Series. Information Age Publishing.
23.
GreeneJ. C.CaracelliV. J.GrahamW. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. https://doi.org/10.3102/01623737011003255
24.
GreeneJ. C.DeStefanoL.BurgonH.HallJ. (2006). An educative, values-engaged approach to evaluating STEM educational programs. New Directions for Evaluation, 2006(109), 53–71. https://doi.org/10.1002/ev.178
25.
GuajardoA. D.Robles-SchraderG. M.Aponte-SotoL.NeubauerL. C. (2020). Latcrit theory as a framework for social justice evaluation: Considerations for evaluation and evaluators. New Directions for Evaluation, 2020(166), 65–75. https://doi.org/10.1002/ev.20409
26.
HallJ. N. (2013). Pragmatism, evidence, and mixed methods evaluation. New Directions for Evaluation, 2013(138), 15–26. https://doi.org/10.1002/ev.20054
27.
HallJ. N.AhnJ.GreeneJ. C. (2012). Values engagement in evaluation: Ideas, illustrations, and implications. American Journal of Evaluation, 33(2), 195–207. https://doi.org/10.1177/1098214011422592
28.
HallJ. N.FreemanM.ColomerS. E. (2020). Being culturally responsive in a formative evaluation of a professional development school: Successes and missed opportunities of an educative, values-engaged evaluation. American Journal of Evaluation, 41(3), 384–403. https://doi.org/10.1177/1098214019885632
29.
HallM. E. (2023). Epilogue: The view from here. New Directions for Evaluation, 2023(180), 105–112. https://doi.org/10.1002/ev.20576
30.
HanafyM.GoodnightM.O’BrienV. (2025). Implementing photovoice in school-based evaluations: Navigating challenges and leveraging benefits. Journal of MultiDisciplinary Evaluation, 21(51), 84–103. https://doi.org/10.56645/jmde.v21i51.1179
HoodS. (1998). Responsive evaluation Amistad style: Perspectives of one African American evaluator. In DavisR. (Ed.), Proceedings of the stake symposium on educational evaluation (pp. 101–112). University of Illinois.
33.
HoodS. (2001). Nobody knows my name: In praise of African American evaluators who were responsive. New Directions for Evaluation, 2001(92), 31–44. https://doi.org/10.1002/ev.33
34.
HoodS. (2014). How will we know it when we see it? A critical friend perspective of the graduate education diversity internship (GEDI) program and its legacy in evaluation. New Directions for Evaluation, 2014(143), 109–121. https://doi.org/10.1002/ev.20097
35.
HoodS. (2019, September 29- October 1). Closing remarks, Sixth International Conference on Culturally Responsive Evaluation and Assessment (CREA) [Online].
36.
HoodS.HopsonR. K.KirkhartK. E. (2015). Culturally responsive evaluation. In Handbook of practical program evaluation (pp. 281–317). John Wiley & Sons, Inc.
37.
HopsonR. (2009). Reclaiming knowledge at the margins: Culturally responsive evaluation in the current evaluation moment. In RyanK.CousinsJ. B. (Eds.), The Sage international handbook of educational evaluation (pp. 429–446). Sage Publications.
38.
HouseE.HoweK. (1999). Values in evaluation and social research. Sage.
39.
HunterO.LeeburgE.HarnarM. (2020). Using PhotoVoice as an evaluation method. Journal of MultiDisciplinary Evaluation, 16(34), 14–20. https://doi.org/10.56645/jmde.v16i34.603
40.
JohnstoneC. J.HayesA.CohenE.NiadH.Laryea-AdjeiG.LetshaboK.ShikweA.AguA. (2024). A human rights-based evaluation approach for inclusive education. American Journal of Evaluation, 45(3), 435–450. https://doi.org/10.1177/10982140231153810
41.
KirkhartK. E. (1995). 1994 Conference theme: Evaluation and social justice seeking multicultural validity: A postcard from the road. Evaluation Practice, 16(1), 1–12.
42.
KirkhartK. E. (2005). Through a cultural lens: Reflections on validity and theory in evaluation. In HoodS.HopsonR.FriersonH. (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth and understanding in evaluative theory and practice (pp. 21–39). Information Age Publishing.
43.
KirkhartK. E. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of Evaluation, 31(3), 400–413. https://doi.org/10.1177/1098214010373645
44.
LaFranceJ.NicholsR.KirkhartK. E. (2012). Culture writes the script: On the centrality of context in indigenous evaluation. New Directions for Evaluation, 2012(135), 59–74. https://doi.org/10.1002/ev.20027
45.
LouisB.SubotnikR. F.BrelandP. S.LewisM. (2000). Establishing criteria for high ability versus selective admission to gifted programs: Implications for policy and practice. Educational Psychology Review, 12(3), 295–314. https://doi.org/10.1023/A:1009017922302
46.
MathieA.GreeneJ. C. (1997). Interest-holder participation in evaluation: How important is diversity?Evaluation and Program Planning, 20(3), 279–285. https://doi.org/10.1016/S0149-7189(97)00006-2
47.
MertensD. (2018). Mixed methods design in evaluation. SAGE Publications. https://doi.org/10.4135/9781506330631
48.
MertensD.WilsonA. (2019). Program evaluation theory and practice: A comprehensive guide (2nd ed.). Guilford Press.
49.
NeubauerL. C.Aponte-SotoL.GuerreroA. (2023). Latine evaluators and evaluation: Advancing representation and collective agency. New Directions for Evaluation, 2023(180), 73–77. https://doi.org/10.1002/ev.20579
50.
PattonM. Q. (2007). Process use as a usefulism. New Directions for Evaluation, 2007(116), 99–112. https://doi.org/10.1002/ev.246
51.
PattonM. Q. (2013). The roots of utilization-focused evaluation. In AlkinM. (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (pp. 293–303). Sage.
52.
PetersS. J.CarterJ. S. (2022). Predictors of access to gifted education: What makes for a successful school?Exceptional Children, 88(4), 341–358. https://doi.org/10.1177/00144029221081092
53.
ReidA. M.BoyceA. S.ChuquirunaJ. A.Somo-AinaO.MalloyC.StephensonS. (2023). A values-engaged, educative training model for novice evaluators. New Directions for Evaluation, 2023(177), 135–143. https://doi.org/10.1002/ev.20544
54.
RodriguezS. (2017). “My eyes were opened to the lack of diversity in our best schools”: Re-conceptualizing competitive school choice policy as a racial formation. The Urban Review, 49(4), 529–550. https://doi.org/10.1007/s11256-017-0415-z
55.
RodriguezS.BonezziD.KoehlerK. (2019). How long do we have to wait?: Examining school choice, selective enrollment schools, and the reproduction of racial inequality in a southern community. In ConchasG. Q.HingaB. M.AbadM. N.GutierrezK. D. (Eds.), The complex web of inequality in North American schools (pp. 23–44). Routledge.
56.
RyanK. E.ChandlerM.SamuelsM. (2007). What should school-based evaluation look like?Studies in Educational Evaluation, 33(3-4), 197–212. https://doi.org/10.1016/j.stueduc.2007.07.001
57.
StakeR. E. (1974). Program evaluation, particularly responsive evaluation: New trends in evaluation (Report No. 35, 1-20). Institute of Education, University of Gothenburg.
58.
StakeR. E. (2004). Standards-based and responsive evaluation. Sage.
59.
SymonetteH. (2004). Walking pathways toward becoming a culturally competent evaluator: Boundaries, borderlands, and border crossings. New Directions for Evaluation, 2004(102), 95–109. https://doi.org/10.1002/ev.118
60.
TeasdaleR. M.AventC. M.MooreC. L.Serrano AbreuM. B.YanX. (2025). “It has to be the common thread”: Weaving attention to racial equity and justice across lines of inquiry and evaluative criteria. American Journal of Evaluation, 46(2), 241–264. https://doi.org/10.1177/10982140241260069
61.
ThomasV.CampbellP. (2021). Evaluation in today’s world: Respecting diversity, improving quality, and promoting usability. SAGE Publications.
62.
ThomasV. G.ParsonsB. A. (2017). Culturally responsive evaluation meets systems-oriented evaluation. American Journal of Evaluation, 38(1), 7–28. https://doi.org/10.1177/1098214016644069
63.
TracyS. J. (2019). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. John Wiley & Sons.
64.
Waapalaneexkweew (Nicole Bowman, Mohican/Lunaape). (2018). Looking backward but moving forward: Honoring the sacred and asserting the sovereign in indigenous evaluation. American Journal of Evaluation, 39(4), 543–568. https://doi.org/10.1177/1098214018790412
65.
Waapalaneexkweew (Nicole Bowman, Mohican/Lunaape)Dodge-FrancisC. (2018). Culturally responsive indigenous evaluation and tribal governments: Understanding the relationship. New Directions for Evaluation, 2018(159), 17–31. https://doi.org/10.1002/ev.20329
66.
WangC.BurrisM. A. (1997). Photovoice: Concept, methodology, and use for participatory needs assessment. Health Education & Behavior, 24(3), 369–387. https://doi.org/10.1177/109019819702400309
67.
WorrellF. C.DixsonD. D. (2022). Achieving equity in gifted education: Ideas and issues. Gifted Child Quarterly, 66(2), 79–81. https://doi.org/10.1177/00169862211068551
68.
YoonS. Y.GentryM. (2009). Racial and ethnic representation in gifted programs: Current status of and implications for gifted Asian American students. Gifted Child Quarterly, 53(2), 121–136. https://doi.org/10.1177/0016986208330564
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
