The purpose of this article is to offer both theoretical and practical support to evaluation professionals preparing to facilitate the utilization phase of evaluation with a program or organization team. The Systems Evaluation Protocol for Participatory Data Use (SEPPDU) presented here is rooted in a partnership approach to evaluation and is therefore designed as a way to structure conversations and facilitate thinking around data interpretation and decision making. The SEPPDU is presented in three main parts: (1) summarizing evaluation results, (2) interpreting results, and (3) planning for action. This article describes specific and practical tips for the facilitation of each part based on field experience in a variety of settings.
AcreeJ. G.ToveyT. L.PetrulisR. (2023). Reflections on the intersections between collaboration and use in evaluation practice. American Journal of Evaluation. https://doi.org/10.1177/10982140231213530
2.
AlkinM. C.KingJ. A. (2016). The historical development of evaluation use. American Journal of Evaluation, 37(4), 568–579. https://doi.org/10.1177/1098214016665164
3.
AlkinM. C.KingJ. A. (2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450. https://doi.org/10.1177/1098214017717015
4.
ArchibaldT.BuckleyJ.SharrockG. (2025). Evaluative thinking: Understanding and applying the foundations of evaluation. In BickmanL.RogD. (Eds.), The evaluation handbook: An evaluator’s companion. Guilford.
5.
BloomB. S. (1956). Taxonomy of educational objectives book 1: Cognitive domain. Addison Wesley Publishing Company.
6.
BrookfieldS. (2012). Teaching for critical thinking: Tools and techniques to help students question their assumptions. Jossey-Bass.
7.
BuckleyJ.ArchibaldT.HargravesM.TrochimW. M. (2015). Defining and teaching evaluative thinking: Insights from research on critical thinking. American Journal of Evaluation, 36(3), 375–388. https://doi.org/10.1177/1098214015581706
8.
BuckleyJ.HargravesM.MoormanL. (2021). The relational nature of evaluation capacity building: Lessons from facilitated evaluation partnerships. New Directions for Evaluation, 2021(169), 47–64. https://doi.org/10.1002/ev.20445
ChauveronL. M.SamtaniS.GronerM. G.UrbanJ. B.LinverM. R. (2021). Including diverse stakeholder voices in youth character program evaluation. American Journal of Evaluation, 42(2), 221–236. https://doi.org/10.1177/1098214020917218
11.
CookT. D.PollardW. E. (1977). Guidelines: How to recognize and avoid some common problems of mis-utilization of evaluation research findings. Evaluation, 4, 161–164.
12.
CousinsJ. B.WhitmoreE.ShulhaL. (2013). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34(1), 7–22. https://doi.org/10.1177/1098214012464037
13.
EnsmingerD. C.KalleymeynL. M.RempartT.WadeJ.PolaninM. (2015). Case study of an evaluation coaching model: Exploring the role of the evaluator. Evaluation and Program Planning, 49, 124–136. https://doi.org/10.1016/j.evalprogplan.2015.01.002
14.
FettermanD.Rodríguez-CamposL.WandersmanA.O’SullivanR. G. (2014). Collaborative, participatory, and empowerment evaluation: Building a strong conceptual foundation for stakeholder involvement approaches to evaluation (a response to Cousins, Whitmore, and Shulha, 2013). American Journal of Evaluation, 35(1), 144–148. https://doi.org/10.1177/1098214013509875
15.
FettermanD. M.Rodríguez-CamposL.ZukoskiA. P. (2017). Collaborative, participatory, and empowerment evaluation: Stakeholder involvement approaches. Guilford.
16.
FleischerD. N.ChristieC. A. (2009). Evaluation use: Results from a survey of US American evaluation association members. American Journal of Evaluation, 30(2), 158–175. https://doi.org/10.1177/1098214008331009
GriffithsK. E.CampbellM. A. (2009). Discovering, applying and integrating: The process of learning in coaching. International Journal of Evidence Based Coaching and Mentoring, 7(2), 16–30.
19.
HunkinsF. P. (1995). Teaching thinking through effective questioning. Christopher-Gordon Publishers.
20.
HurteauM.ArchibaldT. (2023). Practical wisdom for an ethical evaluation practice. Information Age Publishing.
21.
HurteauM.HouleS.MongiatS. (2009). How legitimate and justified are judgments in program evaluation?Evaluation, 15(3), 307–319. https://doi.org/10.1177/1356389009105883
22.
HurteauM.RahmanianJ.HouleS.MarchandM. P. (2020). The role of intuition in evaluative judgment and decision. American Journal of Evaluation, 41(3), 326–338. https://doi.org/10.1177/1098214020908211
23.
JohnsonK. W.GreenseidL. O.ToalS. A.KingJ. A.LawrenzF.VolkovB. (2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30, 377–410. https://doi.org/10.1177/1098214009341660
24.
KingJ. A.AlkinM. C. (2019). The centrality of use: Theories of evaluation use and influence and thoughts on the first 50 years of use research. American Journal of Evaluation, 40(3), 431–458. https://doi.org/10.1177/1098214018796328
25.
KupiecT.Celińska-JanowiczD.PattynV. (2023). Understanding evaluation use from an organisational perspective: A review of the literature and a research agenda. Evaluation, 29(3), 338–355. https://doi.org/10.1177/13563890231185164
26.
LabinS. N.DuffyJ. L.MeyersD. C.WandersmanA.LesesneC. A. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33(3), 307–338. https://doi.org/10.1177/1098214011434608
27.
LewisK. M.EwersT.BirdM.WilkinsT. (2019). Engage stakeholders in program evaluation: Throw them a party!. The Journal of Extension, 57(4), 15. https://doi.org/10.34068/joe.57.04.15
28.
LordC. G.RossL.LepperM. R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098–2109. https://doi.org/10.1037/0022-3514.37.11.2098
29.
MarkM. M.AllenJ. B.GoodwinJ. L. (2021). Stakeholder involvement in evaluation: Does it affect observers’ perceptions of an evaluation? And which stakeholder group(s) do people think should to participate?Evaluation Review, 45(3-4), 166–190. https://doi.org/10.1177/0193841X211055937
30.
MarkM. M.HenryG. T. (2023). Multiple paths to evaluation influence and social betterment. In AlkinM.ChristieT. (Eds.), Evaluation roots: Theory influencing practice (pp. 67–77). Guilford.
31.
MertensD. M.WilsonA. T. (2018). Program evaluation theory and practice: A comprehensive guide. Guilford.
32.
NickersonR. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175
33.
NkwakeA. M. (2013). Working with assumptions in international development program evaluation. Springer.
PattonM. Q. (2021). Utilization-focused evaluation: The new century text (5th ed.). Sage.
37.
PattonM. Q. (2023). The ongoing evolution of utilization-focused evaluation. In AlvinM. C.ChristieC. A. (Eds.), Evaluation roots: Theory influencing practice (pp. 183–192). Guilford.
38.
Rodríguez-CamposL.Rincones-GómezR. (2012). Collaborative evaluations: Step-by-step (2nd ed.). Stanford University Press.
39.
SchwandtT. A.GatesE. F. (2021). Evaluating and valuing in social research. Guilford.
ShadishW.CookT.CampbellD. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
42.
ShulhaL. M.WhitmoreE.CousinsJ. B.GilbertN.al HudibH. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation. American Journal of Evaluation, 37(2), 193–215. https://doi.org/10.1177/1098214015615230
43.
SmallC.LewC. (2021). Mindfulness, moral reasoning and responsibility: Towards virtue in ethical decision-making. Journal of Business Ethics, 169, 103–117. https://doi.org/10.1007/s10551-019-04272-y
44.
StockdillS. H.BaizermanM.ComptonD. W. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 2002(93), 7–26. https://doi.org/10.1002/ev.39
45.
Sanchez, C., & Dunning, D. (2021). Jumping to conclusions: Implications for reasoning errors, false belief, knowledge corruption, and impeded learning. Journal of Personality and Social Psychology, 120(3), 789–815. https://doi.org/10.1037/pspp0000375
Taylor-PowellE. (2010, November). Building evaluation capacity of community organizations. Workshop presented at the American Evaluation Association annual conference, San Antonio, TX.
TrochimW. M.UrbanJ. B. (2021). Theoretical foundations and philosophical orientation of relational systems evaluation. New Directions for Evaluation, 2021(160), 19–30. https://doi.org/10.1002/ev.20449
50.
UrbanJ. B.ArchibaldT. A.HargravesM.BulkleyJ.HubbardC.LinverM. R.TrochimW. (2021). Introduction to relational systems evaluation. New Directions for Evaluation, 2021(169), 11–18. https://doi.org/10.1002/ev.20444
51.
UrbanJ. B.HargravesM.BuckleyJ.ArchibaldT.HebbardC.TrochimW. M. (2021). The systems evaluation protocol for evaluation planning. New Directions for Evaluation, 2021(169), 31–45. https://doi.org/10.1002/ev.20443
52.
UrbanJ. B.HargravesM.TrochimW. M. (2014). Evolutionary evaluation: Implications for evaluators, researchers, practitioners, funders and the evidence-based program mandate. Evaluation and Program Planning, 45, 127–139. https://doi.org/10.1016/j.evalprogplan.2014.03.011
53.
VoA. T.ArchibaldT. (2018). New directions for evaluative thinking. New Directions for Evaluation, 2018(158), 139–147. https://doi.org/10.1002/ev.20317
YarbroughD.ShulhaL.HopsonR.CaruthersF. A. (2011). The program evaluation standards (3rd ed.). Sage.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
