This paper presents the results of a cross-sectional survey on evaluation use completed by 1,140 U.S. American Evaluation Association members. This study had three foci: evaluators' current attitudes, perceptions, and experiences related to evaluation use theory and practice, how these data are similar to those reported in a previous study conducted by Preskill and Caracelli (1997), and to identify characteristics that distinguish high endorsers of use items from others in the sample. Findings suggest a fair level of agreement on several dimensions of use including stakeholder involvement, factors that influence use, and the varied roles of the evaluator. Logistic regression results indicated that external evaluators were less likely to be high item endorsers while those who reported being members of the Evaluation Use Topical Interest Group of the American Evaluation Association were more likely to be high item endorsers.
American Evaluation Association. (2008, April). American Evaluation Association Internal Scan Report to the Membership, by Goodman Research Group. Retrieved September 15, 2008, from www.eval.org.
7.
Amo, C., & Cousins, J.B. (2007). Going through the process: An examination of the operationalization of process use in empirical research on evaluation. New Directions for Evaluation, 116, 5-26.
8.
Bickman, L., & Reich, S.M. (2009). Randomized controlled trials: A gold standard with feet of clay? In S. I. Donaldson, C. A. Christie, & M. M. Mark (Eds.), What counts as credible evidence in applied research and evaluation practice?Thousand Oaks, CA: Sage.
9.
Chelimsky, E. (1998). The role of experience in formulating theories of evaluation practice. American Journal of Evaluation, 19, 35-55.
10.
Chen, H. (1994). Current trends and future directions in program evaluation. American Journal of Evaluation, 15, 229-238.
11.
Christie, C.A. (2003). What guides evaluation? A study of how evaluation practice maps onto evaluation theory. New Directions for Evaluation , 97, 7-35.
12.
Christie, C.A., & Alkin, M.C. (1999). Further reflections on evaluation misutilization . Studies in Educational Evaluation, 25, 1-10.
13.
Cousins, J.B. (2003). Utilization effects of participatory evaluation . In T. Kellaghan, & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 245-266). Netherlands: Kluwer.
14.
Cousins, J.B., Goh, S.C., Clark, S., & Lee, L.E. (2004). Integrating evaluation inquiry into the organizational culture: A review and synthesis of the knowledge base. Canadian Journal of Program Evaluation, 19, 99-141.
15.
Cousins, J.B., Goh, S.C., & Elliot, C. (2007, November). Integrating consequences of evaluation capacity building inquiry. Paper presented at the annual meeting of the American Evaluation Association, Baltimore, MD.
16.
Cousins, J.B., & Leithwood, K.A. (1986). Current empirical research on evaluation utilization. Review of Educational Research , 56, 331-364.
17.
Datta, L.E. (2001). Coming attractions. American Journal of Evaluation, 22, 403-408.
18.
Earl, L., & Katz, S. (2002). Leading schools in a data rich world. In K. Leithwood, P. Hallinger, G. Furman, P. Grann, B. Mulford , & K. Riley (Eds.), The international handbook for leadership. Netherlands : Kluwer.
19.
Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (2003). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Pearson Education, Inc.
20.
Gersten, R., & Hitchcock, J. (2009). What is credible evidence in education? The role of the What Works Clearing-house in informing the process. In S. I. Donaldson, C. A. Christie , & M. M. Mark (Eds.), What counts as credible evidence in applied research and evaluation practice?Thousand Oaks, CA: Sage.
21.
Greene, J.G. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12, 91-116.
22.
Henry, G.T. (2000). Why not use?New Directions for Evaluation, 88, 85-97.
23.
Henry, G.T. (2009). When getting it right matters: The case for high-quality policy and program impact evaluations. In S. I. Donaldson, C. A. Christie, & M. M. Mark (Eds.), What counts as credible evidence in applied research and evaluation practice?Thousand Oaks, CA: Sage.
24.
Henry, G.T., & Rog, D.J. (1998). A realist theory and analysis of utilization. New Directions for Evaluation, 78, 89-102.
25.
House, E.R. (1980). Evaluation with validity. Beverly Hill, CA: Sage.
26.
House, E.R., & Howe, K.R. (1998). The issue of advocacy in evaluations. American Journal of Evaluation, 19, 233-236.
27.
Julnes, G., & Rog, D. (2009). Evaluation methods for producing actionable evidence: Contextual influences on adequacy and appropriateness of method choice. In S. I. Donaldson, C. A. Christie, & M. M. Mark (Eds.), What counts as credible evidence in applied research and evaluation practice?Thousand Oaks, CA: Sage .
28.
Kirkhart, K.E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. New Directions for Evaluation, 88, 5-23.
29.
Leviton, L.C., & Hughes, E.F.X. (1981). Research on the utilization of evaluations: A review and synthesis. Evaluation Review , 5, 525-548.
30.
Osborne Daponte, B. (2008). Evaluation essentials: Methods for conducting sound research. San Francisco: Jossey-Bass.
31.
Owen, J.M., & Rogers, P. (1999). Program evaluation: Forms and approaches. Thousand Oaks, CA: Sage.
Patton, M.Q. (1997a). Utilization-focused evaluation: The new century text (3rd ed.). Thousand Oaks, CA: Sage.
34.
Patton, M.Q. (1997b). Of vacuum cleaners and toolboxes: A response to Fetterman's Response. Evaluation Practice, 18, 267-270.
35.
Patton, M.Q. (2003). Utilization-focused evaluation. In T. Kellaghan, & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 223-244). Netherlands: Kluwer.
36.
Patton, M.Q., Grimes, P.S., Guthrie, K.M., Brennan, N.J., French, B.D., & Blyth, D.A. (1977). In search of impact: An analysis of the utilization of federal health evaluation research. In C. H. Weiss (Ed.), Using social research in public policy making . Lexington, MA: Heath.
37.
Preskill, H. (2008). Evaluation's second act: A spotlight on learning . American Journal of Evaluation, 29, 127-138.
38.
Preskill, H., & Caracelli, V. (1997). Current and developing conceptions of use: Evaluation use TIG survey results. Evaluation Practice, 18, 209-225.
39.
Preskill, H., & Torres, R. (1999). Evaluative inquiry for learning in organizations . Thousand Oaks, CA: Sage.
40.
Preskill, H., & Torres, R. (2000). The learning dimension of evaluation use. New Directions for Evaluation, 88, 25-37.
41.
Preskill, H., Zuckerman, B., & Matthews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24, 423-442.
42.
Reineke, R.A. (1991). Stakeholder involvement in evaluation: Suggestions for practice. American Journal of Evaluation, 12, 39-44.
43.
Rich, R.F. (1977). Uses of social science information by federal bureaucrats: Knowledge for action versus knowledge for understanding. In C. H. Weiss (Ed.), Using social research in public policy making. Lexington, MA : Lexington Books.
44.
Rossman, G.B., & Rallis, S.F. (2000). Critical inquiry and use as action. New Directions for Evaluation, 88, 55-69.
45.
Russ-Eft, D., & Preskill, H. (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York : Perseus.
46.
Scriven, M. (1971). Objectivity and subjectivity in educational research . In G. F. Madaus, M. Scriven, & D. L. Stufflebeam (Eds.), Philosophical redirection of educational research . Boston: Kluwer-Nijhoff.
47.
Shadish, W.R., Cook, T.D., & Leviton, L.C.L. (1991). Foundations of program evaluation. Newbury Park, CA: Sage .
48.
Shulha, L.M. (2000). Evaluative inquiry in university-school professional learning partnerships. In V. J. Caracelli , & H. Preskill (Ed.), The expanding scope of evaluation use. New Directions for Evaluation (Vol. 88, pp. 39-53).
49.
Shulha, L.M., & Cousins, J.B. (1997). Evaluation use: Theory, research, and practice since 1986. Evaluation Practice, 18, 195-208.
50.
Smith, N.C. (1993). Improving evaluation theory through the empirical study of evaluation practice. Evaluation Practice, 14, 237-242.
Taut, S. (2007). Studying self-evaluation capacity building in a large international development organization. American Journal of Evaluation, 28, 45-99.
53.
Taut, S., & Alkin, M.C. (2003). Program staff perceptions of barriers to evaluating implementation. American Journal of Evaluation, 24, 213-226.
54.
Turnbull, B. (1999). The mediating effect of participation efficacy on evaluation use. Evaluation and Program Planning, 22, 131-140.
55.
Weiss, C.H. (1972). Utilization of evaluation: Toward comparative study. In C. H. Weiss (Ed.), Evaluating action programs: Readings in social action and education. Boston: Allyn & Bacon.
56.
Weiss, C.H. (1979). The many meanings of research utilization. Public Administration Review, 39, 426-431.
