Realist evaluation is a form of theory driven evaluation, but is set apart by its explicit philosophical underpinnings. Questions for proponents of theorydriven evaluation. Program theory and the evaluation of a program s theory have gained interest in the evaluation field. It concludes with an indication of some noteworthy benefits and tangible limitations.
Evaluation theory and practice american university online. His book, theorydriven evaluations, has been recognized as one of the landmarks in program evaluation. In this paper, we adhere to the theory driven evaluation terminology of chen 15 for reasons of simplicity, acknowledging the wide range of other terms used in theory of change and realist evaluation. The medical research councils framework for complex interventions has been criticized for not including theory driven approaches to evaluation. It is commonly reported that the function of a program theory is to ascertain the. Although the general conceptual basis appeared far earlier, theorydriven evaluation came to prominence only a few decades ago with the appearance of chens 1990 book theorydriven evaluations.
Gubas 1990 book the paradigm dialog and his 1977 presentation and assessment of six major philosophies in evaluation were provocative. Based upon these domain theories, different types of theorydriven evaluations are constructed. A prospective, theory driven process of intervention. Program theory and theory driven evaluation aea365. The purpose of theorydriven evaluation is to provide information on not only the performance or. If youre looking for a free download links of practical program evaluation.
Practical program evaluation sage publications inc. In 1998 he received the senior biomedical research service award from the cdc. Tsyh chen theorydriven evaluations world of digitals. It is designed to test the implications of a social theory.
Pdf the use of theorydriven evaluation is an emerging practice in the military an aspect generally unknown in the civilian evaluation community find. Theorybased approaches are a logic of enquiry, which complement and can be used in combination with most of the evaluation designs and data collection techniques outlined in program evaluation methods. Theory driven evaluation and the integrated evaluation perspective 2nd ed. A theorydriven evaluation perspective on mixed methods. Program theory and theory driven evaluation, quantitative methods. Correspondence open access how to develop a theory. A call for theory based evaluation the theory driven approach is essential to tracking the many elements of the program or initiative, and assuring that the results identified in the evaluation are firmly. A theory based evaluation of a program is one in which the selection of program features to evaluate is determined by an explicit conceptualization of the program in terms of a theory, a theory which attempts to explain how the program produces the desired effects. The basic idea is to use the programs rationale or theory as the basis of an evaluation to understand the programs development and impact smith, 1994, p. Theory driven evaluation somehow disappeared from the radar during the 1990s, emerging again at the european conference of evaluation in 2002 16. We propose a theory driven approach to the design and evaluation. The conceptual framework of the theorydriven perspective. Pawson and tilley 1997 developed the first realist evaluation approach, although other interpretations have been developed since.
The views expressed in this presentation do not necessarily represent the views of cdc. Furthermore, guidance on how to apply the principles of theory driven evaluation in the domain of health systems research is scarce. Multiple terms have been integrated into discussions of program theory development and evaluation including. What should be stressed in the first instance, however, is the commonality. It features detailed examples of how evaluations actually unfold in practice to develop people, programs, and organizations. Theorybased approaches for practical evaluation steve montague, partner steve. This document is neither an exhaustive training program in theory based evaluation nor a. In appreciation focus on usefulgood theory of change learning is a process not just a product empirical basis for a theory of change a richer vocabulary around theories of change capabilities, opportunities, motivation. Theory as understood in evaluation is fragmented, a problem previously discussed by vaessen and leeuw 2010. Applying program theorydriven approach to design and. Theory driven evaluation for assessing and improving planning, implementation, and effectiveness huey t. I am a doctoral student in quantitative methods in education at the university of minnesota and a quantitative analyst at the minnesota department of education. Lazarsfeld award for contributions to evaluation theory from the american evaluation association.
Applying program theorydriven approach to design and evaluate. Rent practical program evaluation theory driven evaluation and the integrated evaluation perspective 2nd edition 9781412992305 and save up to 80% on textbook rentals and 90% on used textbooks. There is an impressive amount of literature on theory driven evaluation published in the past few decades. First developed during the 1991 gulf war and applied in both the balkans and afghanistan, these techniques. Evaluation research cannot ethically use randomization. Access to society journal content varies across our titles. This approach to evaluation focuses on theoretical rather than methodological issues. I was initially tempted to believe that this would be one of those ordinary evaluation. King an important question if evaluation is a field of practice, then what is an appropriate role for theory in evaluation. Researchers in these fields have largely abandoned the experimental method, in favour of theory driven approaches to evaluation. Theorydriven evaluation can be quite resource and time intensive.
If you have access to a journal via a society or association membership, please browse to your society journal, select an article to view, and follow the instructions in this box. Practical program evaluation theorydriven evaluation and. A theorydriven evaluation perspective on mixed methods research huey t. A wealth of approaches and an untapped potential marielle riche european commissioni have w e got closer t o ident if yi ng the ge nerati ve mechani sms that produce an outcom e f or th e lagging regi ons of t he. Practical program evaluation theorydriven evaluation and the. This approach provides a theory of change, promoting understanding not just of the extent to which in tended outcomes were achieved. Although the framework does include broad guidance on the use of theory, it contains little practical guidance for implementers and there have been calls to develop a more comprehensive approach. Large numbers of theorydriven case studies have been published, encompassing the fields of health, crime and justice, energy and technological innovation, international development and education. With the publication of theory driven evaluations, hueytsyh chen has given us an ambitious volume. There are also various ways the terms evaluation design and approach are used. Pdf a theorydriven evaluation of a wellness initiative. Some of the differences between it and fellow members of the genus programme theory evaluation, theoriesofchange evaluation will be noted in the course of the section. However, there is a growing awareness that program theory is vital for broadening the scope and enhancing the usefulness of program evaluation.
In this report, approach is used to refer to an evaluation strategy that specifies how the fundamental logic of the evaluation is implemented. Theory based utilization focused participatory washington mental health transformation evaluation task group 2. Theory driven evaluation involves the construction of a detailed program theory that is. A theory based evaluation is usually based on an explicit theory of change or logic modelthat explains the theory of a development intervention. The literature devoted to this topic includes four. Commonly accepted strategies for practicing evaluation are outlined, followed by comprehensive accounts of how those strategies have. Theorydriven evaluation and the integrated evaluation perspective pdf, epub, docx and torrent then this site is not for you. Evaluation theory and practice evaluation is a relatively new field that has emerged from a diverse array of applied social sciences. The use of theorydriven evaluation is an emerging practice in the militaryan aspect generally unknown in the civilian evaluation community. Theorybased approaches to evaluation attempt to understand an interventions contribution to observed results through a mechanistic or process interpretation of causation, rather than determining causation through comparison to a counterfactual. This paper presents the development of a study design built on the principles of theory driven evaluation. Pdf the development of theorydriven evaluation in the.
These evaluations included education, crime and safety and transportation. Pinpoint the strengths or weaknesses of a causal pinpoint the strengths or weaknesses of a causal chain 2. Theorydriven evaluation or program theorydriven evaluation is a contextual or holistic assessment of a program based on the conceptual framework of program theory. The theory driven, embedded approach to evaluation is not an additional step to be superimposed upon what. The evaluation is designed to test the theory to see if it holds true. A theory driven evaluation of a wellness initiative. Indeed, it is the aspiration of this book to provide a conceptual framework that integrates the diverse approaches and paradigms of evaluation. The theory driven evaluation approach was used to evaluate an adolescent sexual and reproductive health intervention in mali, burkina faso and cameroon to improve continuity of care through the creation of networks of social and health care providers. Reviews and metaanalyses of theorydriven studies have also been published following different schools and approaches, including realist synthesis studies and theorydriven evaluation science. There is an impressive amount of literature on theorydriven evaluation published in the past few decades. Also the need to deconstruct the influence of the context on.
Theorydriven evaluation and the integrated evaluation perspective 2nd ed. It outlines an evaluation app roach that can be built into your everyday practice. Enhance stakeholdersenhance stakeholders consensus on program theoryconsensus on program theory 4. Assure an evaluation evaluates the right thing assure an evaluation evaluates the right thing 3. The literature devoted to this topic includes four volumes of new directions for evaluation. Theory based approaches for practical evaluation steve montague, partner steve. In theorydriven evaluations, hueytsyh chen introduces a new, comprehensive framework for program evaluation that is designed to bridge the gap between the method and theoryoriented perspectives. While the theoretical base of theorydriven evaluation chen, 1990. Theorybased approaches to evaluation use an explicit theory of change to draw conclusions about whether and how an intervention contributed to observed results. Theorydriven evaluation methods branch of theory tree developed by sociologists huey chen and his advisor, peter rossi chens book published in 1990 normative theory prescriptive, what should be causative theory descriptive, what is research based. Theory and design august 11, 2010 august 17, 2010 hi, my name is christopher moore. Professor chens second edition of practical program evaluation sets a new standard for providing a variety of systematic frameworks for conducting high quality program evaluations. It commences with some core ideas and principles, and then sets out the main research strategies and designs associated with the approach. A prospective, theory driven process of intervention design and evaluation is required to develop complex healthcare interventions which are more likely to be effective, sustainable and scalable.
Nov 30, 2010 this scarcity of theory driven enquiry in health may be due to various reasons. Although it is practiceoriented, there has been a proliferation of research on evaluation theory to prescribe underlying frameworks of. In brief, theory driven evaluation first attempts to map out the programme theory lying behind the intervention and then designs a research evaluation to test out that theory. The use of theory driven evaluation is an emerging practice in the militaryan aspect generally unknown in the civilian evaluation community. Chen university of alabama at birmingham the purpose of this article is to discuss the conceptual framework and strategies used in theorydriven evaluations in relation to mixed methods research and to explore the opportunities and challenges emerging from theorydriven. Theorydriven evaluation or program theorydriven evaluation is a contextual or holistic assessment of. Conceptual framework, application and advancement there is an impressive amount of literature on theorydriven. While there is little agreement either on the terminology or the concepts, there is consistency in the main messages and agreement on the value of theory based approaches. If it does, the task of the evaluator is to produce a plausible case, with evidence, that shows what has. In conducting theory driven evaluations, evaluators need to facilitate stakeholders clarification of their program theories.
Evaluation research is unlike traditional social science research because. It recognizes the preciousness of time, the need for information, and the tension between the two. Chen university of alabama at birmingham the purpose of this article is to discuss the conceptual framework and strategies used in theory driven evaluations in relation to mixed methods research and to explore the opportunities and challenges emerging from theory driven. His book, theory driven evaluations, has been recognized as one of the landmarks in program evaluation. Using theory of change to design and evaluate public. Theorydriven evaluation for assessing and improving planning. Oct 12, 2019 program theory and theory driven evaluation, quantitative methods. A prospective, theory driven process of intervention design and evaluation is required to. Theorydriven evaluation of public health programmes oxford. As a consequence a theorydriven evaluation was applied to this project chen, 2012, using process and outcome evaluation frameworks described below. Since that time, the approach has attracted many supporters as well as detractors. Reviewed by michele tarsilla when i fi rst read the title of this book, i was not quite sure what to expect. Pdf download program theory driven evaluation science.
Conceptual framework, methodology, and application. Understanding what worksand whyin quality improvement. Program stakeholders have influence in how the study is designed. This chapter provides an introduction to the use of theory driven approaches in the evaluation of public health programmes. The development of theorydriven evaluation in the militarytheory on the front line article pdf available in american journal of evaluation 301. Theorybased utilization focused participatory washington mental health transformation evaluation task group 2.
In the orydriven evaluation, the results of the evaluation are formulated as an improved programme theory and as such incorporated into the. For the most part the theorydriven evaluation field has not fully engaged with methods that can help explicate the programme theory more fully, nor has it clearly described how methods can be used to test linkages in the programme theory. This conceptual framework provides a heuristic scheme which. Realist evaluation is a species of theory driven evaluation. A call for theorybased evaluation the theorydriven approach is essential to tracking the many elements of the program or initiative, and assuring that the results identified in the evaluation are firmly. Theorydriven evaluation for assessing and improving.
Program theory driven evaluation science fills the gap between 21st century literature on evaluation and what is happening in practice. Theory based approaches have been discussed in evaluation literature for many years weiss, 1997. The readers will come away with a detailed understanding of both conventional and cuttingedge approaches. Chen has contributed to the development of evaluation theory and methodology, especially in the areas of program theory and theory driven evaluations.