10 Facts About Impact evaluation

Impact evaluation assesses the changes that can be attributed to a particular intervention, such as a project, program or policy, both the intended ones, as well as ideally the unintended ones.

The key challenge in impact evaluation is that the counterfactual cannot be directly observed and must be approximated with reference to a comparison group.

Impact evaluation designs are identified by the type of methods used to generate the counterfactual and can be broadly classified into three categories – experimental, quasi-experimental and non-experimental designs – that vary in feasibility, cost, involvement during design or after implementation phase of the intervention, and degree of selection bias.

Random sample surveys, in which the sample for the Impact evaluation is chosen randomly, should not be confused with experimental Impact evaluation designs, which require the random assignment of the treatment.

The method used in non-experimental Impact evaluation is to compare intervention groups before and after implementation of the intervention.

Impact evaluation needs to accommodate the fact that natural maturational and developmental processes can produce considerable change independently of the program.

Similarly, according to the US Environmental Protection Agency impact evaluation is a form of evaluation that assesses the net effect of a program by comparing program outcomes with an estimate of what would have happened in the absence of a program.

Impact evaluation has been defined differently over the past few decades.

Common definitions of 'impact' used in evaluation generally refer to the totality of longer-term consequences associated with an intervention on quality-of-life outcomes.

Technically, an evaluation could be conducted to assess 'impact' as defined here without reference to a counterfactual.