The GOAL evaluation had three aims. The first was developmental: to support programme development across the six countries by providing evidence during the life of the pilot on programme processes. The second was summative: to assess, as rigorously as possible, the impacts of GOAL on service users and other programme stakeholders. The third aim focused on knowledge cumulation: to provide evidence on programme processes and outcomes in order to support future policy and programme development in the field of adult education guidance.
Five research questions were asked:
- To what degree did programmes achieve their implementation aims across the five intervention objectives, and what factors at programme and policy level appeared to influence the achievement of implementation aims?
- What service user outcomes were achieved, for what groups, and to what degree?
- What programme-level factors were associated with the achievement of positive service user outcomes?
- What policy-level factors were associated with the achievement of those outcomes?
- To what degree were programme expectations met?
This was a mixed methods evaluation. Data were gathered via a range of quantitative and qualitative methods including: client monitoring data; a client satisfaction survey; a client follow-up survey; and qualitative interviews with clients, programme staff, programme partners and policy actors. The collection of client monitoring and satisfaction data was ongoing, qualitative data were collected in two waves in Spring 2016 and 2017, and the follow-up survey was conducted in Spring 2017.
Evaluation design and methods
The methodological approach for this evaluation was shaped by the complexities of the project design, namely that:
- GOAL was multi-site (two sites in each of five countries, and four sites in the Netherlands) and multi-organisational.
- GOAL had multiple objectives.
- GOAL was predicated on cross-organisational collaboration.
- Each partner country had its own unique context and target groups (and target numbers to achieve).
- Programme resources were finite, and were by necessity and logic primarily focused on the interventions rather than the evaluation.
For these reasons, it was neither feasible nor advisable to conduct an experimental or quasi-experimental evaluation involving comparison groups. Instead the evaluation has positioned itself within the broad ‘Theory of Change’ (Weitzman et al., 2002) approach. the GOAL evaluation also draws on a specific type of Theory of Change evaluation: Realist Evaluation (Pawson and Tilley, 1997).
Read the full GOAL methodology
