A post by Dr Cristina Vasilica @cristinavas
I become acquainted with the logic model back in 2009 during a project which involved evaluating the impact of digital media on engaging young people in looking after their sexual health. Evaluation was something I never experienced before, hence it was certainly clear that I had to pick a model that resonated with my interests and skills. The logic model seemed ‘logic’ to me as it explained the relationships between what ‘we do’ and what ‘we get’ as a result of our actions, likewise in system planning and development (my field of work).
The logic model is as much an evaluation method of efficiency and effectiveness of a project as a way to provide a common roadmap or purpose to all the members involved. The approach is situated within the theory driven evaluation. The theory-based approach is an umbrella term for two types of evaluation: realist evaluation and the theory of change. The former is concerned with identifying mechanisms (how change occurs) that work within a specific context (external constraints or setting) to achieve an outcome (results wanted) (Figure 1).
In this blog, I am going to focus on the ‘beauty’ of the theory of change, a concept that in research is often used synonymously with the logic model. There are various logic models developed including the Weaver’s Triangle (Figure 2) and the Wisconsin Model (Figure 3). The Weaver Triangle is visually limited in showing the relationships among aims – outcomes – outputs, therefore used in smaller projects with a limited number of activities.
The Wisconsin model is the most common logic model used.
According to the model, programmes have Resources, Activities, Outputs and Outcomes, explained below.
- Resources, also named Inputs are essential for activities, including human resources, time, technology etc
- Activities are the actions taken as part of the project
- Outputs are the result of activities with the selected audience
- Outcomes refer to the difference made by the programme
- Impact is the ultimate change planned and achieved within organisations/community.
The benefits of the logic model
- Serves as a focal point for discussion regarding evaluation as it outlines ‘when’, where’, and ‘how’ to find information.
- Connects activities with outcomes helping to avoid integrating actions with no intended effect.
- Focuses communication amongst the team on issues that are important.
- Identifies categories of data sources essential to support project operations.
- Allows planning the bigger picture – simplifying reality into key actions.
- The iterative process permits making changes and refining the model as the project progresses
- Generates evidence-based knowledge.
Personally, I used the Wisconsin model because it enabled me to see the wider picture but also draw relationships. For example, I used it to compile a toolkit for a charity to evaluate the impact of their digital services. It provided a clear guide on how to plan and generate evidence-based data. Furthermore, I used the model as a blueprint for a realist evaluation on the impact of social media on patients’ information provision, networking and communication. It enabled me to find some logic and relationships amongst disconnected Context-Mechanism-Outcomes.
Most common pitfalls
Creating the logic model can be difficult and time consuming. There is a fine line between oversimplifying relationships and not adding enough detail. Furthermore, researchers argue that the logic model – unlike realist evaluation – focuses mainly on ‘what works’ not capturing the pitfalls of projects.
- Establish the aims and objectives of the initiative and which aspects of the project could be evaluated. The outcomes of the project will then reinforce the questions that need to be asked to achieve them.
- Select the stakeholders that are going to be part of the evaluation initiative. Stakeholders can include students, funders, project staff, administrators, collaborating agencies and other parties that may have an interest in the programme effectiveness.
- Decide which part of the project you aim to evaluate: inputs, outputs or outcomes. You can choose to evaluate all aspects of the project or only some of them.
- Develop an evaluation question.
- Select indicators – Indicators provides clear signal whether a project or the participants are making any progress towards the outcomes.
- Choose appropriate methods
- Decide and test the resources used
- Gather data
- Analyse and write report
- Knowlton, L. W., & Philips, C. C. (2009) The Logic Model Guidebook: Better Strategies for Great Results Los Angeles: Sage Publications, Inc.
- W.K. Kellogg Foundation Evaluation Handbook – https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide