Skip to toolbar

Posts about: Uncategorized

If you don’t measure results, you can’t tell success from failure

11 April 2016

A post by Dr Cristina Vasilica @cristinavas

I become acquainted with the logic model back in 2009 during a project which involved evaluating the impact of digital media on engaging young people in looking after their sexual health. Evaluation was something I never experienced before, hence it was certainly clear that I had to pick a model that resonated with my interests and skills. The logic model seemed ‘logic’ to me as it explained the relationships between what ‘we do’ and what ‘we get’ as a result of our actions, likewise in system planning and development (my field of work).

The logic model is as much an evaluation method of efficiency and effectiveness of a project as a way to provide a common roadmap or purpose to all the members involved.  The approach is situated within the theory driven evaluation. The theory-based approach is an umbrella term for two types of evaluation: realist evaluation and the theory of change. The former is concerned with identifying mechanisms (how change occurs) that work within a specific context (external constraints or setting) to achieve an outcome (results wanted) (Figure 1).

Figure 1: Realist evaluation, Pawson & Tilley, 1997

Figure 1: Realist evaluation, Pawson & Tilley, 1997

 

 

 

In this blog, I am going to focus on the ‘beauty’ of the theory of change, a concept that in research is often used synonymously with the logic model. There are various logic models developed including the Weaver’s Triangle (Figure 2) and the Wisconsin Model (Figure 3).  The Weaver Triangle is visually limited in showing the relationships among aims – outcomes – outputs, therefore used in smaller projects with a limited number of activities.

Figure 2: Weavers Triangle

Figure 2: Weavers Triangle

 

 

 

 

 

 

 

 

 

The Wisconsin model is the most common logic model used.

Figure 3: Example of the Wisconsin Model

Figure 3: Example of the Wisconsin Model

 

 

 

 

 

 

According to the model, programmes have Resources, Activities, Outputs and Outcomes, explained below.

  • Resources, also named Inputs are essential for activities, including human resources, time, technology etc
  • Activities are the actions taken as part of the project
  • Outputs are the result of activities with the selected audience
  • Outcomes refer to the difference made by the programme
  • Impact is the ultimate change planned and achieved within organisations/community.

The benefits of the logic model

  • Serves as a focal point for discussion regarding evaluation as it outlines ‘when’, where’, and ‘how’ to find information.
  • Connects activities with outcomes helping to avoid integrating actions with no intended effect.
  • Focuses communication amongst the team on issues that are important.
  • Identifies categories of data sources essential to support project operations.
  • Allows planning the bigger picture – simplifying reality into key actions.
  • The iterative process permits making changes and refining the model as the project progresses
  • Generates evidence-based knowledge.

Personally, I used the Wisconsin model because it enabled me to see the wider picture but also draw relationships. For example, I used it to compile a toolkit for a charity to evaluate the impact of their digital services. It provided a clear guide on how to plan and generate evidence-based data. Furthermore, I used the model as a blueprint for a realist evaluation on the impact of social media on patients’ information provision, networking and communication. It enabled me to find some logic and relationships amongst disconnected Context-Mechanism-Outcomes.

Most common pitfalls

Creating the logic model can be difficult and time consuming. There is a fine line between oversimplifying relationships and not adding enough detail. Furthermore, researchers argue that the logic model – unlike realist evaluation  – focuses mainly on ‘what works’ not capturing the pitfalls of projects.

Evaluation Steps:

  1. Establish the aims and objectives of the initiative and which aspects of the project could be evaluated. The outcomes of the project will then reinforce the questions that need to be asked to achieve them.
  2. Select the stakeholders that are going to be part of the evaluation initiative. Stakeholders can include students, funders, project staff, administrators, collaborating agencies and other parties that may have an interest in the programme effectiveness.
  3. Decide which part of the project you aim to evaluate: inputs, outputs or outcomes. You can choose to evaluate all aspects of the project or only some of them.
  4. Develop an evaluation question.
  5. Select indicators – Indicators provides clear signal whether a project or the participants are making any progress towards the outcomes.
  6. Choose appropriate methods
  7. Decide and test the resources used
  8. Gather data
  9. Analyse and write report

Evaluation summary

Evaluation summary

Further reading

Examples of evaluation

Welcome to the Academy of Creative Educators (ACE)

14 March 2016

The School of Nursing, Midwifery, Social Work and Social Sciences has established the Academy of Creative Educators (ACE) in response to the recognised value of moving towards more creative educational pedagogies and assessments for education.

ACE provides an innovative community supporting all academics in the School to deliver excellence in teaching through a creative pedagogical approach which

  • Supports best practice in Teaching & Learning
  • Recognises and rewards excellent practice
  • Champions creative pedagogy and flexible learning
  • Researches into effective Teaching & Learning

The activities of ACE are determined by the ACE Steering Group. The group comprises representatives from all Directorates within the School as well as university and student representation.

The group have already begun to consider ways in which investment in embedding creative ways of offering education can improve outcomes and also have a positive impact on resources such as accommodation and staff time.  Members are also exploring how educational experiences can be enhanced with use of technology, increased digital awareness and skills and other creative ways of encouraging learning.

All staff in the School are encouraged to join in with ACE activities according to your interests and professional development plans. You are also welcome to join the ACE Steering Group or feed in your ideas for what the group should provide to help your development as an educator.

For more information, take a look at our events and follow us on Twitter @SalfordACE