These monthly newsletters aim to keep all member networks updated with news describing how to get involved, as well as information of relevant events, services and new initiatives from the RSC Networks team. If you require further information about any of the items in the newsletter, or have any comments or ideas for content please contact the Networks team.

Committee members and representatives are encouraged to disseminate this information, as appropriate, to colleagues and other members. E-alert request forms can be downloaded from the Useful Forms and Documents page. Up to date lists of members can be obtained by emailing the Networks Team.
 

Public Engagement 5: Evaluate

In Hassun’s fifth and final blog article about public engagement, he explains why no project is complete without an honest and detailed evaluation to measure success. Share your thoughts with Hassun via email at zafarh@rsc.org or on Twitter @HassunElZafar.


Evaluation. That pesky word that rocks up when you think it’s all done and dusted. When you’ve seen done all the planning and delivery. When you’ve pitched the idea, achieved something phenomenal… When you just want to have a break. It pops up: Evaluation.

Let’s start with a simple question: Why do we evaluate?

Here’re the answers I came up with in 30 seconds:
  • Improve and inform our current and future practice
  • Demonstrate the impact of your activity
  • Show value for money
  • Secure funding
  • Meet the terms of funding providers
  • Have recorded evidence of your activity
  • Clearly understanding what success looks like

It’s important that evaluation is planned as an integral part of any public engagement activity, not as an add-on. This could be done by placing it into the core part of your delivery plan. This is an example of how this may look:

e589be3a57d64b845a2127b519a81d84-huge-ev

I often find that our focus on successfully achieving our project outputs (delivery) overshadows our vision for the successful evaluation of our objectives. As mentioned in my previous blog articles, clear objectives (make them SMART!) really do help with not falling into this trap.

Let’s give a working example of how incorporating evaluation into every step of a project plan can improve a project.

 

Play

Scenario (as ever, this is not real, but applicable): We have a diverse group of chemistry researchers working on innovative and boundary pushing research on plastics in one of the UK’s top higher education institutions. The researchers are super keen to do more outreach, they don’t see it as an add-on – but a necessity.

Idea: The researchers have come up with an idea, to produce a video that is as concise as possible in explaining contemporary scientific research into plastics and celebrate the diversity of the team. They want to place the video onto the university webpage, use it at open days, school outreach programmes and summer society exhibitions.

 

Pause

Here’s where evaluation comes really handy. Let’s ask one of the above questions, and answer it using SMART methods:

Q) What does success look like?
  1. 2,000 primary aged students in the local area are engaged with reduce, reuse and recycle programmes ran by schools by the end of the academic year.
Now, let’s add… how we can measure that this has been achieved?
  1. Creating a survey that will be filled out by teachers capturing activity response, interviews with school head teacher on school actions and case study stories from students (via teacher-led focus groups).
 

Stop

Go back and look at the idea I presented to you, it’s not a bad idea – but now I’ve told you what I think success looks like, what I want to actually achieve, and how I plan to evaluate it… Is the idea the best way to achieve what I want to achieve?

Quick answer: No.

 

Rewind

I head back to working board; I need to come up with another idea… but this time, I’m going to inform myself using other people’s evaluations and by talking to my specific target audience (AKA primary schools) to see what works and what does not.

So after reading about excellent projects on websites like the NCCPE and having dialogues with experts and partners, I come up with another idea: To create resources for a project where primary schools are encouraged to collect every piece of plastic they use, while also using a specially designed and co-created tool kit to help them become plastic free.

At the end of the spring term, the schools are tasked with creating artwork with the plastics they have collected, alongside a scheme of work which has been co-created with teacher from those schools (by the way, this is not a full idea! It has flaws, I need to work on it, but hopefully you’ll see the point of it being a better idea, because of the evaluation thought process).

The art project attracts local media attention, which this boosts awareness of the programme – you should capture this, funders love local media attention.

The teachers fill out specifically designed questionnaires with questions on the co-creation process, timelines, resources and what worked well and what didn’t – you should also capture this, other teachers love to hear from other teachers.

The head teacher gives you an interview on how your resource has been applied in their school, the challenges they’ve faced, what they would do differently, and the value the programme has given students – capture this, it’s going to improve your project massively and, as for teachers, head teachers love to hear the thoughts of other head teachers.

With permission from the teachers, you capture group thoughts from the students – you can’t give them a survey, they’ll see it as work (the teachers told you that) so you’ve arranged for them to give their thoughts through a game of “pass the bean bag” – if you have the bean bag you tell us what you think of the plastics now you’ve done the project. They love it because it’s a great game, you asked the teacher to do this before the project too, so you can compare the answers.

 

Stop

I can go on, but I won’t. It goes without saying that the second idea is much more resource intensive – it’s going to take much more time and effort than simply making a video. But its impact… well, I think it’d much more significant.

If I did something like this, I’d target one school in a really underserved community, and do it well, and by doing it “well”, I mean having a solid evaluation of the project. If one school is too much, then try one class – less can mean more impact, more lessons for me and more legacy to my project.

It all goes back to one question: What does success look like?

Answer that question in the most detailed, and honest, way you can. Because that is going the measure to which you evaluate any project.

So the next time that pesky evaluation question comes up, think beyond the words “quantitative” and “qualitative”, dig deeper than just using questionnaires and survey forms: define success, set yourself an ambition, and keep evaluation in mind from the very start.



This is our fifth blog article from Hassun, you can read the others in the series here:
  1. Know Your Audience 
  2. Co-Create, Co-Create, Co-Create
  3. What now?
  4. Outputs ≠ Outcomes

 
Posted by Aurora Walshe on Dec 1, 2020 11:00 AM Europe/London

Share this |

Share to Facebook Share to Twitter Share to Linked More...

Leave a comment?

You must be signed in to leave a comment on MyRSC blogs.

Register free for an account at http://my.rsc.org/registration.