Back when I worked on the staff of this esteemed magazine, I saw nearly a decade’s worth of press releases from youth organisations come across my desk. A few of them celebrated large numbers of teenagers coming to faith at festivals and events, and there were occasionally new-in-post CEOs telling me how impressively young they were. The vast majority however, concerned the launching of new initiatives. And boy, did they talk those new initiatives up.
Finally: a new web-based evangelistic course that was going to help kids understand the Christian faith! At last: a summer mission experience for people who didn’t like all the other ones! Each new thing was pitched as the answer we’d all been searching for, which is presumably also how it was sold to a funder. But as the test of time has revealed, none of them really were: none really matched up to that extraordinary pre-launch hype.
Here’s a not-unconnected fact: in all those years of reading press releases, I can only think of one example of an organisation writing to tell us what the impact of the new ‘thing’ had been. Other than that, the innovators of youth ministry only looked resolutely forward, never stopping to look back and evaluate the impact of their much-heralded resources and events. The main reason? The hype had proved a bit unfounded: expensive websites gathered digital cobwebs, warehouses clogged up with piles of unsold books (my ‘ground-breaking’ book The Beautiful Disciplines is still available, priced £12.99!). Who wanted to tell those stories?
On a local church level, a similar thing can happen. We get excited about a new idea, and pour lots of resources and energy into it, so it’s only natural that we talk up its world-changing potential. Then, when it doesn’t quite achieve what we had hoped, we quietly hope everyone will forget all those bold promises we made to the PCC.
There isn’t a culture of evaluation in youth ministry. We haven’t, as a rule, embraced the practice of deliberately stopping and reflecting on the success and viability of our work, of asking the key question: is this meeting our aims? Evaluation isn’t sexy, in fact, it’s terrifying. The process of turning over rocks is scary because we don’t know what we might find underneath them. For various reasons, evaluation isn’t much of a feature of modern Christian youth work, just as it’s not much of a priority for initiative-spewing-organisations.
That’s important because by ignoring evaluation, I believe we’re often setting ourselves up to fail. Evaluation causes us to ask tough questions, even to close the things that we love down or at least radically reshape them (Editors note: Ouch! We didn’t deliberately put this in the final issue of Premier Youthwork). But it’s vital because not only does it force us to check whether we’re meeting our objectives, it ensures that we actually have stated aims in the first place. Without thinking about the purpose of what we’re doing, and whether or not we’re achieving it, we’re really just flailing around hopefully, keeping our fingers crossed that something will work.
Start with the end in mind
If you’re going to evaluate the success of a project, then you need to already know what ‘success’ looks like. Ideally you’ll have established this at the beginning of your initiative, back when the vision was big and you had international domination in your sights. It is possible to retrofit objectives of course, but only if you’re prepared to be honest. Either way, it’s good to have two or three SMART goals (Specific, Measurable, Assignable, Realistic* and Time-related) for your project, which clearly define a picture of success. Once you have these, you have the basis for your evaluation; something to measure against. (*My friend Tom prefers ‘Risky’. And he’s just successfully planted a fast-growing church, so feel free to go with his version.)
While SMART goals are a bit management-speaky for some, they’re useful because they set up the answers to various questions you’ll address in any evaluation. What were we seeking to achieve? Who were we aiming at? How much were we expect to spend, and perhaps bring in? If you only think about these aims when you come to evaluate a project, you’re much more likely to find you hit them, but you’re also much more likely to be kidding yourself. Conversely, if you set aims at the beginning and make sure that everyone involved knows what they are, you’ll be much more mindful of what you’re actually intending to do - logically you’re much better set up to achieve them.
If it’s not too much of a leap, I think we see this sort of focused awareness of objective in the life of Jesus. He knows his plan, and what he’s setting out to do early on: “Woman why do you involve me?” Jesus asks his mother at the wedding in John 2. “My time has not yet come.” Jesus knows where he’s going, and he also knows the timescale. And while he apparently drifts (or as Richard Passmore puts it in his excellent book Off the beaten track, ‘tacks’, sailing-style) towards Jerusalem, actually we find harmonious accounts across the gospels of how Jesus, “Set his face toward Jerusalem” (eg Luke 9:51). Jesus started out with the end very much in mind.
So how do you actually go about setting aims in a way that doesn’t leave you with an arbitrary list? In fact, you need to start even further back by looking at vision, without which - to paraphrase Proverbs 29:18 - all great youth ministry ideas perish. If you have a vision in place for your project or work, you’re able to define a clear benchmark for ‘success’, and out of this you can start to think about the immediate aims of your current initiative.
If that all sounds a bit abstract, let’s put some flesh on it. Let’s say you are a youth worker in a rural area, where you have responsibility for delivering youth work across five villages. Your overall vision statement might be: “To ensure every young person across the five villages hears that God loves them, and has an opportunity to respond.” That’s a pretty big picture vision, but it enables you to quickly define the sort of things with which you’re not going to get involved (such as a project to develop an urban hip-hop showcase). Once you know the vision however, you can look at how you might create a relevant initiative which helps to fulfil it. Once you’ve got the big idea, you and your team (if you have one) can start to create some SMART aims which fit with your vision and ensure that the project moves you closer toward it.
To create them, you could ask yourself a series of who, what, where, why, when questions.
For example:
- Who are we trying to reach?
- What is the basic idea we’re trying to deliver?
- Where will this take place?
- Why are we doing this?
- When will we do it, and when will we know we’ve ‘finished’?
What you’re really trying to answer with all of these questions is the notoriously tricky matter of ‘what will success look like?’, this, in essence, is the question which we bring to the table when we come to evaluation. To help you think about it more thoroughly however, two or three SMART aims will give some clarity and definition to what you mean by success, and enable you to answer the big evaluation question: did we succeed?
End with the start in mind
Evaluation is becoming a vital part of the way that we develop resources and projects at Youthscape. We think hard about the aims of our new initiatives, and we begin them with the friendly spectre of evaluation hovering in mind. The way in which you choose to evaluate may be radically different, but perhaps it’s helpful to walk through what we do as an illustration of what the process can look like.
Having agreed (and committed in writing) to our aims at the start of the development cycle, we pick a date for evaluation. Usually, that’s no less than three, and no more than six months after launch. This works well with physical resources which are released into a ‘market’; it might not be so applicable to a short-term ministry project or programme. In cases like this, we’d be much more likely to carry out an evaluation straight after the project has finished. And in the case of an ongoing programme, we might evaluate part-way through the journey, rather than waiting for it to eventually come to an end.
Next, we decide who will evaluate. Normally we’ll choose at least two people to do this, one of whom has been working on the project from the start, and another who is coming to it fresh. We might also involve people from outside the organisation who have seen the project running or used the resource in question. The key point here: if you only involve people in this process who are motivated to see a certain result, you may not get an objective evaluation.
At this point we restate the aims, and ask the simple question: to what extent has the project met them? To help us get a fuller answer, we then pose a few supplementary questions: did it work for the intended audience? Did it represent value for money? Was it used / engaged with in the way we anticipated? Writing these answers up, even briefly, creates a document which we and others can refer to helpfully when we come to start the next project.
We’ll also ask some other things in the context of evaluation. For example: what new opportunities have been uncovered by developing and launching this resource or initiative? Lots of the great inventions and discoveries were accidental by-products of a different journey - the microwave, the pacemaker and penicillin among them. When we’re thinking through our development processes, we often discover new things we never anticipated, so it’s worth reflecting on whether that has happened.
Finally, we’ll ask a really important question: what should happen as a result of what we’ve learned in this evaluation?
Change, celebrate or close?
Once you’ve evaluated your project against its original aims, it’s time to decide what action to take. Broadly speaking, there are three possible routes to take.
Enhance the offering
It may be that you’ve identified some significant weaknesses or areas for improvement, which could inform a revised version. That could mean issuing a new edition of a resource, or running the second iteration of a course for young people with radically different content. It might mean taking the project in a whole new direction that you never imagined, or making subtle tweaks which create a big difference. This is perhaps the most useful application of evaluation, where learning from our mistakes and successes drives further innovation.
Celebrate success
Your evaluation might reveal that you have largely met your aims - and if this is what you learn, you should take time to celebrate that achievement. There will still be plenty to learn from your evaluation and changes may still be implemented, but at this point your thoughts might also turn to wider dissemination. At Youthscape our innovation model is about testing new ideas locally, and then sharing what works nationally (or at least, more widely). It could be that you could share your successful project with other youth workers who are searching for youth ministry models that really work.
Give yourself a license to kill
We’re not historically good in youth ministry (or in the wider church) at pulling the plug on things, perhaps that’s because we’re scared of the idea of ‘failure’, when in fact that’s a concept we should all embrace. Vineyard pioneer John Wimber apocryphally became annoyed that more of the church plants in his network weren’t failures, because that obviously demonstrated that the movement wasn’t taking enough risks. So if the evaluation reveals that actually the hypothesis didn’t hold, see that as a badge of innovation honour: perhaps you took a good risk, even if it didn’t pay off. In these cases, we shouldn’t be afraid to cease a project, especially if a robust piece of evaluation work demonstrates that we did everything we could to test and make it work.
It’s my firm belief that unless we embrace a culture of evaluation across the youth ministry world, we’ll continue to develop half-baked strategies and resources which might have been truly great if only we’d put a bit more thought in. It might sound like we’re adding unnecessary process and paper-work, but in fact we’re just doing the job of development properly. Whether you’re trying to create a youth group context which really meets the needs of young people, or an expensive video resource to communicate the Christian faith to a whole generation, setting clear aims and evaluating your work against them is vital. Dare to turn over the rocks - brave what’s underneath. Those discoveries might just help us to make a real breakthrough in our work among young people.
