In simple words, evaluation means looking at the strengths and weaknesses of something. But, the term could be interpreted differently by different people. For instance, there is still a strong belief that an evaluation is conducted for accountability and making judgements. While this might be true for funders and commissioners who require evidence to guide their decision of funding allocation, seeing an evaluation solely for this purpose could create missed opportunities for organisations and individuals who want to make a real, positive difference in society.  

Shifting the focus of an evaluation to be more about learning for improvements promotes the culture of honesty, openness and creativity, where innovative ideas are encouraged and learning is openly shared with others.

A well planned evaluation, driven by our intentions to make improvements, help see what a real impact interventions could have on its users and the wider society. Without a timely evaluation, interventions, even designed with good intentions and for achieving good, could end up delivering negative outcomes. To illustrate this point, let’s look at a real life example.

In 1995, the Play Pump^® was an innovative idea of achildren’s roundabout,when spun would pump clean water from deep underground up to a storage tank. It was a ground breaking solution to utilise the power of children at play to provide a sustainable water supply for the poor communities in Africa, where access to drinking water was a huge problem and playground equipment for children were not common.

The idea of the Play Pump^® was endorsed by the world leaders, famous celebrities, world leading charities, and even awarded with a World Bank Market Place Award in 2000. It ignited altruistic behaviour resulting in many people donating and investing millions, including a $16.4 million grant awarded by then First lady Laura Bush for developing and deploying Play Pump^® across Africa.

The Play Pump^® was considered a great development idea, driven by passion of ‘doing good’ for the poor communities. But, did this idea work for local communities? Did it have a positive impact on people and society? The answers to these questions only became apparent when UNESCO published its evaluation report in 2007¹.

The report concluded that despite of its innovative technology, the use of the Play Pump^® was a failure at many levels. The local communities were not involved or asked if they wanted a play pump in the first place. When asked later, many said they would prefer a ‘hand pump’, which was easier to use and would discharge more water as compared to the new play pumps. Children were unwilling to play due to the hard work involved in spinning a roundabout built with heavy machinery.

A well supported and well-hyped solution failed to achieve its intended outcomes. Mainly, because it was purely driven by emotions rather than an evidence base. The independent evaluation reports provided the Play Pump^® anevidence for learning, helping them solve the problem of a sustainable water supply in a more improved way.

In the current challenging times when the whole world is dealing with the unprecedented COVID-19 crisis, it is important more than ever before that we evaluate our efforts of doing good for communities. A timely evaluation can help see what is working and what is not working, hence informing what could be improved to achieve a real positive impact of supporting communities in need.

Dr Shehla Khalid
Evaluation and Project Improvement Officer

¹ UNESCO, 2007: An evaluation of the Play Pump^® Water System as an Appropriate Technology for Water, sanitation and Hygiene programmes