Susan M. Weber: Reframing evaluation

Why do religious organizations find evaluation work so mysterious? What do they need to help them appreciate the importance of evaluation and to do evaluation well?

I asked these questions when I first began coaching new grantees of the religion division of Lilly Endowment Inc. As I sat alongside grantees, I often noticed a hesitation and reluctance about embarking on evaluation. It wasn’t as if evaluation was a new idea. Rather, I realized that the mere word “evaluation” raised less-than-pleasant memories that limited their openness to this work. Grantees wondered: Is my work being judged? Will someone form an opinion about the merit or quality of our program?

Indeed, how many of us hold a memory about an evaluation experience gone bad or a moment of being judged or of having someone else’s opinion supersede our own?

At the heart of the word “evaluation” is “value” — to find the value of — and this notion guides my evaluation coaching work to this day. At its most effective, evaluation is an activity of valuing and learning from one’s work, not a judgment. Principles such as collaboration, inquiry, learning, curiosity and mutuality guide such an approach to evaluation.

This evaluative lens recognizes that projects are conceived with the aim of creating good in the world and ultimately making a tangible difference in people’s faith lives. It also changes the tone and character of the questions one asks:

• How is our program/project making a difference in the lives of young people, pastoral leaders, teachers, students or other target audience? How do we know?
• What conditions will change as a result of our program? What new knowledge is being gained? What new actions are being taken as a result of our program, and why? What new attitudes are being formed?

Such purposeful questions engage stakeholders’ curiosity and allow them to be open to developments they did not anticipate in the original program design. From my studies of Appreciative Inquiry, I know how thoughtful, positive questions can change the trajectory of an evaluation. For example, reframing, “What is not working in our program?” to, “What gives life to our program when it is functioning at its best?” can open up constructive, nondefensive conversation and new methods for discovering answers.

If projects are to improve and grow, they will do so as a function of what we choose to ask and study. In “Appreciative Inquiry Handbook: For Leaders of Change,” the authors write:

“Inquiry and change are not truly separate moments. … Inquiry is intervention. The seeds of change are the things people think and talk about, the things people discover and learn, and the things that inform dialogue and inspire images of the future. They are implicit in the very first questions asked. … The questions set the stage for what is ‘found’ and what is ‘discovered’ (the data). These data become the stories out of which the future is conceived, discussed, and constructed.”

If projects are to be sustainable and potentially imbedded in the very institutions that house them, then reflective conversations must be woven into the life of a project from the beginning. Moreover, a project director cannot engage in this kind of thoughtful conversation alone. Creating communities of learning and practice is key if grantees are to become their own active agents of evaluation and to gain confidence in this work.

“What is at stake is the efficacy of love and care and service” writes Craig Dykstra, former vice president for religion at Lilly Endowment, in Kathleen Cahalan’s book “Projects That Matter: Successful Planning and Evaluation for Religious Organizations.”

“What is at stake is how human beings engage with and relate to one another. What is at stake is how minds are illuminated, hearts are moved, burdens are lifted, wounds are healed. And because so much is ultimately at stake, it matters a lot that the projects and programs be done really well.”

A community of learning and practice embraces the responsibility to inquire and to learn from its work, to improve its project as a result of that learning and, in turn, to contribute to strengthening the larger landscape of Christian life, ministry and institutions. A community of learning and practice is made up of thoughtful stakeholders who care deeply about the work at hand and can ask the kinds of hard questions about a program that are essential to program improvement and sustainability.

So what is the result when evaluation is focused on learning and not judging?

This approach provides a lens through which and a structure out of which grantees can assess the impact of their work, learn from that work, and revisit original questions and assumptions to strengthen, sustain and bring the program to full maturity within the fabric and mission of the institution. Their organizations can develop renewed energy to risk and experiment with innovative strategies. Their team members can become more enthusiastic for and appreciative of evaluation and can experience it as joyful work.

Reframing evaluation can also broaden grantees’ awareness, to a wider community. The body of knowledge, information and insight gained through evaluation has great value not only to the institution running the program but to a larger community that cares deeply about the future of religious institutions entrusted with caring, teaching and serving others.

Finally, this approach to evaluation can offer a pathway to theological reflection. Evaluative findings can provide meaningful moments for grantees to pause and reflect on how a project is advancing the mission and ministry of the institution and to ask deeper questions about how they understand the work in relationship to the call and presence of God.

As inspiring stories from projects begin to emerge — how a program has changed the vocational choices of a young adult, taken a pastor out of isolation into a trusted circle of peers, reshaped the culture and priorities of an institution — they can open our eyes to a larger canvas of vital institutions carrying out the mission of the church and flourishing as a result.