The Mighty Blog

The Evaluation Trap - A Way Out

Gary Romano
Posted by Gary Romano on Jul 25, 2018 9:12:00 AM

When Executive directors and CEOs of nonprofit organizations share their greatest challenges with me, the list of common frustrations can include- fundraising, board development, and effectively managing personnel.    When I start to focus on how to overcome their biggest obstacles, these senior managers begin to realize that one of the biggest hurdles they face was not even on the list, yet it is integral part of all the challenges facing them. The need for data.  The need for data leads directly into the “evaluation trap”.  The good news is there is a solution to this trap, most of nonprofit leadership is simply not aware of it.

bigstock-Data-Analytics-Concept-Vector-239842609What is the Evaluation Trap?

The first lesson is to understand how the trap works- Nonprofit leaders are consistently told they need independent evaluations to grow their organizations, to show impact and success and to solicit additional funding in the private or public sector.  These evaluations can be costly and time consuming.  The trap sets up the nonprofit with an understanding for the need of essential data and independent review to garner greater funds and respect, without the funds to commission such an evaluation.  The nonprofit needs the independent evaluation to show to funders and stakeholders its effectively executing part or all its mission, yet feels the cost associated with an evaluation is an insurmountable obstacle from the perspective of a nonprofit budget.   

The typical CEO or Executive Director absolutely understands the need for and desires independent evaluation for themselves and external stakeholders. Internally, nonprofit leaders are often uncertain about model fidelity, impact, and how to keep improving the impact of their work. In reviewing their own internal data, questions quickly arise – is the internal data good enough? Are they collecting the right data? Are the people they serve just being nice not honest about their services?  Is the way we service are stakeholders worth sharing?  All fair questions, and the answer to many of them is perspective and independent review from an individual outside your organization.

The nonprofit leaders I work with feel this pressure, not only from foundations but more sophisticated corporate and individual donors who want to understand their return on investment.  This need is real and fair – these funders are putting their money into your work; shouldn’t they have some sense of what impact it may have?

And this is where CEOs and executive directors are stuck in the evaluation trap- they want and need independent data to understand how effective they are executing their mission. They want and need independent data to gain understanding on how to grow the organization. They want and need independent data to show donors how they believe that their mission has helped their community.   They look at the “gold standard“ evaluation as essential yet unattainable.   Every organization I’ve known, regardless of size is humbled by the cost and scope of the perceived evaluation. They are easily facing hundreds of thousands of dollars and in many cases for a randomized controlled trial over a few years, $1 million or more.

This finds nonprofit leaders stuck between the data they want and truly need but they believe they can’t afford due to budget concerns.

My firm often acts intermediary for philanthropic foundations looking to invest in high quality programs. Ten years ago, our clients would be happy if there was one independent evaluation on a perspective grantee’s program. Now, the question they ask is “How MANY independent evaluations do they have?” Without any independent assessment, they would likely not even consider an investment.

The way out of the Evaluation Trap

There are newer approaches to evaluation which are starting to gain traction and can help nonprofit leaders find a way out of the “evaluation trap”. What continues to baffle me is how few nonprofit leaders have an understanding and awareness that there are options.  They mostly perceive it is an all or nothing scenario.

First off, you need to understand that the world of independent data isn’t the binary system you’ve been misled to believe it is. When I talk with my clients about independent data, I see three levels:

  • Level 3- This is the traditional evaluation you’re already thinking about. And independent organization comes collects their own data (likely quantitative and qualitative) and analyze it for you. Data can be collected over a year are multiple years.
  • Level 2- The independent of evaluator reviews your existing data and supplements it with additional data they collect, such as qualitative one on one interviews, focus groups, or surveys. They provide an independent analysis that is now based on some of their own data but also leveraging what you already have.
  • Level 1- An independent evaluator just analyzes you’re existing data. No new additional data is collected.   Yes, it’s still your data that you could have looked at anyway but now you have an independent perspective and expertise applied to it for additional credibility.

Level one and level two are evaluations rarely considered by executive directors and CEOs and that’s a shame. They are most likely overlooked due to lack of awareness.  I have seen many, many instances where funders look at these types of evaluations and understand are not the same caliber as a level 3 but also recognize them as what they are – an independent assessment of your organization and can value them accordingly. Best of all these evaluations are can be an affordable option, allowing nonprofits to consider them as a viable option within their budget constraints.

My blog post "The Evaluation Options the Pros Never Tell You About" will give you the details on "rapid evaluations" and how they can affordably impact your organization.

I would happy to talk about finding a way out of the "Evaluation Trap", please feel free to set up a free consultation. 

Topics: nonprofit, What every CEO and ED should know, evaluation, rapid evaluation

Achieve Your Vision

The Mighty Blog is only a start

Civitas Strategies has been helping leaders just like you achieve their vision of service since 2010. We help leaders of organizations from startups to large nonprofits and foundations by helping them:

  • Attract and retain top talent.
  • Craft and implement effective strategies.
  • Use evaluation data to improve effectivenss.

Subscribe to Email Updates

Recent Posts