Measuring the Effectiveness of Capacity Building Support: A Metrics Menu

By Sammy Politziner and Scott Thomas, Arbor Brothers

As funders increasingly seek to supply resources beyond financial support to their grantees, many are engaging third-party consultants to provide capacity building assistance. But how can funders know whether these resources are really strengthening grantees? What are some reasonable benchmarks for evaluating this assistance?

We at Arbor Brothers are fervent believers in the value of capacity building—so much so that we have organized our entire grantmaking model around it. We certainly do not claim to be experts, but we are pleased to share here our perspective on this challenging but important topic.

First, a little context. My business partner Sammy and I launched Arbor Brothers in 2010 to find, fund, and support the most promising early-stage education nonprofits in the New York City area. Our grants are composed of two parts: up to 3 years of unrestricted financial support totaling more than $250,000 and hundreds of hours per year of capacity building consulting, which we deliver personally.

Our consulting efforts focus on developing key grantee competencies such as financial management (budgeting, earned income streams), strategic planning (multisite expansion, board engagement), and outcomes measurement (program dashboards, staff accountability). By helping nonprofit leaders create the tools to manage systematically, we believe that over time they can build a sustainable, outcomes-focused organizational culture. This culture is essential, in our view, to achieve long-term social impact.

As we deployed our strategy and began to reflect on our efforts, we sought to provide our board a framework for assessing our work and holding us accountable for improving. We cast about for industry standard benchmarks for evaluating nonprofit consultants, figuring there must be some commonly accepted guidelines for success in these engagements.

As it turns out, every single funder and capacity building consultant we have spoken to (including a number of large brand names) either operates with no meaningful set of standards for evaluating consulting work or has invented their own. To our knowledge, there is no publicly available, one-size-fits-all gauge for measuring capacity building effectiveness. And for good reason: These services are not only hard to deliver, they are exceedingly difficult to evaluate.

After four years of implementation, tinkering, and reflection, here is our conclusion: An assessment of capacity building support ought to be objective, timely, and mission-aligned. As no single yardstick features all these qualities, effectiveness is best evaluated by triangulating among a handful of metrics.

The “metrics menu” below probably merits a dozen footnotes and caveats. Here are two crucial ones:

  • Trade-offs—Each metric presents a different mix of objectivity (990 tax filing vs. someone’s biased opinion), timeliness (during the engagement vs. 1-2 years later), and mission alignment (is scoring high tantamount to serving effectively?).
  • Judgment—Picking the right blend of metrics requires making a judgment largely based on the scope of the engagement (smaller projects aimed at discrete end-products may best be evaluated with a checklist or survey, but deeper collaborations with more ambitious goals are better analyzed via longer-term metrics).
Objectivity Timeliness Mission Alignment Metrics Arbor Brothers Example
MEDIUM HIGH LOW Did the consultant provide the agreed-upon deliverables? “Do we now have a 3-year fundraising plan?” This should be a cut-and-dried question, right? But quality and ongoing utility are matters of judgment. Does management understand it? Does the board believe in it? Is it flexible enough to be adapted? Progress report
LOW HIGH LOW What did the grantee think of the engagement? “Oh yes. That was super-helpful. In unrelated news, I really appreciate your continued financial support of my organization.” Power dynamics between grantees and funders are so skewed that candid feedback on the usefulness of consultants is rarer than hen’s teeth. That said, it is still worth asking in case meaningful critiques or suggestions emerge. This also allows grantmakers to check in along the way and potentially make course corrections. Grantee survey
HIGH LOW LOW How did grantee revenue grow during and after the engagement? Growth is great, but fundraising results may only be relevant for certain engagements (development planning, board engagement, creation of earned-income streams), are often a 1-2 year lagging indicator, and can act as a perverse incentive to pursue scale before the foundation of quality programming is secure. Revenue growth
MEDIUM MEDIUM MEDIUM How did the grantee fare with key local funders? Some funders matter more than others. In your community, which are the handful of funders whose reputations and deep-vetting approaches are likely to send a strong signal that the grantee is “of quality”? How many of those funders did the consultant connect or prepare your grantee to pitch to, and what was the grantee’s success rate? Signature funder batting average
LOW MEDIUM HIGH To what degree is the grantee able to meet the current programmatic, financial, and staff needs without compromising the future? Has the engagement helped the grantee develop a comprehensive approach to managing revenue, budgeting, staff, board and external stakeholders? A rubric can help quantify progress in this long-term, continual process. Sustainability rubric
LOW MEDIUM HIGH To what degree is the grantee managing through systems and aligning resources efficiently to generate outcomes? Has the engagement helped the grantee tighten programmatic focus, marshal data, and motivate the staff and board in a relentless pursuit of better outcomes? A rubric can help here as well. Outcomes-focused culture rubric

At Arbor Brothers, we track each of these metrics with our grantees, but, over the long term, we hold ourselves most accountable for helping to develop outcomes-focused cultures.

Related posts
Grantee Capacity Building: First, Do No Harm
Convening to Build Capacity

Related resources
Capacity Building Resources for Funders, a list prepared by the National Council of Nonprofits
Fire, Aim, Ready: Why Most Foundation-Funded Capacity Building Efforts Miss the Mark

Arbor Brothers (@ArborBrothers) is committed to finding promising social entrepreneurs and helping grow their ideas into powerful and sustainable forces for change. To help meet this challenge, Arbor Brothers employs an engaged philanthropy approach: providing both financial support and in-depth consulting to entrepreneurs whose organizations have moved off the drawing board and onto the growth curve. The Arbor Brothers portfolio and related consulting engagements are supported by a limited pool of funders including family foundations, individuals, and donor-advised funds. 





4 thoughts on “Measuring the Effectiveness of Capacity Building Support: A Metrics Menu

  1. Pingback: 11 Paths to Effective Giving – Recapturing What Is Powerful | PhilanthroFiles

  2. Pingback: 9 Ways to Make a Difference: Strategies From Small-Staffed Foundations | PhilanthroFiles

  3. Pingback: 15 Top Posts From 2015 | PhilanthroFiles

  4. Pingback: New Report Details How Foundations Create Change | PhilanthroFiles

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s