Estimated reading time: 5 minutes, 58 seconds
Do You Know What Your Organization is Delivering?
In May 2005, the independent Panel on the Nonprivate Sector recommended to the Senate Finance Committee that "charitable organizations establish procedures for measuring and evaluating their program accomplishments based on specific goals and objectives."
More than two and a half years later, the Wall Street Journal devoted most of its Dec. 10, 2007 "Philanthropy" section to the issue of accountability and measurement. The subtitle of the lengthy and critical front page story by Sally Beatty indicates how little progress has been made in that area: "We give billions of dollars to causes we believe in. So why do we know so little about how effectively the money is used?"
In other words, nonprofits are facing increasing scrutiny on every level. Donors are frustrated that, as a whole, charitable organizations aren't transparent enough about how they spend their money, with 71 percent of respondents to a 2006 survey conducted by NYU's Wagner School of Public Service agreeing with the statement that "Charitable organizations waste a great deal or fair amount of money." Perhaps even more damning was that fewer than two in 10 -- 18 percent, to be precise -- believe that "Charitable organizations do a very good job running their programs and services."
In a nutshell, nonprofits are facing a major credibility program, one that's arisen because increasing calls for accountability, both from donors and from the federal government, have gone largely unheeded. As a whole, charities provide the public with too little information on successes and failures, on progress toward meeting both general missions and specific program goals.
Are You Keeping Secrets?
Beatty argues that much of this problem can be traced to an entrenched "culture of secrecy." But secrecy may be masking an even more serious problem: most nonprofits simply don't measure program effectiveness. And as Steve Butz, the co-founder and President of Baltimore-based Social Solutions, puts it, "If you're not measuring what you're doing, you call into question your existence."
Butz has a vested interest in making such a statement. His company makes measurement software called ETO ("Efforts to Outcomes"). The application, which is highly customizable, went on sale in 2002 and already boasts a user base of more than 1,200 distinct 501(c)3 organizations. The Boys and Girls Clubs of America, one of the country's largest charities, makes up a large chunk of ETO's user base. Butz says ETO is unique -- he can't think of a direct competitor -- because it enables a wide variety of organizations to view "the efforts of staff" and understand "the relationship of what it (the organization) is doing in relation to outcomes."
ETO doesn't come cheap. The "Standard Edition," for non-profits with up to 50 staffers, costs $7,500, and the "Professional Edition," for larger organizations, costs $25,000. Butz says that "the return on that little $7,500 investment will be 10-fold," because it enables organizations to "better tell their story," which results in more donations.
It's a believable argument, especially considering NYU's survey results and that, according to Beatty, nonprofits often deny requests for information from watchdog organizations like the Wise Giving Alliance , and "routinely" fail to fully complete IRS Form 990. Form 990, a financial report that forms the basis for charity ratings by groups such as Charity Navigator , also includes a section that requires they "describe their exempt purpose achievements," including "the number of clients served, publications issued, etc." and "qualitative achievements."
What's More Important? Output or Outcome?
The concept of measurement may be easy to talk about, says Thomas H. Pollak, but it's not easy to do. Pollak is the program director for the Urban Institute's National Center for Charitable Statistics . "It's not that simple, especially for smaller organizations, to measure outcome," he says. "To get at outcomes, as opposed to outputs, is very expensive." Pollak says that while a question such as "What is our impact on people and community we served?" is simple, answering it is complicated. Many variables are involved, and determining cause-and-effect can be elusive, especially for social services, human services, and arts organizations. For example, says Pollak, many nonprofit arts organizations "distinguish themselves from their for-profit brethren" by being innovative. But how do you measure innovation?
The Urban Institute is actively wrestling with such questions. In December 2006, it issued a report, along with The Center for What Works , suggesting 14 "core indicators" that nonprofits can use to measure performance. But the preface to the report, "Building a Common Outcome Framework to Measure Nonprofit Performance" cautions that its contents consist of "initial material" and the indicators "require further research."
Social Solutions isn't waiting for the research. Butz says that his company, founded in 2000, was borne out of his personal frustration while working with at-risk youth. He found his job description, "to mentor and be an advocate to at-risk youth" vague and confusing. "I didn't even know how to take one step forward with that job description," he says. Meanwhile, the organizations had no way to "tell whether I'm any good at it."
One of the biggest barriers to implementing and reporting consistent, clear metrics, argue Pollak and others, is simply time. Shorthanded nonprofits have enough trouble serving their clients, and determining things like client satisfaction (through surveys) and long-term success (by following what happens to clients after they leave the program) requires, says Pollak, "a champion within each organization to really get the data collection to where it needs to be. I've seen the pushback -- it's hard for direct service organizations and their busy staffs to make a commitment to do this."
Butz argues that measurement isn't difficult -- but it does require organizations to shift their priorities. Nonprofits should "serve less people and measure what they're doing," he says, and he believes that qualitative measurement, while challenging, is far from impossible. To that end, Social Solutions has shifted its focus "to take care of the (how do you measure?) problem. This year we've developed a core set of evaluation tools, outcomes, assessments, things that we give to our customer base to help them get a starting point." As an example, Butz mentions simple surveys that organizations can give to their clients that get at more elusive, subjective outcomes. While not perfect, collecting and reporting such data can help nonprofits get grants and assure donors that they're making an attempt to measure what can be measured.
But Pollak and Butz -- and others concerned about nonprofit accountability -- agree on the importance of such a move. "We have a team of people at the Urban Institute who've been working on performance measurement, outcome measurement in the context of federal, state and local governments for decades," says Pollak. Measurement also goes "to the heart of the legitimacy of the nonprofit sector."
The crucial question, he adds, is "What is the value that the society gets back?" Clear answers don't seem to be on the horizon. Last modified on Sunday, 19 May 2013
- Panel on the Nonprofit Sector,
- Senate Finance Committee,
- Urban Institute,
- The Center for What Works,
- National Center for Charitable Statistics,
- Wall Street Journal,
- Social Solutions,
- Steve Butz,
- Boys and Girls Clubs of America,
- Wise Giving Alliance,
- Sally Beatty,
- Thomas H. Pollak,
- Wagner School of Public Service,
- Charity Navigator,
- performance measurement,
- outcome measurement,
- Form 990,
- Joomla templates,
Jul 22 2016
Written by Jennifer Flaten
Feb 16 2009
Written by Kurt Martin
Sep 12 2013
Written by Jennifer Flaten
Jun 02 2008
Written by News