Measuring The Impact Of Your Charitable Donations

Transcript

ARI SHAPIRO, HOST:

This is TALK OF THE NATION. I'm Ari Shapiro in Washington. Neal Conan is away. This week, many of us are making year-end charitable donations. There are countless needy people and worthy causes competing for our dollars. So many philanthropists today want proof that an organization actually succeeds at its mission.

This hour we'll learn how to tell whether your donation is doing what it's supposed to. We want to hear from you. If you're a donor, how do you measure the impact that your money is having? If your organization collects charitable gifts, how do you prove that you're succeeding at your mission? Our number is 1-800-989-8255. Our email address is talk@npr.org. And you can join the conversation at our website too. Go to npr.org, and click on TALK OF THE NATION.

Later on the program, how to deal with tragedy during the holiday season. But first, Ken Berger is the president and CEO of Charity Navigator. His organization rates charities, and he joins us by phone from his home in Fairlawn, New Jersey. Ken, thanks for being with us.

KEN BERGER: Sure.

SHAPIRO: What is this term evidence-based giving? What does it mean?

BERGER: Well, it really is this notion of measuring the performance of an organization to show that in fact there is evidence that it is doing what it states that it wants to do. So it's a way to measure performance among charities.

SHAPIRO: Is it a relatively new thing?

BERGER: Actually it's been around for a very long time. There are people who indicate it's gone back many decades, even to the time of Carnegie, going all the way back to the early 1900s. But it's really gained momentum as almost a movement within the past decade or so.

SHAPIRO: I remember years ago I heard about your organization, Charity Navigator, as a way of finding out whether a charity spends, say, 10 percent of its donations on overhead or 40 percent of its donations on overhead. That seems like a pretty easy thing to assess; you just look at a budget document. It seems much more challenging, though, to evaluate how effective a charity is at meeting its goals. How do you do that?

BERGER: Well, in fact that's very true, and it certainly is one of the most challenging things, and today the vast majority of information that you get out there that's standardized is more on the financial measures of things and some other performance measures, but not really on evidence.

So it's - there are a variety of ways that it can be gone about, and we are planning to begin measuring it in some particular ways. One example of a way that's pretty universal that we think could be used to measure, and some charities have begun to, is coming from the world of customer satisfaction surveys. But in this case it's the beneficiary or the client, asking the client themselves whether or not the services were what they needed and were satisfactory. That's one example.

SHAPIRO: Can you give us others? I mean it seems like with such a wide array of charities, you would need just as wide array of ways of measuring outcomes.

BERGER: Yes, and I think that each charity needs to basically create a roadmap. It's been called a theory of change, but it's really like a roadmap, saying here's the problem, and here's what we want to get to, and here is how we are going to measure how we get from here to there. The evidence-based issue is if there's been research already done in a particular type of charitable endeavor, you would hope that the charity would go and look at that evidence that already exists and model itself, and at least some of the measures that it would utilize would be from that evidence-based research so that, you know, people have been on this map before, they've gone down this road before, and they know where the potholes are, and they know some of the ways to get down that road as quickly and effectively as possible.

And that's part of the spirit of this evidence-based learning that charities should be relying on.

SHAPIRO: This sounds almost parallel to something I've heard about in medicine, where doctors who are used to doing the same thing just because it's the way they've always done it have to be taught to study which practices actually work and do more of that and less of the things that don't.

BERGER: Yeah, and in fact that's also been a definition of insanity, you know, doing the same thing over and over again and expecting a different result. So - and I think that's part of the challenge here, which is that historically charity, a lot of the work in the charitable sector has been based on gut, you know, I know I'm doing something well because I see the interactions and I see the change in my direct work.

But really having a disciplined kind of, you know, research to determine what is the very best way is still very much in its infancy.

SHAPIRO: Is this being spurred from within, from charities that want to do better, or from without, from donors who want to know that their money is going to the right place?

BERGER: Well, I think it's coming from a number of directions. There are some charities, an example would be the Harlem Children's Zone, that the founders were very committed to this model. And at the same time there's definitely a movement among some philanthropists, you know, people from the business world like Bill Gates that, you know, want to see a return on investment kind of a model. So some of that has definitely been an influence as well.

SHAPIRO: We want to hear more from the people who are on the front lines of this. If you're a donor, and you want to know that your money is going to the right place, what questions do you ask? What statistics do you analyze? Our number is 800-989-8255. Or email us at talk@npr.org. We're here with Ken Berger, who is president and CEO of Charity Navigator. And Ken, I've seen lists of the sort of top charities for donations in the country. Do those tend to correlate with the charities that do the best job at the kinds of outcomes that you're talking about?

BERGER: Not necessarily. It depends on how the lists are compiled and what data's used. So some of the lists are purely based on financial analysis. Some of the lists have a combination of measures. It's really not always clear. You have to know what the list is made up of.

SHAPIRO: So I can totally imagine how you measure performance in an organization that is dedicated to providing beds for the homeless - you know, how many beds did we get? Or reducing teen pregnancy - how many teenagers got pregnant? But if the goal is to inspire a love of the arts, how do you measure something like that?

BERGER: Well, you know, the thing is that actually when you - one of the ways to think about what we call measuring results is that the further away you get from the individual to something broader, more general, the more challenging it may be to be able to measure it.

But even in the example of the arts, I mean one thing that you can do is to survey the people who are experiencing that art and seeing if it's a fulfilling, enhancing experience for that person. You know, there's also studies of communities and how vital to the culture of a society the arts are and, you know, sort of measuring the results of the art.

There's also, you know, there's another measure - the most rigorous measure is a randomized control trial. So you could take a community where there's a real paucity of art and then another where there's a great deal of museums and ballet companies and so forth and just look at the general satisfaction of people in those communities, as one example.

SHAPIRO: So what do you say to a charity that says look, you're already judging me on what percentage of my donations are spent on something other than providing services. Now you want me to spend a bigger percent of my donations on measuring how we did at providing those services.

BERGER: Yeah, well actually we believe that there are three pillars in making a wise charitable giving decision. One of them is finance. Another is this results, and the third is looking at the governance of the charity. All three are relevant, but clearly, obviously, the most important are the results of the work.

And so, I mean, as we're evolving our rating system, we are going to be mindful of the fact that if a charity has very robust management, performance management for results, then there's a cost associated with that. And so the overhead in that case might be higher, but it's worthwhile because the most important thing of all, the quality of its results, are there.

So there may be, let's say, less money going into program, but the effectiveness and the quality is so much higher that that trumps the overhead.

SHAPIRO: We have an email here from Emily(ph) in Ontario, California, who says: I donate to local nonprofit animal rescue groups because it's easy to see where the money goes, and I like knowing I'm helping those working in the trenches. Often, she writes, my donations go directly to the veterinarian treating a particular sick or injured animal in the organization's care, so there's worry of misuse of funds.

Ken, it seems as though having that sort of small chain between the donor and the recipient makes a difference in transparency certainly.

BERGER: Yes, and, you know, there are more charities in America than any other country in the world. There are 1.6 million nonprofits. And the vast majority of these organizations are very small. I think half are 25,000 or less each year. They're very small, local efforts. And really the best way is to eyeball it yourself and to up close and personal make an assessment.

And a lot of this is just common sense, seeing - and I think in this example, where you can clearly see those results if you have a close look.

SHAPIRO: Great, let's go to a call. This is Elsa(ph) in Jacksonville, Florida. Hi Elsa.

ELSA: Hi, how are you?

SHAPIRO: Good thanks. Go ahead, what's your experience?

ELSA: I went to college on a program called Southern Scholarship in Tallahassee, Florida. I was accepted to FSU and in fact went to FSU because I had a good chance of getting into this program...

SHAPIRO: That's Florida State.

ELSA: Which allowed me to go through undergraduate school and law school and only come out owing about $3,500, including my law school. And, you know, this was some years back, so unfortunately kids come out owing a little more. But it was a cooperative living program where you lived together, cooked your food together.

You know, the housing was provided, and then the members of the house, about 25 students depending on the size of the house, worked cooperative to feed themselves and, you know, maintain the house, that sort of thing. The administration costs of that program were pretty low, and, you know, we kind of took care of our own houses.

But sometimes the housing wasn't as wonderful as one might like, as the - you know, oftentimes - when I graduated from law school, I was elected to the board and later became president of the organization. So I saw it both as a student, and I saw it from the governance end.

And I think oftentimes we think we want to lower the cost of administration of a program, but you can lower them so much that you're not effective in, you know, providing the program.

SHAPIRO: And just briefly, Elsa, as a grownup now who presumably donates to some charities yourself, do your experiences as a recipient inform where you decide to give?

ELSA: I think, you know, absolutely. I think - I guess I was one of those people who when I had the debate about, you know, do you teach a man to fish, or do you provide him the fish, I always believed it was better to teach a man to fish. And so I think when there's a program that allows you to get an education, that's always been really important to me.

I think any moneys that we give to people rather than things is also - you know, especially during these times is very important.

SHAPIRO: Sure, thanks for the call, Elsa. And coming up, more on tracking your charitable giving. Plus we'll talk with journalist Matthew Bishop about the people who drove this trend, the wealthiest among us. And we also want to hear from you. The number is 1-800-989-8255. Or email us, talk@npr.org. Stay with us. I'm Ari Shapiro, this is TALK OF THE NATION from NPR News.

(SOUNDBITE OF MUSIC)

SHAPIRO: This is TALK OF THE NATION from NPR News. I'm Ari Shapiro. Christmas may be over, but we're still in the season of charitable giving. This hour we're talking about how to tell where your dollars going when you give. And we want to hear from you. If you're a donor, how do you measure the impact that your money is having? 800-989-8255 is the phone number. And our email is talk@npr.org. You can also join the conversation at our website. Go to npr.org, and click on TALK OF THE NATION.

We're going to take another caller. This is Mark from Nashville. Hi Mark.

MARK: Hi, my name's Mark, and I support this organization called Centerstone, and they provide information to us donors periodically, and one of the one that impresses me is this in-home treatment program, which deals with kids who are victims of abuse or other kinds of trauma in their lives.

And they show me statistics and reports that show that 93 percent of the kids who they get engaged with remain in their homes, and only two percent get hospitalization, and they document this through one of their agents and one of their, I don't know, one of their segments of their business that is called the Centerstone Research Institute.

And then they did the same thing for, you know, children of veterans, and they showed that 100 - out of 522, 160 were prevented from being homeless, and 362 were placed in rapid re-housing, what's called rapid re-housing. And that's a program...

SHAPIRO: And do those statistical data, do those statistical data make you more likely to give?

MARK: Well, you know, I look at it and say, you know, this is really - you know, I've heard some of the other veterans programs that raise a lot of money, and the money doesn't go to provide services, and this Centerstone Research Institute, which provides information on what Centerstone is doing, really gives information, even gives some information from the federal government that ranked them as a mentor program that they're going to try to do their program nationally.

So yeah, to me it just shows that, you know, here we're giving money, and it's not going to marketing, but it's going actually to direct service.

SHAPIRO: All right, thanks for the call, Mark.

MARK: Thank you very much.

SHAPIRO: Let's bring Matthew Bishop into the conversation. He's U.S. business editor and New York bureau chief at The Economist magazine. He co-authored a book called "Philanthrocapitalists: How Giving Can Save the World." He joins us by phone from London. Matthew, thanks for being on the program, and sorry for tripping over the title of your book there.

MATTHEW BISHOP: Yeah, that's the trouble with writing a book with an unpronounceable title, isn't it? It's "Philanthrocapitalism."

BERGER: "Philanthrocapitalism." We have an email question here that I think you would be able to answer. Gayle(ph) writes: I've been involved in the revamping of United Way's funding model. It seems to be going on nationwide for evidence-based results. What's the origin of the movement? Do you have an answer for her?

BISHOP: Yes, I mean evidence-based philanthropy, it comes out of this concern that a lot of people were writing checks, and they didn't know whether the money was making a difference. And so they decided to try and measure effectively what happened to the money and not just to measure it but also to have an idea as to how it ought to look if things were going well so that they could compare what actually happened with their predictions because you can measure all sorts of things, but if you don't actually know what you really expected, then that data doesn't tell you very much.

And so amongst the leading philanthropists, there's now this approach to try and have a theory of change and to set out a whole series of data that you can use to measure whether you're achieving your goals and if you're not achieving your goals then to go back to the drawing board and change the strategy and hopefully get close to achieving what social change you're trying to do.

SHAPIRO: And is that because some of the billionaire philanthropists who are engaged in this effort came from the world of business, where accountability is everything?

BISHOP: I think that's part of it. I think if you compare the charitable world with the business world, there's actually a lot more data, a lot more numbers, a lot more theories about what ought to happen. The charitable world is very opaque. It's not just at the levels individuals where we put our money in a tin or send a check off to the United Way. It's actually a lot of the big foundations, they don't really measure very well what the money is doing.

And so I think these new businesspeople, people like Bill Gates came in and said look, if I'm going to give all this money away, I want to make sure the money's going somewhere useful. And so they did start to measure in a much more systematic way than we've seen before, and it has changed how they've behaved.

So Bill Gates gave a lot of money initially to support the idea that if you had very small classrooms in public schools that that would improve the quality of education. And it turned out when they measured it that having a small class didn't really make a lot of difference. There were other reasons why some schools performed well, and some schools performed badly.

SHAPIRO: Aside from Bill Gates, who are sort of the figureheads of this movement?

BISHOP: Well, there's an organization in New York City called the Robin Hood Foundation, which is a bunch of hedge funders who have a very high-profile gala each year, but they have been very systematic in putting measurement, reporting back and so forth in their model. And they're behind some - they were big funders of something called the Harlem's Children Zone, which is a really comprehensive, you know, organization that takes kids from birth through their graduation from college and helps them, people who would never have a chance to get through and get a college degree.

And they really put data at the heart of their model, and that's now being scaled up by the Obama administration in different parts of the country.

SHAPIRO: We have an email here from somebody named Pablito(ph), who writes: Two things I worry about with increased data-driven giving: One, social service organizations will over-serve the populations that they know can demonstrate results to avoid looking like they're not doing enough for their donors; and two, funders have more power to determine what studies and ways that organizations on the ground measure their work.

Sometimes the metrics that funders think up don't capture the work organizations are doing, or worse, force organizations to measure things they are not doing to get the money. He writes: In theory, data-driven philanthropy is good idea, but the question of who decides what counts and why do those with the power decide to have that power, those questions still remain.

Do you have anything to - any response to those concerns?

BISHOP: Well, I think the questions are very good questions because, you know, I think a lot of people use data to cover their backs, to make - they're operating and giving money to causes that they don't really know whether they're doing the right thing or not, and so they say give us lots of data, and they feel reassured by that, even though it's not actually helpful to anybody.

So the sort of data or evidence-based philanthropy I'm interested in is the kind that has a theory of social change and that is really there to help everybody in the process figure out whether they're doing the best they can and if not to flag up the problems quickly and allow them to change strategy.

And, you know, I think what we're actually going to see is, as social media takes off more and more, than there's going to be more and more feedback loops, more and more information coming back, more and more sites, websites like Kiva or (unintelligible) that give a lot of information back to ordinary people as givers, as well, that create much more transparency in the system and hopefully don't allow just a few powerful people to set the agenda as to what data matters but allows a much more crowd-sourced data-gathering process.

SHAPIRO: So how can the principles that billionaires and hedge fund managers are putting into place apply to those of us who are donating significantly smaller amounts than the Bill Gates of the world?

BISHOP: Well partly it's down to our own responsibility. When I interviewed Bill Gates for "Philanthrocapitalism," he basically said - I asked him this question, and he said look, everybody has to pick an area that they really care about, and they should really focus on becoming an expert. However little you're giving, you should really know about your area.

And that then enables you to actually ask the right questions and to understand the data much better than if you're just constantly spreading your money around lots of causes that you don't really know a lot about. But secondly, you know, I do think more and more that the online charities are getting much better at feeding information back to you.

And so if you give money to Kiva, you lend money to small farmers in Africa, you're going to get a report back on how that money's been used. Likewise if you give money to classrooms through DonorsChoose, you're going to get a whole series of letters back from the kids in the classroom, plus photographs, probably, and all sorts of information about how the money was used from the teacher.

And so we're all going to be empowered in a much more, you know, visual and effective way than we've ever been before. So I actually think this is a very good moment for the rest of us starting to have some of the abilities that people with much more money have to get the kind of information we want and how our money is going to the right place.

SHAPIRO: That's Matthew Bishop. He's the Economist's U.S. business editor and New York bureau chief, co-author of the book "Philanthrocapitalism: How Giving Can Save the World." And he joined us by phone from London. Thanks, Matthew.

BISHOP: Thank you.

SHAPIRO: And we still have Ken Berger with us, who is president and CEO of Charity Navigator. And Ken, there's a question here from someone named Ron(ph), who writes: Can you measure if a program is helping to minimize a slide in the wrong direction by keeping or slowing down the speed at which the problem is getting worse? Not all problems can see positive results as much as a diminished worsening. Is that possible, Ken?

BERGER: Absolutely. I think that there can be measures in any direction: slowing things down, speeding things up, having an impact in a variety of ways. And again, I think that there is research on this in some areas, but sometimes a charity has to create a theory and a hypothesis, if you will, and then go and track it and see if indeed is it slowing down that impact or that problem.

You know, I just - if I could just say one other thing about an earlier comment, I think that we really are at a crossroads right now where there is - as this pressure for results from some big funders grows, there is a danger that jargon and he who does the best marketing is going to win rather than substance. And so, you know, the traditional measures of results have been counting heads.

So, for example, an employment program says 100 people got employment. That's counting heads. The real measure, the meaningful measure is what we are trying to get to, and that would be how many people are gainfully employed for a period of six months, for a period of a year? And so the challenge is for us to have meaningful measures rather than the danger of jargoneering.

And lastly, the public. The research shows that 90 percent of the public is very interested in finding out about the results of the work. But only about 2 percent actually go and get it, and there's a debate as to why that is. One theory, one argument is because that's the way giving always has been. It's based on emotion, and that's the way it'll always be.

The second notion, which I think that Matthew and I share among others, is that it's a rational choice, to a degree, in the sense that the vast majority of charities currently do not provide public data on their results in a meaningful way.

SHAPIRO: And if they make you work to get the data...

BERGER: Yes.

SHAPIRO: ...you have the time to...

BERGER: Right. And so we've got to move towards making that data available, and that's what a lot of us are working on.

SHAPIRO: Let's take another call from Carl(ph) in Nashville. Hi, Carl.

CARL: Hey. Thanks for having me on the show.

SHAPIRO: My pleasure.

CARL: It's taken me I don't know how many times to finally get on here.

SHAPIRO: Well, tell us about your giving.

CARL: I recently got involved with a organization called the Fender Music Foundation, and I got involved through them through a website called reverbnation.com, where I have songs posted. I'm a musician and songwriter. And there was a ad on the website talking about the Fender Music Foundation, and what they do is provide musical instruments to children who cannot afford them. And, you know, being a musician who baled hay to buy my first bass, I understand what that's like. So...

SHAPIRO: And how do you measure their effectiveness?

CARL: I just got involved with them within the last week. And before I signed up to do this, which I can opt out at any time if I choose, but the way that I - before I made a decision, I went on the - or clicked on the link that they provided to go to the Fender Music Foundation website and did some reading on it. And when I found out that they're, you know, using 100 percent of the proceeds towards their program and their goal, I was in right there, you know? I have...

SHAPIRO: Well, let's put the question to Ken, who has extensive experience with this. What kinds of questions should Carl be asking of a charity, like the Fender Foundation, to figure out whether it's the most effective use of his charitable donations?

BERGER: Well, the first is what is it that they want to change? What - and if the specific change is that we want to give musical instruments to kids that otherwise would not have those instruments, that's a very - you know, that's an immediate short-term outcome. But hopefully they would say to you that they want to do something to change the quality of life, to enrich the life of this child, and then here's how we're going to try to measure that over time.

You know, one of the things, though, that I will say to you is this is a movement that's in its early stages. There still is a lot of work to be done, and you're putting your finger on the arts, which, I would say, has the most distance to travel to get to those kind of measures. Same thing with advocacy groups. We're really trying to extend this to all types of charities, but I do think that there is certain intangibles there.

And as you go down this road, I think we're going to be learning the answers to some of these questions together as the charities experiment and explore with the different indicators - outcome indicators, measures that they're going to use to try to track their success.

SHAPIRO: Ken, thanks for the advice, and, Carl, thanks for the call.

CARL: Yeah. Can I say one more thing quick?

SHAPIRO: Just briefly.

CARL: Yeah. You know, I hear what he had to say, and basically that's what I'm going to do, is keep track of what their progress is. Like I said, I can opt out at any time.

SHAPIRO: Glad to hear it. Thanks. And you're listening to TALK OF THE NATION from NPR News.

Let's take another call. This is Heather(ph) in Soil(ph), Oklahoma. Hi, Heather.

HEATHER: Hi. I just wanted to say the social media aspect has played a huge part in my giving. Wild Heart Ranch is a local wildlife animal rescue and rehab, and when they get an animal in, the pictures immediately go up on Facebook.

SHAPIRO: Wow.

HEATHER: We can see the injury. If there's vet bills (unintelligible), I can donate directly to the vet. We can watch the progress of the animals as they heal, including the release, online. And they also put - there's a supplies list with different online vendors and what they need when. So I can go and buy the supplies that they need now...

SHAPIRO: So it's not just that you're funding a refuge. You're funding this medication or this cast for a broken wing or this rehabilitation of a specific turtle.

HEATHER: Absolutely. And they even crowd-source innovative solutions for some of the bizarre situations they get and different skill sets that their fans and supporters have.

SHAPIRO: Like what?

HEATHER: Well, there was a duck that had had its bill ripped off somehow, but was captured and brought in. And they don't make prosthetic duck bills, so everyone...

SHAPIRO: What did they do?

HEATHER: ...was crowdsourcing what options there were, and they ended up using one that was removed from a decoy duck...

(LAUGHTER)

SHAPIRO: Oh, my goodness.

HEATHER: ...and various - discussions on various epoxies and glues and that kind of stuff to give her an upper bill because she had a lower and her tongue and everything dysfunctional...

SHAPIRO: Wow. Well...

HEATHER: ...and, you know, took enclosures and all kinds of support.

SHAPIRO: Ken, this seems to go far beyond an expenditure spreadsheet.

BERGER: Yeah. Well - and, in fact - and, actually, crowdsourcing is being used in a number of ways in the whole towards measuring results as well where there's a group that we work with that crowd-sources donors and opinions on a charity and how they've experienced that charity.

There's another group that's crowdsourcing the opinions of experts on the - opinions about how a charity is performing as well in addition to the clients. So crowdsourcing - I agree. I think social media and that whole realm of crowdsourcing is going to be more and more helpful in getting data.

SHAPIRO: So, Ken, it's great if somebody can find a charity they believe in that has all this kind of information at their fingertips. But if they can't in the mean time, what's your advice to a donor or a perspective donor?

BERGER: I think that the key is in those cases where they don't have the publicly-available data, is to call them, to go to their website and to pointedly ask for that kind of information. What is the problem you're trying to solve, how do you measure to make sure you're getting there? Do you change based on if you're not meeting those measures? You know. So I think you have to directly ask that question and you should get a reasonable and thoughtful answer. If you get a dodge or it seems like, you know, more sales than content, then that should tell you something. But I think that common sense can go a long way here. You can come to websites like Charity Navigator because we're about the launch our tools for measuring the quality of how a charity reports on its results.

SHAPIRO: Right.

BERGER: And that - and we'll also be giving guides to people for even more information on that.

SHAPIRO: Thanks, Ken. Ken Berger is president and CEO of Charity Navigator. He joined us by phone from his home in Fair Lawn, New Jersey. Appreciate your being on the program.

BERGER: My pleasure.

SHAPIRO: Coming up, everything that makes the holidays a special time can also make them especially hard on people who suffered a loss. We've got advice for getting through it after the break. We want to hear from you too. Give us a call at 1-800-989-8255. I'm Ari Shapiro. It's TALK OF THE NATION from NPR News. Transcript provided by NPR, Copyright NPR.