By Kim Ammann Howard

Flying back from the Grantmakers for Effective Organization’s National Conference, I reflected on the conversations I had about how to support nonprofits to enhance their effectiveness. I was particularly struck by the last session, “Can’t We All Play Nice: Reducing Government Barriers to Working with Nonprofits,” which highlighted the disproportionate amount of nonprofit support from the public sector as compared to philanthropy (about 3:1). While government agencies provide a large percentage of support for nonprofits that address pressing social issues, they often lack the appropriate knowledge, experience or capacity to support nonprofits as effectively as desired. The session presenter acknowledged that funders often hesitate to provide financial support to government agencies (for many valid reasons), but challenged the audience to consider ways that foundations and government agencies can work together to address similar social issues.

Initially, I thought of “big P” partnerships—highly visible public-private initiatives that focus on ambitious, long-term goals. However, thinking back on my own experience, I also remembered “small p” partnerships—behind the scenes partnerships between funders and government that fill small but critical needs that enhance government’s capacity to support nonprofits. These partnerships often take on issues that are unlikely to be addressed solely by government (or at all). Most importantly, these partnerships take advantage of foundations’ expertise, staffing and grantmaking agility. Examples of these successful partnerships include foundations supporting government agencies to:

• Conduct targeted research about evidence-based practices
• Analyze the pros and cons of currently debated policy options
• Build an infrastructure to connect and share multiple types of data to make the case for and achieve nonprofit goals (e.g., interactive public databases, geo-mapping)
• Hold trainings to translate nonprofit experience and learnings from foundation staff to public sector staff
• Augment human capital in the public sector by supporting the temporary assignment of foundation staff to public sector positions

Given the big role that government plays in the nonprofit sector, I think it is time to seriously consider how foundations can support “small p” partnerships, which enhance the capacity of the government sector. While the risks of such investments are high, the rewards and impacts may be too.

By Ria Sengupta Bhatt

For those of us immersed in the evaluation field, we know that a well-executed evaluation can do much more than serve as a method of accountability or fulfill a requirement. Especially in the field of philanthropy, a comprehensive evaluation can facilitate continuous learning within foundations and the field at large, increase the effectiveness of a grantmaking strategy, and increase innovation. But not all grantmakers fully embrace this approach. GEO’s most recent national study of philanthropic practice found that while 70% of grantmakers engage in evaluation, most are still “focused on proof and accountability rather than learning with their peers inside and outside of their organization.”

This trend seems to be changing, however. I recently attended GEO’s National Conference in Seattle, Washington, which reinforced how evaluation can help foster collaboration among grantmakers. For example, one session focused on the idea that large-scale, long-term social change can be achieved more effectively through collective impact. The group discussed how shared measurement systems are an essential component of collaboration, as it creates agreement around what constitutes success. At another session focused on incorporating evaluation into philanthropy, an attendee pondered why more grantmakers don’t attempt to coordinate reporting requirements with other grantmakers to reduce reporting burden, particularly when they are trying to achieve the same results.

The intersection of the themes of evaluation and collaboration among funders shows yet another way that evaluation can be useful for purposes beyond accountability. While the results of GEO’s survey show that there is room for growth in the use of evaluation in philanthropy, I have seen a perceptible shift in this type of thinking in my work. I am further encouraged by the conversations that took place at the conference that suggest this shift is occurring across various types of grantmakers. The more often grantmakers get together to discuss evaluation, the faster the field of philanthropy will be able to take advantage of its many uses and strengths.

By Gagandeep Kaur & Regina Sheridan

The corporate world is notorious for collecting metrics to guide their business strategy. This is evident by executives’ fixation on cost-benefit analyses, return on investment reports and performance measurements. However, this rigor often seems to be absent when it comes to corporate giving programs.

A report by the Global Reporting Initiative, the University of Hong Kong and CSR Asia notes that corporations often track company outputs (e.g., money granted, hours volunteered by employees, products donated) and rely on anecdotal stories and individual testimonials to describe the impact of their programs on communities. In other words, many fall short when determining the actual effectiveness and impact of their social investments.

This lack of measurement rigor may be due to the fact that social giving makes up such a small portion of most corporations’ overall budgets. According to a 2010 Corporate Giving Standard survey, 184 corporations gave away a total of $15.5 billion in cash and products. While this is no small chunk of change, it only makes up 0.11% of these corporations’ revenue. Or perhaps it is because corporate giving programs are often designed to benefit the corporation (e.g., improving the corporation’s reputation, increasing employees’ job satisfaction), as much as it is designed to benefit the community.

Whatever the reason, we would argue that corporate giving programs could and should be more strategic, standardized, and transparent in their measurement and reporting. A good place to start is by using a standardized framework, such as those offered by the Global Reporting Initiative or the London Benchmarking Group. To get the “biggest bang for their buck” in community giving programs, corporations need to go beyond measuring their own performance and increasingly focus on measuring the value and impact of their investments on the communities they serve.

By Kim Ammann Howard

While the importance of networks to advance social change seems well accepted, how to assess their effectiveness is not. In a recent webinar I held on cultivating networks with Claire Reinelt from the Leadership Learning Community and Melanie Moore from See Change, many participants asked questions about measurement. How do we know if the networks have an impact? What is the best way to measure networks? How do we know how much the network contributes to desired outcomes relative to grants, technical assistance or other supports? Network mapping and network assessments are some of the promising new tools that can help us answer these evaluation questions and also serve an equally important purpose of informing network implementation.

Evaluating networks may seem relatively new, but it is important not to overlook the application of good, familiar evaluation practices. Network evaluation questions that focus on “who, what, where, when, why and how” (e.g., how is the network used to reach desired goals, to what extent are they being realized) still benefit from clearly articulated theories of change, corresponding evaluation questions and traditional data collection methods.

Too often the complex nature of networks provokes the search for a new and equally complex evaluation strategy, which can yield data with limited utility (e.g., inability to separate the contribution of network impact from other supports, lack of information about the processes and context). These limitations commonly lead observers to question whether evaluation resources are used effectively, which can negatively impact future considerations about evaluation. As we continue to move forward with the development of new and exciting evaluation tools, let us remember to not throw out the baby with the bath water.

By Kris Helé

My college basketball brackets have busted. I’m disgusted and embarrassed. Why is it that people who select their brackets on the basis of uniform colors or mascots tend to do just as well or better than those who spend hours poring over stats or those who have at least a basic understanding of sports?

“They call it March Madness for a reason,” my friend tells me. This further agitates my analytical mind, awash in anxiety and grief over the unpredictability of it all.

As I lamented my risky choice for the champion (c’mon Baylor, what happened?), I got to thinking about March Madness as a metaphor for the nonprofit sector and the choices faced by philanthropy in placing their investments. Sometimes the big name teams succeed, just as you thought they would (Kentucky is on its way). Sometimes the underdogs, the teams that make you ask, “Is that even a college?”–come out on top (congrats on your win, Murray State). Sometimes last year’s high performer is this year’s disappointment (better luck next time, UConn).

In the nonprofit sector, the stakes are much higher and there are far more than 68 entities vying for the resources of any given funder. How can nonprofits increase their value and effectiveness? Likewise, how can foundations make smarter bets on nonprofits? Investing in evaluation is a good start. When evaluation provides high quality data to inform change and improvement, nonprofits are further along their path to success, and foundations are able to make more effective investments.

Evaluation isn’t without its flaws, and it can’t predict the future, but it certainly brings method to the madness.

By Sheila Wilcox

I was reminded recently, when my project team switched the response scale for a survey question from an agreement scale to a frequency scale that, too often evaluators focus on developing survey questions while giving less attention to response options. When agreement scales are used as the fail safe response option, the result can be high-quality questions without the most meaningful answers to those questions.

Agreement scales are extremely common in the evaluation world, to the point of overuse. To be fair, evaluators cannot be completely blamed. While well-documented standards for writing survey items and how to design a survey exist, there are far fewer and less established standards for writing response options.

Rather than defaulting to an agreement scale, it is important to match survey response options to the information that is most important to know. Is it that participants agree with something, or is it frequency, importance or some other construct?

There is also a time and place for agreement scales, such as measuring attitudinal constructs. In addition, a Stanford professor’s recent research on construct-specific response options highlights that agreement scales can produce poorer quality data due to acquiescence bias. Respondents agree with statements for a number of reasons, which for most is not because they actually agree with the statement. A survey should make it easy for people to determine how to answer each question directly.

So whether you are in the role of creating surveys or commissioning surveys, before agreeing (or strongly agreeing) to an agreement scale, spend some time considering whether it helps you achieve your goals. Sometimes it will, but oftentimes it is not the best option.

By Ria Sengupta Bhatt

The topic of human rights protection often evokes images of poor and disadvantaged individuals and the activities conducted and laws passed to ensure that they are not mistreated. A less common image is that of evaluators collecting data from and about these very same individuals, yet this is also a real piece in the human rights protection puzzle.

Evaluations of grantmaking strategies and programs document the experiences of grantees and program constituents through surveys, interviews, focus groups and observations. As such, it is important to protect the information communicated through these interactions. But whose job is it to ensure the rights of study participants?

It is the responsibility of both the evaluator and the grantmaker.

Evaluators are accountable to their clients, their clients’ grantees and program constituents involved in their studies. As evaluators, we are required to take action to ensure that personal information and opinions are protected (sometimes under lock and key) and that sensitive data (like health status and contact information) is kept confidential. Foundation staff are also responsible for the rights of participants in the studies that they fund. This may seem obvious; however, the established standards for protection are not always clear.

BTW has created resource documents and a training toolkit to help foundation staff understand how study participants should be protected and how philanthropic organizations can navigate these issues. These materials are written specifically for foundation staff responsible for commissioning evaluation and research studies. They are intended to share the appropriate amount of information and detail without overwhelming funders and to serve as a reference when questions arise about the protection of study participants.

We hope you find these materials useful to you and your colleagues at foundations!