|
|
 |
 |
Philanthropic Research Sample
Much has been said about the New Philanthropy and it's hands on approach to giving. The Old Philanthropy wrote checks, staying away from the day-to-day 'trench' of social problems. The New Philanthropy asks, 'What is the impact of my donation?' This is a big change from the foundations spreading money around relative spots of perceived social needs. "If foundations serve only as passive middlemen, as mere conduits for giving, then they fall far short of their potential and of society's expectations of them," writes Harvard Business School professor Michael Porter, a strategy expert. As we shift gears, the needs of foundations change.
The majority of funding agencies start with a big heart. Indubitably, this is an excellent place to begin. Reaching out, they look for the best program match for their mission. The programs they find often have excellent ideas and opportunities to help, but somehow they fall short of their ultimate fulfillment. Foundations, donors and grant-makers often assume these issues have been resolved, and the program is a finely tuned organization, with a clear mission and outcome measures. While passion for a cause is important, actually meeting the needs of those we most want to help is as important, if not more, as if our beneficiaries never see their intended help, we're wasting our money, time and everyone's hopes.
Porter and Mark Kramer, a founder of the Center for Effective Philanthropy drew attention to the need for philanthropic performance measurement in the December 1999 Harvard Business Review,
'The goal is superior performance in a chosen area. A foundation should measure its success by the performance of the organizations that it funds. This view is not widely held today. However, because grant-making foundations can create value only through others, they must accept responsibility for the success or failure of their grantees.'
As the New Philanthropy grows, it's interest shifts towards examining each potential causes like business opportunities, seeking maximum return in terms of social impact. Mario Marino of the Marino Institute framed the new perspective well; 'The same forces transforming the private sector can be harnessed to drive home real reform'and results'in the nonprofit sector, especially where its role is growing most: providing social services. We can leverage the lessons of the New Economy to build a New Philanthropy.'
However, as Bruce Sievers points out in the November/December 2000 issue of Philanthropy, 'The ultimate goal and success of a business investor is the level of profit. But what does this mean when translated in to the social world' There is plenty of talk about 'social metrics' 'seeking to create a proxy for profit. Nonprofit activity has a complex and often intangible range of aims'.
When the new philanthropy joins an organization at any phase of its development, they're often faced with a confusing mix of questions and problems. Most of these questions have to do with organization, implementation of the program and its impact on the people involved.
| • |
What are the goals of the organization? |
| • |
What material and human resources will be required to fulfill those goals? |
| • |
What should the structure of the program be? |
| • |
Who can we benefit the most? |
| • |
What selection tools will be used for participation in the program? |
| • |
How many people will have to be staffed' Should there be volunteers? |
| • |
How should they be trained? |
| • |
What criteria should be used to judge program performance? |
How many organizations systematically ask themselves these questions before they reach out to help others' This article aims to address the gap between the perspectives above by taking the foundations needs and the non-profits hopes from the perspective of evaluation research.
What is Evaluation research?
Evaluation research can help foundations; donors and grant-makers make clear decisions about funding, as well offering the programs themselves an opportunity to succeed where just more money won't help. Evaluation research is systematic procedure for gathering, documenting, and analyzing five basic aspects of a program or policy:
Example: Hoop Dreams
An example of an excellent program that a major foundation was supporting was Hoop Dreams. Hoop Dreams hoped to:
| • |
Create an opportunity for inner-city high school students to excel in school with the long-term goal of getting into a university. |
| • |
Create an avenue for inner-city youth and working professionals to share learning opportunities and Shift some stereotypes of either side. |
| • |
Create a scholarship fund to provide financial aid for the participating students. |
|
|
We'll return to our example as we walk through the full process of evaluation research.
1) Needs Assessment
Needs Assessment is research designed to determine the nature and extent of the need a program or policy is expected to satisfy.
A needs assessment is used to:
| • |
Understand the need for intervention |
| • |
Clarify the objectives of an intervention |
| • |
Establish the priorities among the objectives |
| • |
Establish a baseline for future process or outcome evaluation |
| No needs assessment was conducted when the main driver of Hoop Dreams created program. Thus, there were vague ideas about outcome measures, and the best ways to achieve goals. |
|
2) Formative Evaluation
Formative Evaluation is research designed to first to define the program goals, then to assist with the conceptualization and design of the program. Importantly, formative research helps to resolve intervention and evaluation issues before a program is widely implemented.
Some questions that drive the formative evaluation process are:
| • |
What options are currently available to resolve the problem? |
| • |
What works/doesn't work with these programs? What could be applied to the current problem (avoiding the reinvention of the wheel)? |
| • |
How should the program be organized? |
| • |
How do different program options compare in terms of impact/cost? |
| • |
Are the outcomes being measured appropriately? |
| Hoop Dreams didn't perform a thorough literature review to capitalize on previous behavioral research on educational attainment and stereotype reduction. As these issues were not explored from the start, the program directors job was probably more trying, while not getting the results wanted. |
|
3) Process Evaluation
Process Evaluation is research conducted to evaluate the implementation and ongoing activities of a program.
Process evaluation is used to:
| • |
Discover potential problems |
| • |
Identify where the program needs improvement |
| • |
Describe and monitor program activities |
| • |
Provide management with information as to how the program is doing |
| • |
Assure quality and accuracy |
Hoop Dreams didn't give much thought to process evaluation until the foundation started asking about the process by which students were selected to receive funding. An independent evaluation researcher was contacted and immediately identified a host of issues through reviewing relevant literature, interviews with stakeholders, and focus groups. Among the issues:
| • |
A substantial behavioral research literature exists around educational, training and programmatic issues for inner-city high school seniors. The literature offered best practices, benchmarks and practices to be avoided. |
| • |
Educational research shared the best predictors of academic performance and matriculation, as well as approaches parents could take to foster positive educational environments for their students. |
| • |
Social Psychological research shared the best ways of dealing with stereotype reduction, as well as workshops, helping students and mentors to work together effectively. |
| • |
Program participation was based on subjective criteria, and as the program is growing to more than one high school, would make selecting students not only difficult, but also dangerous to the program goals, as the wrong students might be selected. |
|
|
4) Outcome Evaluation Research
Outcome Evaluation Research is research designed to assess the effectiveness of a program in achieving its goals and objectives, as well as the impact of the program.
Some questions that drive the outcome evaluation are:
| • |
Was the program effective in achieving its intended goals? |
| • |
Are the desirable outcomes attributable to the program? |
| • |
Are there unexpected outcomes, either positive or negative, attributable to the program? |
| • |
Can alternative processes that aren't covered by the program explain the results of the program? |
| • |
What are the costs to deliver the services and benefits to program participants? |
| • |
Was the program an efficient use of resources, compared to alternatives? |
While Hoop Dreams is just beginning to be evaluated, a host of beneficial actions have been taken:
| • |
Stakeholders have been interviewed and brought together to come to consensus as to project goals and objectives |
| • |
New selection criteria is being used now, with the standard being used to ensure valid, defensible and most importantly, predictive selections. |
| • |
A literature review has been performed, with standardized measures being offered to assess the progress of students, as well as measures of volunteer and student attitudes towards each other. |
| • |
Outcome measures have been identified and will be measured in a longitudinal manner. This will insure the program will best practices give the program its' best opportunity to succeed. |
|
|
Happy endings and new beginnings were ensured by Ellen Folts, Executive Director of the Leonsis Foundation in her careful implementation of the evaluation. She contacted an evaluation researcher, had opening meetings and information gathering sessions with both Hoop Dreams and Daniel Martin, the independent evaluator. This was crucial for the mutual success of the evaluation, as non-profits can be skeptical of funders trying to shift missions, and the assurance of independence helps all involved realize the effort is for the betterment of all parties, from the students, to the foundation.
| |
|
|
|
 |
|
 |
|
 |
|
