Posts Tagged ‘metrics’

The Measure of Metrics

Tuesday, July 28th, 2009

The basis of socialmarkets’ brand of social capital is the metrics we employ to approximate social value.  For starters, we try to keep up with the social science on non-profit outcomes and their social value (see our research section here.)

The database of such values is large and growing, and we are grateful for it.  Every study that contributes data on the social return of a meal served to the hungry, on medical treatment to the uninsured, etc. helps us more accurately produce SROIs to associate with the related project listings on our site.

But the database of outcomes and their social values is not nearly comprehensive.  This would prevent us from compiling SROIs for many non-profits, if we didn’t have the help of our crowdsourcing mechanism.

One of the most important functions of socialmarkets is capturing our users’ opinions on which outcomes they value most.  For better or worse, opinions are never in short supply, so our own database of user input can cover the entire non-profit space.

Ideally we would have both subjective and objective data available for the SROI calculation of a given project, but we know this won’t always be the case.  In any case, the amount of available data is significant.  Like in any market, size matters in socialmarkets.  The more players in the market, the more likely we can mitigate gaming, bias and other distortions that prey upon markets with few participants.

The wisdom of crowds presents another argument for more crowdsourcing data. Consider a classic guessing game, like when a random crowd is asked to guess the number of jelly beans in a jar, or the weight of a prize bull.  It turns out that while the average guess is off by a wide margin, the average of all guesses becomes more accurate as the number of guesses grows.  The bigger the crowd, the wiser they seem to be.   This is a fascinating emergent property of society, and it certainly applies to socialmarkets - at least to the extent we can reconcile votes on social priorities with guesses at bean counts.

The bottom line is that socialmarkets runs on social metrics, and is both producer and consumer of the data it needs.  Using the data generated by our own users, and by others doing related social science, we are piecing together a truer picture of the costs and benefits of social projects.  Ultimately, this will enable us to see how our social investments can generate the greatest social returns.

The Art of Science

Monday, March 31st, 2008

A recent post thread at Tactical Philanthropy on the topic of measuring nonprofit effectiveness caught my attention. It discusses the issues related to recent (and possibly over-enthusiastic) efforts to apply scientific measurement techniques to help evaluate the work of nonprofits. I could not resisting throwing in my $0.02, which is reproduced below (and can also be found in the originating post here):

It seems to me a recurring theme of the ‘metrics mania’ debate is that it involves both art and science. If the ‘art’ position is that non-profit value defies objective analysis, and the ’science’ is that non-profit worth can be reduced to equations of input and output, I expect I’m somewhere in the middle.

This is surprising, since in the interest of full disclosure, I should say that I am co-founder of socialmarkets.org, where donors “invest” in nonprofit projects based on their SROI (Social Return On Investment.) This sounds like pretty hard science, but there is actually quite a lot of room for art to soften the edges.

The stock market analogy already seen in this thread is spot on. Apple’s stock price is influenced rather than defined by the financial science that slices and dices its cash flows. The beauty of a market is the marvelous job it does boiling down a large, complex set of valuation inputs into a single output called “price”. This number is useful on both an absolute scale and relative to other offerings in the market.

In a testament to the wisdom (or lunacy) of crowds, Apple’s stock price reflects the collective opinion of financial analysts, status-conscious teens and everyone in between. The potential to harness the same power to “price” The Red Cross or your local community foundation seems both possible and useful to me.

There are plenty of donors looking for a “best-bang-for-the-buck” (i.e. maximum SROI) approach to non-profit investment, and right now there is not much useful data out there for them. The success of sites like Charity Navigator are a testament to the need for metrics, but they only tell potential donors about what nonprofits spend, rather than what they accomplish. Surely we can do better than that.

Those with a less scientific approach may not find metrics like SROI as compelling, but still potentially useful. Consider Albert Ruesga’s story of the high-risk, low-return homelessness project that he presents as an argument against metrics. This is where the difference between the sectors becomes significant.

For starters, unlike the for-profit model, there are often donors willing to invest in hard-luck nonprofit cases - as they ultimately did in Mr. Ruesga’s example. More importantly, since nonprofit metrics is still a new field, we can - and should - redefine the notion of return to more accurately capture the total social value being added. That seems to be the most constructive cross-product of art and science in this space: a more artistic approach to the science of metrics.

Standards and Deviations

Tuesday, February 19th, 2008

We spend so much time promoting the outcomes-based performance measurement model for the non-profit sector, that we sometimes forget one of it’s most fundamental problems: it does not currently offer a generally accepted standard of outcomes. It is often challenge enough to convince the relevant donors and/or charities of the value of outcomes measurement, but even after winning that challenge we are quickly confronted with another: which are the “correct” outcomes to measure?

Consider the market of charities working for affordable housing, particularly critical here in NYC where real estate prices stubbornly defy reports of credit crunches and imminent recession. “Number of available housing units” is an obvious metric of quantity, but what about issues of quality? Where does safety fit into the equation? What about measuring subjective outcomes like quality of life? We critically need at least an initial set of outcomes that considers all these questions - and fortunately we have one very promising source for just that: The Center for What Works (CWW).

CWW states their mission as:

“improving social sector results through benchmarking for nonprofits, a strategy to facilitate outcomes measurement and reporting by linking nonprofits and funders through a common language.”

This kind of talk is music to socialmarket’s ears. The notion of a common language is particularly compelling, since it would be tragic to see the coalition of support for outcomes measurement fall apart for lack of clarity on implementation. CWW (along with the Urban Institute) has created an impressively coherent road map for fourteen non-profit markets (including affordable housing) as well as a generic taxonomy to implement those not yet covered. I encourage you to click the link above (or here) to look for yourself - and see if you agree.

With enough momentum behind them, we envision CWW becoming the de-facto standard for performance benchmarking across the nonprofit sector. We think that their outcomes taxonomies are a sensational way to move nonprofit outcomes measurement forward in a highly practical - and measured - manner.


Clicky Web Analytics

google