Social mobile and the missing metrics

Scenario 1: Five hundred people gather together for three days. They talk, they discuss, they share and they learn. And then they leave. Some stay in touch, others have picked up enough to start a project of their own. Others just leave with a satisfied curiosity, others with the odd new blog post behind them

Scenario 2: A charitable foundation funds the creation of a new mobile tool. Over a one year period there is software development, a new website, user testing and roll-out

Scenario 3: A university professor embarks on a piece of field-based research to examine the impact of a mobile-based health initiative in Africa. He or she writes a paper, highlights what did and didn’t work, gets it published and presents it at a conference

Question: What do these three scenarios have in common?
Answer: It’s unlikely we’ll ever know their full, or real, impact

Let’s assume, for one moment, that everyone working in social mobile wants to see their work have real, tangible impact on the ground. That would equate to:

  • A patient receiving health information through their phone which can be directly attributed to improving their health, or their likelihood of staying alive
  • A farmer receiving agricultural information which can be directly attributed to better family nutrition, or an increase in income or standard of living
  • A team of human rights activist reporting violations which can be directly attributed to the fall of an evil regime, or the passing of new legislation, or the saving of a specific person’s life
  • And so on…

Fine. But are things ever this clear cut? Ever this black or white?

The social mobile world is full of anecdotes. Qualitative data on how certain services in certain places have been used to apparent great effect by end-users. But what we so often lack is the quantitive data which donors and critics clamour for. You know – real numbers. Take the 2007 Nigerian Presidential elections, an event close to my own heart because of the role of FrontlineSMS. This year – 2010 – will witness another election in Nigeria. What was the lasting impact of the 2007 mobile election monitoring project? Will things be done any differently this year because of it? Did it have any long-term impact on behaviour, or anti-corruption efforts?

Much of the data we have on FrontlineSMS falls into the anecdotal and qualitative categories. Like many – maybe most – mobile-based projects, we have a lot of work to do in determining the very real, on-the-ground impact of our technology on individuals. We regularly write and talk about these challenges. But it’s not just about having the funding or the time to do it. It’s figuring out how we measure it.

If a farmer increases his income through a FrontlineSMS-powered agriculture initiative, for example, but then spends that extra money on beer, that’s hardly a positive outcome. But it is if he passes it to his wife who then uses it to send their third or fourth daughter to school. How on earth do we track this, make sense of it, monitor it, measure it, or even decide how we do all of these things? Do we even need bother at all?

Of course, as my recent Tweet suggests, we shouldn’t get too obsessed with the data. But it’s important that we don’t forget it altogether, either. We need to recognise the scale of the challenge – not just us as software developers or innovators, but also the mobile conference or workshop organiser, and the professor, both of whom need to face up to exactly the same set of questions. The case of the missing metrics applies just as much to one as it does to the others, and we all need to be part of finding the answer.

Talking to the trees?

The UK’s Guardian newspaper ran an interesting photo gallery last week showing mobile phone masts ‘dressed’ up as trees. For a while it seemed these might catch on as increasing numbers of people complained about the appearance of ‘ugly’ metal masts in their neighbourhoods. While inner-city masts can be hidden, in the country there are fewer options. Disguising them as trees is a favourite.

More images and descriptions on the Guardian website gallery here.

When it’s reasonable to be Unreasonable

One of the most exciting things about my work is the incredibly talented people I get to meet. Up-and-coming entrepreneurs with a never-say-die, get-up-and-go attitude. I’m continually inspired and more than happy to offer my help in any way I can, particularly to those looking to implement FrontlineSMS one way or another in their work. Today, two of these projects have made it through to the next round of a major competition and are looking for your help. I hope you feel equally inspired to show your support – it’s only a few mouse clicks away and costs little.

What’s it all about?

The Unreasonable Institute unites up to 25 high-impact social entrepreneurs from around the world, who attend an intensive 10-week summer institute. There, entrepreneurs incubate their ventures with rigorous skills training and expert mentorship. At the end of the ten weeks, the Institute connects the Fellows with start-up capital and a global network of support. In short, the opportunity provided by the Institute will make a significant contribution to the growth of these early-stage initiatives.

Finalist #1: FrontlineSMS:Credit

FrontlineSMS:Credit, run by Ben Lyon, is one of the finalists. FrontlineSMS:Credit aims to make every formal financial service available to the entrepreneurial poor in 160 characters or less. By meshing the functionality of FrontlineSMS with local mobile payment systems, implementing institutions will be able to provide a full range of customizable services, from savings and credit to insurance and payroll. Ben is currently in Sierra Leone testing his system, and signing up local organisations for early pilots. Visit Ben’s page on the competition website and support his pioneering work!

Finalist #2: Light Up Malawi

Light Up Malawi – run byRaina Kumra – is a venture which aims to bring 100% alternative energy to Malawi through policy reform and the creation of a sanctioned pilot program for all manufacturers of solar, biomass, and wind energy products. In a sign that collaboration is alive and well even among competition finalists, one of the key systems they plan to deploy is FrontlineSMS:Credit, which will be used to assist with the setting up of rural distribution programs, and allow for the creation of finance systems for village entrepreneus. Please show your support for Raina’s work  by visiting her competition page here.

Both projects need pledges of just $10 from 650 individuals. The first 25 projects from the 37 finalists to raise $6,500 will secure their place on the summer program. Please take a moment to show your support. You pledge now and pay later, and it only takes a minute. Thank you!

(You can read more about how the Unreasonable Marketplace works here).