Big Data, Small Data

"Big Data" has become the new darling of the IT world. Since May, 2011 Google references to it have jumped by a factor of five. "Data Mining" on the other hand has fallen by a factor of three since its peak in 2004.





The IT industry has a way of reinventing itself, and changing the language to suit. One could argue that these two topics are the same; the difference may be one of scale.

The challenge for non-profits with respect to modern views of data are on two ends of a long spectrum. On the one hand, is the opportunity to gain insights and take actions based on mining big data trends (Google.org Flu Trends data comes to mind). On the other hand, is what I'd call the challenge of small data, namely the lack of good data (input and impact measures come to mind) and the relatively small, siloed data that is not being shared. 

There are two sets of motivation problems at each end of this spectrum. On the big data side is how to motivate non-profits to look at the possibilities in mining what are external data, and as Gisli Olafsson recognizes, is often at the wrong "zoom level" to be actionable. On the small data side, what is the incentive to share data and develop common standards? For corporations, it's the expectation of accessing larger consumer markets. I doubt altruism alone will drive NGOs to share. There has to be a hunger and scarcity on the one hand (what drove the NetHope members to collaborate) or a large economic benefit (also a NetHope driver, through its member deals and discounts). Otherwise, I don't see non-profits sharing data in a meaningful, aggregated way.

For further thoughts about Big Data, see the Wikipedia entry on Big Data and the Gartner definition: "Big Data are high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization." That's a mouthful definition.

On the small data side, look at the case of Blackbaud's donor data products and how they have motivated many NGOs to share their most private data: about donors and their giving transactions. Chuck Longfield, Blackbaud's Chief Scientist solved the problem of "what's in it for me" by creating a huge economic benefit to sharing data: learning how my fundraising prowess stacks up against an aggregate of my peers and what I can therefore do differently.

Whether big or small data, motivation to share and use is the key. I suspect the small data issues need to be solved for NGO's before we will see the benefits of big data extending to our sector.

"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."

Vision and Values

For my annual NetHope keynote address this week, my focus was on the vision and values of NetHope. My thesis for this brief talk was that the guideposts for the future of technology for our nonprofit members is our founding values. In the midst of massive change, we can point to some major themes, but we cannot know the future; we can however apply our values to guide our decisions as we go forward.

At our Summit in 2007, one of my colleagues and friends, Jean-Louis, showed the 2006 "Did You Know?" video. It was an amazing show of technology impact facts and figures created by a high school teacher and his students in Colorado. The conclusion was that "shift happens." I thought it would be interesting to watch the update six years later. Look for the factoids in this video that impact you the most.

The astounding thing about this video is that most of the information is about the current state of the world and how it's changing now. If we connect some of the dots, I see three themes: the explosion of data, explosion of connections and explosion of applications. Let's look at each (for a copy of the presentation slide deck, click here.)

For the explosion of information, the slide in the video that jumped out for me is that 40 exabytes of information are being created this year (that's 40,000,000,000,000,000,000 characters, about the same as the number of stars in the universe). And it's doubling every two years.

A recent Cisco paper forecast that the number of connected mobile devices will exceed the population of 7 billion this year and will reach 10 billion by 2016. Mary Meeker, a leading technology analyst at Morgan Stanley, forecast that the number of mobile internet users will exceed desktop internet users in the next year. Mobile connections are exploding.

If we look at the app's store growth, we see that iPhone/iPad apps are approaching a million in just four years. And most of these cost a dollar.

In 2010 the IFRC and Accenture completed a survey of 120 National Societies. We calculated an ICT index of 18 factors versus the Human Development Index (HDI). The result was a direct relationship that mirrored the digital divide between the north and west, and the south. While the good news is that digital divide programs such as the one at IFRC are bridging the divide, the strong countries are gaining ICT capacity strength faster than the weaker counties. How will they keep pace?

If all this doesn't scare you, check your pulse! The good news is that where there is great change, I believe there is great opportunity. The key question is how will we make sense of all this for our organizations and what decisions do we need to make?

For us as NetHope members, to face the explosion it is important to return to our basic values. Why? Strategies and tactics may change, but values endure. Our fundamental principles are the anchors in the storm and provide a framework for thinking about the future. Let's look at each of our six values and some potential insights.

1) We believe technology matters; it has impact on our missions and work as non-profit organizations. We continue to be the organization at the intersection of technology and non-profit work. We are the glue, the translators, the advocates and provocateurs. We interpret the ICT benefits for the sector.

2) We believe that benefitting all benefits one. This is fundamental to our collaboration; the whole is indeed greater than the sum of the parts. Our brand of partnering means shifting from a "do then share" to a "share than do" mentality. That means approaching needs and projects with the question of how we can do this together rather than solo. That may seem like an extra step, but we believe it reduces the massive reinventing of the wheel that goes on within our own far-flung organizations let alone among us.

A further thought: Large organizations, such as my own, need to take a leadership position in collaborating as part of our give-back; where we may have the resources to go it alone, we must begin with sharing and working together. Why? To lead is to serve, and only the humble learn.

3) We believe in learning through collaboration. This is a variation on the ready-fire-aim approach that Peters and Waters identified as a key theme of excellence 30 years ago. Henry Mintzberg noted that this is fine as long as you get to fire again in a fire-aim-fire-aim sequence. The point here is that we gain a huge opportunity by learning from each other's pilots, successes and failures in parallel instead of a longer sequential process.

4) We believe in building for the Field. This means two important things: cultivating a greater sense of humility in our headquarters, and expect to learn from the far reaches of your organization and those you serve. The greatest opportunity for us may be a flipping of the pyramid where we learn more about technology that works from beneficiaries and consumers than from the traditional IT department. This is part and parcel to the Discover and Harvest approach I've advocated elsewhere.

5) We have a bias for action. We have always been a group of doers, with an impatience for the results that we know technology can deliver. Jim Collins reminds us of a fundamental law from biology is that when times are uncertain, smart organizations "vary like mad". Varying means running more pilot programs to increase the chances of a winner that can be taken to scale (and we must scale-up to have impact.) I believe Michael Schrage made a compelling case for prototyping as core competency. I would add to that and say agility--the ability to change quickly--is as important in a rapidly changing world.

6) We believe in trust above all else". Trust takes time and is based on the experience of working together. We should be proud of the faith and integrity we have developed with each other. We can depend on each other to face and overcome any obstacle the future throws at us.

I closed by recounting the scene of the Battle of Zama in Gladiator. Russell Crowe admonishes his fellow fighters to remember their training. "Whatever comes out of these gates," he says, "we got a better chance of survival if we work together."[1] By circling the group behind their shields, the larger force of archers and charioteers is fended off in a surprising reversal of history; the Carthaginians defeat the Romans, and Crowe wins the favor of the Coliseum crowd. The point of the story is that by banding together a small group can overcome larger obstacles. That is the power of collaboration.

Here are some questions to ponder:

1) How are you amplifying the voice of technology from beneficiaries?
2) How are you increasing ICT agility to adjust faster, better, cheaper?
3) How do we enable the beneficiary portfolio of applications?

I've talked about our values and I've made some comments about the future. One of the questions I was asked was based on the Wayne Gretsky question about "where the puck is going to be." The short answer I gave is that the puck is on the citizen side of the court and the hockey stick is the mobile phone. I believe that, but I also believe Mark Twain when he said, "I've seen a heap of trouble in my life, and most of it never came to pass". If we do things together we will succeed.

[1] For the video scene, see 6:07 - 7:11 in this trailer.

"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."

Collaborate or Perish

The following article was published in the Berlin Civil Society Center publication "Global Futures – How international civil society organisations can make a real difference," which was launched at the BCSC Global Perspectives meeting in October, 2012. It is published here for the NetHope Summit with BCSC permission.

Collaborate or Perish –
How working together with technology can change the non-profit sector
Edward G. Happ
Global CIO, International Federation of Red Cross and Red Crescent Societies
 
In the next ten years, smart organisations will get amazing work done, with new ways of delivering services, working with leaner processes, and more efficient tools; they will be agile enough to turn on a dime – all in the face of a more difficult economic climate. This will happen not as a result of some new flash of technology, but by banding together and pooling resources and talent in strong cross-sector collaborations based on a foundation of trust. How is this possible?

On the bright side, communities of people want to work together, especially if it’s for a common cause – it’s built into our DNA. On the dark side, collaboration is an unnatural act; it requires us to trust people we know little about and have no control over, especially if we believe we can do things faster and better on our own. This is the paradox of collaboration: it is something we want to do, and act to avoid.

As a result, we have not done a good enough job of collaborating, especially in the use of technology. As it is increasingly used to run all the services in our organisations, technology may be the microcosm for an organisation’s operations and execution savvy. Yet we continue to pursue the corporation path of the past two decades, moving to larger and more complex systems to run our businesses. We do this despite the 5:1 per person investment in technology that our corporate colleagues continue to make. I know of no board or senior management team who would approve a doubling of the Information Technology (IT) budget, let alone a five-fold increase.

This should drive us to partner and share technology more, especially for functions that really do not differentiate us. But for a variety of reasons, we haven’t done this. The barriers to collaboration include ‘Not-Invented-Here’ (NIH), abundance, and proximity. More funding is not the answer; in fact a large budget may be an obstacle to innovation and partnering. The recent history in the US housing boom provides the case of the ‘starter castle’ mindset and the need to protect and heat what is too large. For some of our organisations the ‘lights-on’ infrastructure has become the tail that wags the dog.

The lack of services collaboration, including IT, in our organisations is a call-to-arms. If we connect the dots among the evidence, there is a looming train wreck on the horizon for IT and NGOs. Large NGOs are pursuing a corporate IT path that they cannot afford or sustain. And as change gets harder and more expensive, it will ossify these organisations and become the likely targets for cuts as a post-recession downturn takes hold.

There are a number of cases to illustrate this. The donor management upgrade project at a leading NGO is a case in point: 50 per cent behind in time, with almost as much in cost overruns. What happens if this NGO is hit with a 30 per cent cost reduction in headquarters like one of its sister NGOs? The time and the cost of the change, plus the operating and maintenance costs are too high to begin with; now they become a prime target for cutting losses. It will take another three years to change to something smaller in scale and more sustainable or, worse, to restart and complete the project at a later date.

Collaboration is often driven by a scarcity of resources, a shared need, and the desire to band together as a social group. These were certainly factors that have pushed 35 of the largest international non-profit Chief Information Officers (CIOs) to join NetHope and work together on initiatives. We’ve built a model of trust and collaboration over the past decade. But we’ve only just begun.

What do we need to do to succeed? First and foremost, we need what I call ‘headquarters humility’, the openness to solutions coming from the far reaches of our organisation and others. The best answers may in fact come from the poorest countries, from people we least expect. We need to discover and harvest the best of what is happening in the field. Second, we need to shift from a ‘do then share’ to a ‘share then do’ mentality. We need to look first to how we can partner to meet a need, instead of developing me-too solutions and sharing the war stories later. Sharing stories may be essential to starting a collaboration, but there must be a shift to doing things together from the outset. Finally, the larger organisations among us, who have the resources to go it alone, need to take a leadership position on collaboration. This is part of our give-back to the non-profit community. Like a good manager who learns to accomplish goals through others, we need to get the business of non-profit services done through and with each other.

Abstract from the forthcoming book – Collaborate or Perish: How working together with technology can change the non-profit sector, © 2011, 2012 Edward G. Happ.  Advance copies of select chapters are available on Blogspot.  
 

Futures

Today was departure day. All the students packing up, checking out, heading to the buses for the airport.



Many of the students included a "future" slide in their presentations. They spoke about their plans for their product, next steps for their business aspirations, and where they wanted to take their ideas. All were full of hope and optimism.

The question is how many teams will be around next year? We heard from a number of former contestants who had turned their project into a viable business over the past ten years. No doubt there are more. And many could benefit humanitarian work and help change the world. Yet it was clear that great ideas take incredible persistence to become great businesses.

We learned through Steve Jobs that insanely great products are all important. It was something our business schools overlooked. But the adage that great products without marketing and sales are great shelf-ware is also true, as is under-funding will kill the product. The smart teams had a marketing person on their team, and the best ones were high-energy sales people. And they had done their homework on the financials.

One of the signs about Imagine Cup posted in the conference center said "No DREAM too BIG".



One of the hallmarks of Microsoft is to think big. Passing this on to students is a natural. I heard a quip once that set "all things come to he who waits... as long as you work like hell in the meantime." As others have said, the competition is the beginning of a journey, not the end. I look forward to seeing what next year brings. See you in St. Petersburg!




"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."

Teamwork

Today we saw the final six teams compete for the software design awards at Imagine Cup. These were the best of the best, cool technology and well rehearsed presentations. Today, all of the demo's worked.

During one presentation, the speaker blanked on one of the slides. It was an awkward moment of silence. Then something special happened. One of his teammates asked him a leading question. The speaker replied and was back on track.

This was a simple and brilliant move. It's a variation on improv theatre, where one actor plays on another's line and extends it in a way that appears seamless.

It's also a great example of teamwork. No member of a team has a solo part. We are all completing each other's sentences.


"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."

Juggling the Demo

During a break between demos at Imagine Cup, I stepped outside for some fresh air. On the promenade, a crowd had gathered around a juggler. I stepped up to the circle and watched him toss an assortment of knives and one-liners. At one point he dropped a knife, and immediately shifted to some on-the-ground "juggling", which he did slowly, knives on the pavement, so we would not miss a turn. "I don't need your pity," he barked with a cheshire grin!

Of the nine teams I saw at the Imagine Cup, the demo's for more than half failed at some point. The angle of the Kinect box was off for one, a battery pack had drained for another, a laptop failed, an Internet connection was painfully slow, a slide did not build with its image, and so on.

There are a few lessons from our juggler that apply to the software demo. To start, practice, practice, practice. The juggler made sure we knew he had been at his craft for twenty years. He had obviously tossed these knives a few times before.

Plan on something failing. As Murphy's Law states "anything that can go wrong, will go wrong." A knife will fall. How will you handle that? (Pun intended :-)

Have a "b-plan". When your first plan fails, what will you do instead? Do you have an on-the-ground demo you can substitute?

Last but not least, use some humor! Nothing cuts the stress of the moment like some levity.

I received an update flag on my iPad today for the Yelp app I have installed. The explanation was priceless: "Fixed a bug causing the iPad app to crash for Italian users. No bosons were harmed during this collision." No less a software design judge, instead of being annoyed, I was entertained.

"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."

Anticipation

Anticipation, [an-tis-uh-pey-sh uh n], noun
2. realisation in advance; foretaste.
3. expectation or hope.

Sitting in Starbucks sipping a reminder from the US, I watch the contestants walking by on the harbour promenade. They are on their way from the hotel to the convention center. The first round of the Imagine Cup competition begins soon and will run into the evening.  There is an urgency in their stride.

I see the Italian team in the coffee shop and say hello. They are nervous, and eye my badge. I ask about their universities.  They are from three cities, scattered across Italy.  I won't be seeing them today.  They are relieved.

We were told yesterday that the teams have been preparing for weeks, getting their presentations down to a rhythm, anticipating the questions we will ask.  Anticipating has a double edge.  As Pasteur reminded us, "chance favours the prepared mind".  Like the athletes in the summer games soon to start in London, there are months of conditioning and rehearsal.  A colleague challenges leaders to imagine the end of the movie, how this will play out.   Anticipating the ending, is rattled by the anticipation in the stomach for it to begin. Many of the teams take a deep breath then start.

The imagination that began the idea that got them here, is also needed for imagining what the audience will hear.  That is beyond the judges to the customers and users of their solutions and products.   Can they imagine the satisfaction, the "ah hah" of their audience.  

This is often an unnatural act for technology wizards, for whom the solution is "obvious". They know the end of the story.  But it is those who can tell the story in the words of their listeners, who will bring their audience along.  They will win the day.   

Tonight we selected 20 students from 72 amazing entries for software design, that in turn beat out almost 400,000 ideas from an equal number of applicants who registered on the site a year ago. The teams in the Sydney auditorium were aching with anticipation. Those that told the good stories of why, how and for how much, anticiapated what their audience needed to hear. Their solutions were, by and large, the best.


"The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated."