Big Data: When Is It Too Much?
When is Big Data Too Much?
First example: too much data can be overwhelming. “It’s not our job to go down to the lowest level of data, but to know how to aggregate outcomes so it can be converted into an insightful report,” – James Miln, Senior Finance Director, Global Operations Finance, Yahoo!
Ask yourself: are you already using the data you routinely capture in your planning activities? Are you thirsting for new sources of data, having exhausted or marked as irrelevant for planning the data already readily available? If not, then you’re probably already overwhelmed by the data you have, and need to consider why that is, before seeking out new forms of data such as client footfall and social media presence. We often work with clients who, when we ask them what they use their huge volumes of data for respond: “We like to have it there, just in case”. That data costs money to store and money to retrieve, so if you are already overwhelmed by too much data then review the situation before you get into the ‘cool’ stuff (that you read about in blogs like these).
Second example: data is not information. Do you really have the appetite and capabilities to turn the available Big Data in to Big Information?
Simply put: have you genuinely started identifying which questions need to be answered? If not, you aren't ready to approach the subject of Big Data: it will be too much. Let your imagination run riot: you will be able to get the data you need, once you identify what questions you need to answer. But don't start from the point of asking what is available. Ask: what questions do I need to answer to make my organisation more successful?
Consider these four questions:
Do you genuinely know what drives your business forward? Before approaching Big Data you need to intimately understand the fundamental drivers and metrics of your business. Only then can you assess new, less structured sources of data.
Do you know what your customers care about most? Why do customers choose your business’s products and services? How are those customers acquired, and what makes them come back for more? Why track Facebook data unless you think that it provides you information about the behaviour of your customers in the context of your knowledge of what makes the act.
Are you really open-minded about taking data from different sources? Are you more concerned about checking every fact or about nimbly sifting data and allowing the laws of large numbers to filter put the silt?
Are you comfortable about getting results that don't always provide clear-cut decisions? Some of this data won't be reconcilable (you can’t regenerate it after you take the sample: it is gone) and sometimes it will be conflicting. This is a totally different ball-game from extrapolating the most recent trial balance and by-product sales data, each of which reconcile back to trusted in-house systems.
In summary Big Data is too much when it is not yet in the mind-set of the people creating planning data. Is it a bit of a slightly different fad, like for ZBB, The Balanced Scorecard and Chartism of yesteryear: fashionable, sensible as far as it goes, but little more than a talking point for most people and a slick sales-pitch for others?
But to take a positive stance: can you afford not to turn this Big Data into Big Information when your competitors are making that investment?