Do you remember when nobody was talking about Big Data? Or Bigger Data? It’s the focus of so many headlines that we’ve become almost desensitized to it.
At the outset of 2013 the tribe in my Friday #BIWisdom tweetchats on Twitter said their hope for how business intelligence would improve this year is that it would be “the year that Big Data settles down” and that it would be commonly known as just “data.” They said they’d like to see the attention being given to the size and mechanics of Big Data be redirected to leveraging business intelligence value from data.
Understandably, their attention immediately peaked when I shared the first glimmerings of responses from the earliest participants in our 2013 Wisdom of Crowds® Business Intelligence Market Study. Based on very preliminary data, the sentiment toward Big Data is trending down and will be lower than in 2012.
“It’s a big symptom that the hype curve has started sliding into the disillusionment valley,” a member tweeted.
Another tweeted: “Some current discussions on Big Data will settle down, but there are new aspects that are just starting up. There are big bucks to be made in derivative businesses, and the hype won’t go away until it’s replaced with the next catalyst.”
Someone else responded, “I’d love to say it’s true that the hype curve has ended, but quite honestly I doubt it. Scary as it is, I think it’s still on the way up, thanks to Big Data’s role in the presidential election and thanks to vendors promising the world.”
And as the #BIWisdom tribe tweeted last Friday, tragically we now can add to that the role of Big Data in finding the suspects in the Boston Marathon bombings. Interesting questions arose because of the impact of crowdsourcing (in social media) and data quality related to the suspects search.
“I think behavior can be predicted to a degree, but the problem is that you need a lot of variables, which is confounding,” tweeted one member. Another person added that “Automating that requires knowing what to exclude; so it’s a capability problem.”
Another tribe member stated that 55 percent of Big Data projects don’t get completed, due in large part to technical roadblocks. More tweeted input to the discussion: integration is central to Big Data but there are few integration tools today.
Another issue is that Big Data done right is difficult and costly. Many use cases are esoteric and not mainstream, which makes it difficult for vendors to develop a mainstream repeatable packaged solution.
The real challenge in Big Data is dealing with context, a tribe member said. That requires improving the signal-to-noise ratio in data. Although some technologies that focus on context exist today, the variety of contexts inherent in Big Data requires more technologies.
Their final thoughts? Big Data takes off where business intelligence left off, tweeted a tribe member. Another concluded that Big Data has lost meaning and relatability to business intelligence, and the business executives are frustrated in not knowing how to fix the dilemma.
Bottom line: Hype is a good agitator, but it’s empty if it doesn’t result in actionable insights. Hype must be replaced by clarity, substance and proof points. It’s a process. And as the tribe pointed out, context is critical to that process.
The Boston bombings brought to light an important issue related to context: Analysis is most effective when starting with a question, but what about situations like the Boston event where the question is yet to exist? What if BI and Big Data could preemptively detect threats as opposed to investigating after the fact?
The challenge lies in knowing how to put the pieces together in business intelligence or Big Data initiatives when the context is not yet understood. Without understanding the context, it’s rather like playing chess against an expert when you don’t even know the rules.
I’d love to know your opinions and experiences with determining the context for BI initiatives. Please take a moment and click to add your comments to the discussion.
Howard Dresner is president, founder and chief research officer at Dresner Advisory Services, LLC, an independent advisory firm. He is one of the foremost thought leaders in Business Intelligence and Performance Management, having coined the term “Business Intelligence” in 1989. He has published two books on the subject, The Performance Management Revolution — Business Results through Insight and Action, and Profiles in Performance — Business Intelligence Journeys and the Roadmap for Change. He hosts a weekly tweet chat (#BIWisdom) on Twitter each Friday. Prior to Dresner Advisory Services, Howard served as chief strategy officer at Hyperion Solutions and was a research fellow at Gartner, where he led its Business Intelligence research practice for 13 years.