Data doesn’t matter. Big Data matters even less. Why? Data is data – just bits and bytes of facts and fiction. Data is not necessarily information, insight, and wisdom we can use or act upon. More data means less insight. This is our dilemma.
The roar of Big Noise from Big Data has already reached deafening levels. Making matters worse, much of this data does not reside inside your company. Eighty percent of it is outside your company. (See my article: Five Challenges of managing big data in the supply chain.)
Traditional business intelligence (BI) solutions – designed to help provide insights in the pre-Big Data world – were bad enough. Big Data just made a bad problem worse. Most of today’s analytics solutions are, unfortunately, mostly bolt-on solutions to existing ERP systems. Backward-looking reporting is useful to some degree, but busy managers and end users still lack real-time actionable information. Thus many of these solutions have questionable value and are a harder sell to users who are very cautious about IT spending in tough economic times. Cloud-based solutions are emerging. (See my article, The Real Value of Analytics in the Cloud.)
Facebook, Twitter, and the all other social sites – including the so-called enterprise social solutions – are draining whatever productive time people have left to do their real jobs. Unless you are in the hair-dressing business, you could care less if one of your followers is having a “bad hair day.” Do you care if someone is having a “100th cup of coffee of the day” or “going to sleep now” or “fed up with work, going home now?”
Enter social business analytics tools, promising to help you uncover some significance in all that noise. However, these tools have the same issue in that they do not provide any actionable intelligence or true analytical capabilities such as correlation analysis or mining of significance of Twitter or Facebook messages. For the most part, companies are using these tools for company branding and listening to customer sentiment, etc. The real value is when these tools can provide early warning signals of emerging issues or opportunities.
Where do you have the bandwidth to ferret out the real “signal” in all that “noise?” Where is that proverbial needle in the haystack? Is that “signal” relevant and insightful enough for you to make a difference in your business – that is:
- Help you win a customer against your competition
- Improve productivity
- Reduce costs
- Let you sleep at night, knowing that you are not missing opportunities or taking on risks that are hidden in that data
Whether we acknowledge it or not, we are all looking for useful, relevant, timely information and wisdom that we can act upon to help us reach our goals. That is the Holy Grail of the next-generation Big Data solutions.
Let the Big Data wars begin: a look at the companies in this space
Many companies are jumping on the bandwagon promising to rescue their customers from drowning in Big Data. Innumerable startups – Cloudera, MapR, 10gen, Revolution Analytics, to name a few – are clamoring to solve the many Big Data challenges.
Oracle is widely speculated to announce some big-data solutions at its Oracle OpenWorld 2011 annual conference this week, including Hadoop and NoSQL connectors to its Exadata appliances. Oracle is expected to unveil an in-memory database offering, positioning itself against SAP and IBM.
Meanwhile, SAP has been innovating on the in-memory computing platform with its HANA solutions. Designed from scratch using cloud principles, the in-memory, elastic, scale-out features of HANA means that it is well-suited for both private and public clouds. On the high-end, HANA enables large-scale, terabyte analytics with real-time response rates. HANA was also shown running on a Mac Mini in this year’s SAP SAPPHIRE NOW conference. HANA is an exciting innovation from SAP and can define its long-term growth story. Now the impending similar announcements from Oracle validate the market. (See my key takeaways from SAP SAPPHIRE article o get a sense of SAP innovations.)
Ingres, the 30-year veteran of enterprise-class databases and data management solutions, recently re-branded itself as Actian. Explaining their transformation from Ingres to Actian, Steve Shine, CEO and President of Actian, had this to say on the company blog:
“While most BI tools are good at providing insight (if you have the time and training to navigate the tools), they do little to drive results. Why? Because results come through action, but BI is action free. Our answer to this challenge, and our solution, are Action Apps running on the world’s first Cloud Action Platform powered by the record-breaking Vectorwise big data analytics engine.
“… Big Data analysis and Big Data actions. In short, we will provide businesses and organizations with the ability to develop their own action apps to automatically implement business actions whenever data events occur.”
From data to information to insight to action at the time it matters – that is what I’m talking about. The real value is when these tools can provide early-warning signals of emerging issues or opportunities.
Big Data is not just about analytics
Insights and actions are well and good, but let’s not forget that Big Data infrastructure and platforms are equally – if not more – important to access, process, and managing petabytes of data. This is where cloud-based systems shine. However, as I pointed out in my article Database-as-a-service: The next frontier in the cloud, many challenges remain to be addressed in the areas of performance and latency, data migration, bandwidth limitations, and application architectures.
This is not all. A recent Big Data study released by Saugatuck Technology titled “Understanding Big Data: A Management Study” enumerates a comprehensive list of Big Data complexities. Emphasizing the infrastructure and platform challenges, the study states:
“Big Data is not just about Analytics, though this is perhaps the most urgent area. It is also about organization, categorization, and access to data. There is an increasing realization that all data is not alike, and this means that the uniform models previously used to manage, store, analyze and retrieve it in the past no longer operate so effectively. Not only is the amount much greater, but the differentiation is also greater, and techniques used to shoehorn unwilling data objects (BLOBs, for example) into unnatural arrangements soon break down when
any kind of real access is required.”
Cloud architectures are the way forward; but this is still a very immature area, requiring much R&D investment and innovative new ways of handling the complexities of Big Data that marry the best of current NoSQL, Hadoop, MapReduce technologies with the traditional ways of handling data.
The current band of startups, the big database players like Oracle, enterprise business software players like SAP and companies like Actian (formerly Ingres) have an exciting opportunity in front of them as they tackle these complexities with innovative solutions. (Note: we are yet to see a Big Data offering from Microsoft.)
Being a cloud computing buff, I can’t but help but end by saying Big Data cloud infrastructure and storage are the only way to solve these challenges. Cloud platforms and storage are already here; it’s next a matter of exploiting their power to solve the real challenge of the day: Big Data.
Kamesh Pemmaraju heads cloud computing research for Sand Hill Group. Follow him on Twitter @kpemmaraju.