Is In-Memory database technology the next major innovation in the world of business intelligence? Is it the game changer that will provide the unfair advantage that leads to the competitive advantage that every CEO wants today? There is certainly a lot of hype around this (not new technology) and many vendors tout it as the next greatest advance in the business intelligence tool chest. In this article I will avoid the technobabble of In-Memory computing and instead look at it from a practical business perspective.
One of the most amazing experiences of my enterprise computing career has been when technology optimizes business processes. In-Memory has great potential to enable real-time decision making in many aspects of business, especially operational business intelligence (BI) and performance management applications.
What is In-Memory?
There are many definitions of In-Memory, but I like Wikipedia’s definition the best:
“An in-memory database is a database management system that primarily relies on main memory for computer data storage. It is contrasted with database management systems, which employ a disk storage mechanism. Main memory databases are faster than disk-optimized databases since the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk.”
The real magic here from my perspective is the ability to replicate an OLTP database even on multiple nodes. This enables lightning-fast analytics performance and will be a major enabling technology for running data-centric enterprise applications in the cloud.
In-Memory is here now in various forms:
- As a standalone diskless database
- As an appliance with built-in analytical modules (IBM-Netezza, SAP-Hana, Oracle Exalytics)
- Embedded in many ERP systems and other business applications that are data intensive
- Embedded in business intelligence platforms
- Embedded in data-centric cloud applications and infrastructures
- Open source: WebDNA7
- Strong set of analytical tools and modules integrated with In-Memory, often optimized for industries and business functions such as operational reporting.
The ability to replicate, store and perform analysis on real-time data even from OLTP systems enables organizations to make real-time decisions on nearly all aspects of business operations that have been digitized. Way back in 1999 when I ran the BI/DW Research Program at Survey.com, our research and surveys found that BI managers wanted the ability to data mine real-time transactional data bases; In-Memory now makes this is a reality.
However, the rigorous disciplines of business intelligence like extraction, loading and cleansing of data in addition to the appropriate application of analytical tools still remain. Remember the old business intelligence adage of the 1990s: “garbage in, garbage out.” Decisions made on unreliable or incomplete data sets lead only to bad decisions, real time or not.
The business value of In-Memory
The inherent business value of In-Memory revolves around the ability to make real-time decisions based on accurate information about seminal business processes. An Oxford Economics study (www.businessinrealtime.com) of 500+ C-level executives conducted in the spring of 2011 focused on real-time business in the oil and gas, retail, consumer products and high-tech industries. This research had some great findings and insights about how companies can gain competitive advantage with a real-time business approach enabled by In-Memory including:
- “At an operational level: by speeding up data capture and simplifying processes executives can reduce inventories, minimize business risks, lower operational costs, accelerate speed to market, foster productivity and better meet customer needs.”
- “At the management level: by accelerating decision making and planning executives can exploit market opportunities faster, identify competitive threats sooner, cope with market shifts more quickly and transform stagnating businesses.
There are a significant number of academic researchers studying In-Memory in North America. They include Sam Madden at MIT, Ousterhout at Stanford, Abadi at Columbia, Mack Franklin at Berkeley and Peter Weil at MIT’s CISR, as well as Tom Davenport at Babson. However, aside from Davenport, the majority of the research focuses on the technical aspects of In-Memory and not the business application and value. Davenport authored another interesting article related to In-Memory: “How Fast and Flexible Do You Want Your Information, Really” (MIT Sloan Management Review, Spring 2011).
The business intelligence arena is strewn with acronyms that most business people don’t understand and that unfortunately are the language of computers. Line-of-business people, managers and even CEOs will derive the most business value from In-Memory and could care less about the technology and the BI acronym jungle. There are many business processes and business intelligence applications that will be innovated with In-Memory such as query and reporting, online analytical processing (OLAP), predictive analytics, scenario planning, simulation, price optimization, and planning and budgeting.
Key areas where In-Memory delivers business value
- Finance
- Performance management
- ERP systems
- Operational business intelligence
- CRM
- Mobile business intelligence
Finance and performance management
In my view these are the most important application areas that can benefit from In-Memory-enabled business intelligence. Many organizations spend a great deal of time running financial queries and reports on accounts receivable. For example, queries can take hours or even days depending on their complexity. One In-Memory vendor touts that it reduced an account receivable query from 77 minutes to 13 seconds. The ROI on this is somewhat debatable. Would waiting an hour for a query to run vs. getting it in seconds make that much of a difference in decision making?
Business analysts often spend 90 percent of their time running queries and significantly less time doing actual analysis. Faster queries, more time for analysis should ultimately lead to overall improved business performance.
Innovation in ERP
In-Memory is a huge innovation for all ERP customers that have been waiting decades for real-time access to transactional ERP data. Vendors will undoubtedly target industries with large embedded ERP systems that deal with huge volumes of data such as:
- Retail
- Manufacturing
- Financial hedge funds, brokerage, insurance
- Energy
- High technology
- Telecommunications
- Consumer Product Goods
- Cloud infrastructure service companies
Innovation in CRM
The ability to know and understand the customer experience is paramount in the new millennium as organizations strive to improve customer service, keep customers loyal and gain greater insights into customer purchasing patterns. This is even more important as a result of social media and social media networks that are now the new “word-of-mouth platform.”
In-Memory-enabled BI tools will provide real-time data not only from transactional systems but also will allow organizations to harvest and manage unstructured data from the social media sphere.
Innovation in mobile business intelligence
Demand for access to data from mobile devices is exploding, especially in the retail industry, and IT is hard pressed to deal with it. In-Memory databases, especially in the cloud domain, will play an increasing role in delivering on the promise of real-time data, anywhere anytime. There are a number of new mobile BI applications with easy user interfaces that will greatly benefit from this potentially game-changing technology.
Net/net
In the age of Big Data, In-Memory is a welcome addition to the business intelligence arsenal, and it will undoubtedly significantly change and optimize many business processes. More importantly, it has the potential to allow organizations, line-of-business managers and the C-suite to spend more time on simulations, scenario planning and analysis, and less time on building and waiting for queries. This is important in the age of the Internet where information travels at a business velocity slowed only by geographical time zones.
Peter J. Auditore is the principal researcher at The Data Dog, a boutique consultancy focused on strategic and executable marketing communications for this millennium. Formerly he was head of the SAP Business Influencer Group and a veteran of four technology startups including Zona Research (cofounder), Hummingbird (VP Marketing Americas), Survey.com (president) and Exigen Group (VP Corporate Communications). He has over 20 years of experience in selling and marketing software to LE and SME organizations worldwide.