Big Data

The Rise of the Algorithm in Business Intelligence

  • author image

Until now, business intelligence has been like finding information on the Web in the early 1990s, the pre-Google era, where all we had was manually curated Web directories like that of Yahoo!. Information search was limited to data that was previously listed in canned directories. But as information on the Web exploded, it was clear that human editors could not keep pace with the rate of information growth, leaving only a small percentage of information on the Web discoverable through dated directories. 

That’s where Google came in. They replaced those manual directories with algorithms that helped people discover information, regardless of where that information resided or when their initial queries were generated. They were the first company to make information discovery truly usable. They also have been instrumental in bringing natural language search to the forefront of data in recent years.  

With history as our guide, we shall see in the coming months and years of business intelligence (BI) innovation that algorithms will replace manual efforts to enable data discovery in the growing web of enterprise data. Natural language question answering systems provide natural and easy user interfaces on top of sophisticated algorithmic platforms that automatically understand data residing across different sources and their relationships, which will drive business intelligence to its true era of success. 

This is even more important now that Big Data has grown beyond just a buzzword to an initiative companies are focusing large efforts to tackle successfully. Manual approaches to BI data modeling and warehousing cannot scale to the volume, velocity and variety of data that’s being produced today. 

What’s needed for BI to become truly pervasive is a system that helps organizations make faster, smarter, operational decisions based on all data that a business has access to without having to learn complex software or spend dollars and months in setup. What’s needed in BI today is an embedded business intelligence model that will put analytics in the path of everyday business activities so that employees can focus their efforts on making intelligent business decisions based on the data rather than spending their time and knowledge on learning and managing BI installations. What would speed the process along even further is the ability to actually ask your data a question via text or voice and get a significant response back in real time to further drive productivity. 

C-level executives, to justify the investment in BI, have specific requirements. They want real-time benefits and the added value of context with regards to the original business queries. This dictates more science than art, which leads us to the algorithmic approach. Algorithmic equations from the front end to the back end, as opposed to manual setups, will yield big results for companies of all sizes; they can finally expect to see the returns previously promised by traditional BI vendors. 

One key tactic to getting current and accurate data in a timely manner is to ensure that the processes that move data from source to target systems run smoothly and quickly and any errors can be identified and resolved quickly without having IT staff waste valuable time finding and fixing issues manually. Business intelligence systems that address the constant changes that happen in data sets must allow software companies, hardware companies and information service providers to implement and go to market faster and more cost-effectively while at the same time processing at rapid speeds without the hindrance of an IT professional constantly adjusting internal systems based on specific queries.  

For decades now, the business world has struggled to analyze its data. Companies have used humans to painstakingly understand their data structure and analytics needs and manually build out data warehouses, canned reports and dashboards. These time-intensive systems were not only expensive but often hurt the business. By the time IT was able to dive deeper into the analytics platform the business needs had moved on and a new initiative was under way. This also made business intelligence solutions cost-prohibitive for small and medium businesses that lack the budget and resources to spend on BI cycles or on professional services. 

As the news this year has showcased, technological innovations like that of IBM’s Watson have proven to the tech world that natural language question-answering systems will dramatically change the way we interact with search and with our data. 

Intelligent machines with sophisticated algorithms are rising to save the business world millions of dollars and incremental amounts of time building data warehouse models, which will likely become obsolete soon. Natural language search driven by intelligent virtual agents outfitted with sophisticated algorithms is the future of data discovery and intelligence in 2014. Removing manual setups is the next logical step in this equation. In today’s fast-paced world, time is more crucial than ever. Organizations need a system that can provide the capability to draw inferences and conclusions. 

Before you hire that next data scientist, just remember that being able to make better business decisions and judgments based on instant intelligence from data is key — and algorithms provide that capability. 

Buckle up, because the rise of the algorithms is upon us and will change the way we view business intelligence and interact with Big Data in big ways. 

Sundeep Sanghavi is CEO of DataRPM. He is a highly accomplished technologist, innovator and entrepreneur with a proven track record of successful startup companies. Prior to starting DataRPM, he founded Razorsight, where he led the company’s efforts to deliver innovative solutions to global communications providers and became a leading provider of analytics and business intelligence.

 

 

 

 

 

 

 

 

 

Comments

By Chris Kenton

I don’t doubt the value or disruptive capacity of improved data algorithms, but the enthusiasm over how quickly this will revolutionize business doesn’t seem to be shared by IBM, who has been struggling to ramp Watson from a Jeopardy whiz to a scalable business. (See WSJ: IBM Struggles to Turn Watson Computer Into Big Business, Jan 7.)

Improved algorithms don’t solve the problem of how intelligence has to be shaped with a deep understanding of the subtleties and context of business decisions the intel is supposed to drive. Engineers and VCs always emphasize the elimination of humans in the pursuit of scale and reduced costs — which inevitably leads to brute force questions over processing speed, storage capacity, throughput, etc. They forget that at some point, humans still have to make decisions about how to shape the questions that need answering, and how to execute on the answers that are returned.

At this stage of the game, I put more stock in brilliant workflow that puts data technology in the hands of non-technical users than bigger, faster, stronger algorithms.

Post Your Comment




Leave another comment.

In order to post a comment, you must complete the fields indicated above.

Post Your Comment Close

Thank you for your comment.

Thank you for submitting your comment, your opinion is an important part of SandHill.com

Your comment has been submitted for review and will be posted to this article as soon as it is approved.

Back to Article

Topics Related to this Article