In recent years, organizations have improved their abilities to gather, consolidate and analyze data in order to unearth trends, patterns, insights and other information upon which they can develop effective business strategies and processes. Beginning with spreadsheet programs and evolving to today’s enterprise data warehouses, growing and diverse populations of users have been giving enterprise data “the deep dive” as a means of improving the targeting, retention and value of customers; developing, implementing, monitoring and evaluating marketing initiatives; making the most effective investment decisions; creating new products and more.
Automating information processing and developing increasingly sophisticated and powerful databases and business intelligence tools have provided the foundation on which analytics capabilities have advanced. Organizations that have implemented decision support systems, data warehouses and business intelligence applications have seen impressive returns on their investments in terms of productivity gains, revenue and profitability increases, and have achieved substantial competitive advantages. Numerous studies have demonstrated quantitative and qualitative benefits of employing analytics in formulating business decisions.
As the pace of business has accelerated and competition has grown increasingly ferocious, organizations desire to gain more competitive capabilities from analytics. The good news is that analytics technology has continued to advance, providing not just increased performance and scalability, but the ability to explore and analyze data in ways that have not been possibleuntil now.
It’s About Gaining Foresight
By now, you’ve probably stumbled across any number of articles about Predictive Analytics. Still, you may have some questions about what it is, what it allows you to do and what sort of benefits you’re likely to achieve, should you choose to embrace it. You may also harbor a bit of skepticism, given some of the metaphoric allusions in media and analyst reports to predictive analytics as a digital version of a crystal ball.
To be certain, it is not a crystal ball as we envision such a thing from works of myth and fantasy.
It is, however, an advanced analytics activity that differs from conventional analytics and BI activities in that it focuses not on what happened or how it happened, but what might happen or could happen. It’s not simply about uncovering insights, but gaining foresight.
In short, predictive analytics is primarily about examining data through a future-facing lens in order to predict future trends, behavior patterns, external forces and other factors that will determine future business results. It is a natural extension of many analytic capabilities we now have, all of which are ultimately intended to help us design innovative business strategies that will lead to improvements in all the key metrics – revenue, profitability, customer retention, market share, etc. – that define success.
As such, it’s no wonder that Predictive Analytics is growing in popularity as evidenced by its increasing usage in areas such as:
- Sifting through data assets to look for patterns and relationships that explain certain observations such as the disparity of customer retention numbers in urban and rural areas or unexpected increases in fraudulent claims.
Predicting future outcomes based on historical data analysis. For instance:
- What will next year’s revenue look like based on current trends?
- How inflation trends may affect future profits?
Algorithmic representations that predict behaviors and events. For instance:
- What is the expected behavior of a particular customer if the price of monthly subscription is raised by a certain percentage?
- When will the next outlier economic event take place?
Improving upon existing decisions for better outcomes. For example:
- What package routing alternatives exist that will lower cost of delivery without displeasing customers?
- What productivity gains can be expected if three cost factors are eliminated instead of the two?
Predictive Analytics in Action
Predictive Analytics can be widely applied across most major industries. To illustrate this, here are a few examples.
- Telecommunications. Telecommunications firms constantly deal with the challenge of customer churn. Their ability to retain their more profitable customers can have a major impact on their bottom lines. These organizations often employ Predictive Analytics to determine what offers to extend to targeted customers to ensure profitability. Similarly, they can use Predictive Analytics to target customers whose profiles suggest they will become profitable once acquired, and to fashion offers designed to achieve that objective.
- Insurance. Insurance claim fraud is pervasive and costly with estimated losses exceeding billions of dollars worldwide. Insurance fraud cases are difficult to uncover because of the complexity of linkages in activities that lead up to fraudulent claims. Predictive Analytics techniques are frequently used to sift through large volumes of historical data to find specific types of activities and relating them to recent claims helps flag suspicious claims for investigation.
- Marketing Services. Marketing services firms combine industry wide data with client data to search for insights that will lead to better outcomes in their campaigns. For example, a common practice is to build a high quality list of prospects based on characteristics and behaviors that indicate which prospects are likely to respond to a particular campaign offer. Searching through lists of customers and prospects and predicting who is most likely to respond requires heavy usage of data mining and predictive modeling techniques. It is well worth the effort, though, as a high quality list is more likely to yield the desired results and produce a significant return on investment.
- Banking. Banks are prime users of Predictive Analytics because many business decisions banks make each day are often based on predictions drawn from analytical processes and tools that consider several input factors. For example, predicting the likelihood of a given customer defaulting (i.e. the default factor) on his or her loan is based on a number of input variables related to employment status, payment history, credit history and other personal financial factors, any of which can change, possibly requiring an account review and restructuring.
- Government. Predictive Analytics is widely used by some government agencies to identify tax or customs fraud, monitor criminal activity and conduct other analyses designed to uncover unlawful activities or practices on the part of individuals and organizations.
Predictive Analytics is clearly much more than the latest buzzword. It is a technology-based approach to understanding the business ecosystems in which we work, and to identifying opportunities and trends upon which we can capitalize. It’s worth taking a few moments to discuss Predictive Analytics’ underlying concepts and technology to appreciate how Predictive Analytics can be integrated into existing enterprises. Predictive Analytics is primarily based on the concept of modeling business problems using mathematical and statistical algorithms. These algorithms use a set of input variables from a given data set to predict target variables that will support decision-making.
From a process perspective, Predictive Analytics involves a number of key steps:
- Understanding the business problem
- Tying prediction variables to the problem
- Selecting the appropriate statistical techniques relevant to the business problem
- Preparing the input data for application of the model
- Validating the model with test data, and finally
- Applying the model to the production data and observing the accuracy over time to make adjustments to the model.
Invariably, any conceptual discussion like this is bound to lead to more technology-specific questions such as:
- How to deal with large data sets?
- How to improve the accuracy of models?
- How to achieve the performance required for timely predictions?
And needless to say, choosing the right methodology along with the right tools and technologies is critical.
There are several categories of tools available to help in Predictive Analytics initiatives.
The backbone for most Predictive Analytics applications is the data management repository for the data set(s) used to build and score the Predictive Analytics models. These tend to be high performance database servers. Increasingly, organizations are deploying columnar databases designed specifically for analytics as opposed to traditional relational databases for their predictive analytical applications.
In addition to the analytics database server, enterprises require data movement and data quality software to ensure accurate representation of the input data before it can be used for predictions.
A particular requirement for Predictive Analytics projects is a predictive analytics tooling platform to build and score the Predictive Analytics models. These platforms provide a comprehensive set of design, development, algorithmic and other aids essential for Predictive Analytics operations.
A fairly recent technology innovation – In-Database Analytics – wherein application logic is embedded within the analytics database server to increase performance, latency, and security, is emerging as a very popular and useful technology for optimizing Predictive Analytics applications.
No doubt, as more users analyze more data from more perspectives for more specific business purposes, software companies will deliver additional technologies will appear to keep pace with end user demands.
A New Core Competency
Predictive Analytics is here to stay. Its value in solving specific business problems across a variety of industries is already well established and growing.
Nonetheless, organizations need to consider several factors before jumping into the Predictive Analytics fray. They need to understand the types of business problems that may benefit from a Predictive Analytics approach, review the applicability of Predictive Analytics to specific problems in their enterprises, and evaluate the technologies they will need to integrate into their existing IT architectures.
Many organizations have done this, taken the plunge, and realized substantial business benefits from doing so. As businesses grow ever more data intensive and dependent, we expect many other organizations will follow. In fact, in some industries and within some organizations, Predictive Analytics is likely to become the next must-have competency for those determined to thrive whatever the business climate.
Dr. Raj Nathan is Chief Marketing Officer and Senior VP of Worldwide Marketing and Business Solutions Operations and Joydeep Das is a Senior Product Manager at Sybase.