In our typical day, we use integrated technologies and don’t even think about it. You log into your personal applications using your Facebook or Twitter account; your Google+ account recommends new contacts in your Circles based on your Gmail activity. You simply enter your banking and credit card account information in Mint and quickly you have all your financial information at your fingertips to budget, plan and track. We live in an integrated world and don’t think much about it day to day. Until we’re at the office and attempting to service, sell or market to our customers. Need to know what your customer bought over the past two years? You need to request that information from accounting because it’s not listed in your CRM system and you’re not allowed access to the financial systems. Want to understand your customers’ website visits, what webinars they attended, questions they may have asked about your new product? Good luck finding all that in one place!
Each year, Scribe surveys partners, customers and prospects to better understand the “who, what, why and how” of customer data integration. Over 900 business and IT leaders responded this year from over 50 countries, from small business to global companies. We’ve compiled these findings into our 2013 State of Data Integration Report, which has a wealth of interesting information about how and why people are integrating customer data.
One of the standouts from the report was that we didn’t see a significant uptick in respondents believing they had full integration between their core business systems this year from last year — only 16 percent of companies reported having full integration. While it wasn’t surprising, this finding is significant. Why would businesses continue with manual approaches when a variety of data integration products, solutions and platforms are available today? This article explores why and discusses some reasonable steps to integrating customer data more easily than defaulting to a do-it-yourself approach.
The state of customer data integration
Integration, and particularly customer data, is a hot topic with media and analysts. As we’ve seen in numerous articles, reports and blogs, everyone talks about the imperative to integrate business systems to bring together all customer data for the holy grail of “the 360 view” and the amazing operational and revenue improvements integration brings to the business.
It’s great to be a software provider in a hot technology space; but if you ask any integration vendor, you’ll find that the biggest competition for all of us is manual data integration, or worse, inertia and not integrating at all. 2013 State of Data Integration Report reveals that while everyone gets the benefits of integrating customer data, the actual doing of the integration is not happening at the same pace. Integrations with business intelligence (BI) and service and marketing applications were ranked as most important by over 70 percent of respondents. Social was listed as important by almost half of the respondents, which was a 15 percent increase from 2012. Yet only 16 percent of businesses reported having full integration with their business systems and only 50% reported they were “partially” integrated.
When we asked how these integrations were done, 48 percent of the respondents were using custom code and another 32 percent relied on manual data entry or other processes. Given the number and variety of integration products available today, one would think that custom code or manual data entry would be the exception, not the rule.
Figure 1
The explosion of data sources available on an ever-growing array of technology platforms combined with decentralized technology purchasing are two key trends that influence the continued use of manual integration techniques. Departments are under pressure to get data into the hands of their employees and customers quickly. On the surface, just doing the integration manually or writing a few lines of code to alleviate the immediate need can be more expedient than purchasing, learning and implementing an outside solution.
Departments outside IT might not know there are integration solutions available and believe their only option is to go it alone. IT should be responsible for the data management strategy across the business and partner with other areas of the business to integrate these new sources of data in a standardized way if they want to avoid a pastiche of custom code and manual data entry.
Businesses and IT departments that continue to evaluate integration approaches based on thinking formed a decade ago are at risk. Certainly in the past, integration offerings were fewer, expensive and required senior-level technology skills to implement and maintain, but in recent years the landscape of integration offerings has radically changed.
Today integration platforms have proven they can provide significant optimization to the business. In one industry case study, a business reported saving four employee workdays per month after implementing integration.
Figure 2
When asked why businesses continue to use these manual approaches, not surprisingly two of the top three reasons shared were cost and control. Companies typically continue to rely on custom coding and manual data input due to a lack of education, either remaining unaware of integration solution platforms or unconvinced of the improved benefits possible from such an investment.
The hidden costs of manual data integration
IT may eschew integration solutions or pre-built connectors due to their preconceived thoughts on costs, either direct costs or because of the cost to their flexibility in meeting business needs. That thinking is dangerous today as the costs of custom coding and manual integrations add up quickly, and worse, these approaches cannot deliver the speed and agility the business needs today. Given the increasing amount and variety of technology environments, applications, data sources, and vendor APIs, it is impossible for the IT department to keep current with their manual integration techniques.
IT’s reliance on custom code poses resource constraints that limit companies’ ability to integrate systems fully, and this lack of integration in turn plays a role in the CMO Council’s finding that only four percent of marketers and seven percent of IT executives believe they are prepared to exploit the proliferation of data and channels, many of which will remain siloed. Custom code coupled with manual data entry costs businesses millions of dollars annually but aren’t immediately visible to IT and business leaders. The latest Oracle-commissioned study found that 52 percent of businesses reported missing deadlines due to poor integration across cloud vendors and 75 percent of businesses surveyed blame cloud integration problems for stunting innovation initiatives.
Hidden costs from custom coding extend beyond the cloud and include the loss of real-time data and analytics, which is a tremendous opportunity cost for businesses, affecting their ability to drive revenue and customer satisfaction. Custom code also increases the total cost of enterprise IT systems ownership by adding resources for training, along with additional development and maintenance time. Moreover, custom coding adds possible hiccups in data conversion that can result in loss of data or added time to de-duplicate or reconcile data. For example, Ventana Research found that combining spreadsheets is time-consuming in 56 percent of organizations, mostly due to getting the data into the right shape.
All of the above shapes up to this basic truth:businesses need access to data and they need it now, but custom code hasn’t provided companies with the scale and efficiency required to meet this need.
Embracing a modern integration approach
Although companies have been slow to change, 2013 State of Data Integration Report showed custom coding experiencing a significant drop of 11 percent from 59 percent of businesses who reported using custom code in 2012. The use of other integration methods has risen in the past year, which includes software vendor solutions (41 percent) followed by third-party integration platforms (32 percent).
We’re seeing a shift in thinking with businesses less prone to building each integration on their own. In fact, delivering value quickly and responding to change necessitates not building integrations from scratch. Integration vendors are creating more pre-built connectors and solutions than ever before, improving and simplifying the user experience, and offering a broader range of price points to make integration more accessible to a wider market and technical expertise.
Looking ahead: How to deliver value
Consider taking another look at today’s integration offerings before you default to custom code or manual integration. Chances are that you can find an integration product that fits your budget and technical expertise. These tools can be the best value to provide integration quickly to your business.
You’ll also find that many integration vendors recognize the need for advanced customization, including custom code, and most platforms today offer developer kits, APIs and the ability to edit integrations at the source code level giving you the best of both worlds.
System integrators are another resource that can assist with your integration needs. The majority of systems integrators have embraced and standardized on integration solutions; they are an excellent source of expertise and experience with integration platforms. 2013 State of Data Integration Report confirms that businesses that use systems integrators report more satisfaction overall with their integrations and tend to integrate systems at three to four times more than businesses that don’t use systems integrators.
Betsy Bilhorn is VP of Product, Scribe Software. As vice president, product management, Betsy is responsible for leading Scribe‘s new product initiatives. Prior to joining Scribe, she held several leadership positions at SaaS pioneer WebTrends, including participation in the evolution of WebTrends OnDemand. She has also held leadership positions at system integrator, Lease Dimensions, where she managed large deployments of global CRM, ERP, and financial systems, including acquisition and integration.