Editor’s Note: In this SandHill interview, Sohaib Abbasi, CEO of Informatica, discusses trends fueling Big Data innovation, examples of business value, and the biggest mistake IT organizations make regarding Big Data.
SandHill.com: There is a lot of talk about the challenges of Big Data today. How do you describe Big Data?
Sohaib Abbasi: Big Data is a confluence of three related but distinct technology trends: Big Transaction Data, Big Interaction Data and Big Data Processing. Each trend represents breakthrough innovations and promising potential. Now, as the combination of all these innovations, Big Data opens up tremendous new possibilities.
Most of all, Big Data delivers big business value as illustrated by these three customer examples:
- By processing five times more trading transaction data, DirectEdge became one of the top stock exchanges in the US.
- By leveraging Facebook interaction data, IKEA increased sales by 15 percent.
- By using highly scalable Big Data processing, Visa decreased time for risk modeling from one month to 13 minutes.
Clearly, Big Data delivers big business value!
SandHill.com: What impact is the cloud having on shaping Big Data solutions?
Sohaib Abbasi: It’s more than the cloud. Big Data, like our overall industry, is being reshaped by the nexus of three technology trends: Cloud computing, social computing and mobile computing. They are simultaneously redefining the “where,” “what” and “how” of the computer industry.
- The “where” is moving from on-premise computing to cloud computing, changing the economics of computing. By changing the economics of computing, cloud computing enables even more transaction data to be cost-effectively captured by cloud service providers.
- The “what” is shifting from “transaction processing” with business applications like SAP and Oracle to “interaction processing” with social media services like Facebook, Twitter and LinkedIn, changing the role of computing. By enabling the next generation of data-centric applications, social computing represents unprecedented volumes of interaction data.
- And, finally, the “how” of computing is adapting from desktop computers to mobile devices, changing the face of computing. By empowering users with context-aware and location-based services, mobile computing represents the next surge in device sensor interaction data.
These secular technology megatrends are driving Big Data – larger volumes, broader variety, higher velocity and more business value. In other words, Big Data is here to stay for a long time.
SandHill.com: What do you see as the promise of social and mobile computing technologies?
Sohaib Abbasi: The primary role of business computing has changed from automating transactions to facilitating interactions. Social computing represents the next wave of data-centric applications capturing interactions with social networking services like Facebook.
Mobile computing represents the next surge in data-access devices capturing device-sensor interactions such as location data. Adoption of social computing and mobile computing is further driving higher volumes and even greater variety of data.
The value proposition of business computing is shifting beyond operational efficiency to organizational effectiveness. Until recently, organizations focused on operational efficiency by automating transaction processing with department applications like ERP and CRM. Now organizations are moving beyond business management to brand management to facilitate organizational effectiveness. Social media enables holistic brand management, including proactive customer engagement and consumer sentiment analysis. To gain a competitive advantage, organizations now aspire to become social enterprises.
Social networking is one of the top growth phenomena of all times! In less than a decade, the consumer social media has grown from zero to more than one billion users. Social media is now a mainstream technology for consumers and is fast becoming a mainstream technology for businesses as well. In fact, the vast majority of enterprises already regard social media as an important marketing tool. Before long, social media adoption will be pervasive to drive organizational effectiveness.
The purpose of computing devices has changed from personal productivity to collaborative productivity. Until recently, the personal computer was primarily used to run Microsoft Office for a few hours a day. Now, a broad range of mobile devices are used to run half a million apps for most of the day.
A leading venture capitalist published a chart to underscore that smartphones overtook the personal computer last year as the popular device to access the Internet. And, for computing devices, there is no turning back.
SandHill.com: Why is there so much excitement around the Hadoop technology?
Sohaib Abbasi: The latest open source technology, Hadoop, is designed for highly scalable processing. In a recent recognition award, Hadoop was billed as the “Swiss army knife of the 21st century” and described as a “greater catalyst for innovation than the iPad.” Will it live up to the hype?
The key technology, MapReduce, is a programming language developed by Google. It leverages parallel processing for high volumes of data, including unstructured data. The persistence layer, developed by Yahoo, is a file system named HDFS. More recent frameworks, like Jive and PIG, provide productivity aids for developers. First published in 2004, MapReduce is now a well-established programming language. Backed by industry leaders, Hadoop has a powerful ecosystem. With a vibrant open source community, it continues to evolve as a broad and versatile platform.
The primary strength of Hadoop is proven cost-effective scalability by leveraging commodity hardware. As an example, Facebook has a Hadoop cluster with 30 petabytes of storage. MapReduce is a general-purpose language to process all types of data. And the open extensibility mechanism enables developers to augment Hadoop with specialized capabilities for a broad range of applications.
SandHill.com: What is the biggest mistake companies make in seeking/selecting a solution for integrating their data? What mindsets cause them to make this mistake?
Sohaib Abbasi: The most common mistake by IT organizations is to manually hand-code their data integration projects; they regard these tasks as tactical. Such a labor-intensive approach is very expensive both for the initial implementation and, even more, for subsequent maintenance.
By regarding data integration projects as tactical, data integration projects are typically managed by different groups within IT. This redundancy invariably leads to increased costs and inconsistency. And this, in turns, often compromises the business value of data.
Leading practitioners recognize that data integration is increasingly business-critical. These organizations are establishing centralized Integration Competency Centers, or ICCs, to both automate data integration tasks and implement best practices. With an ICC, organizations are better positioned to maximize the return on their data: increasing value by delivering relevant, trustworthy and timely data. At the same time, such a disciplined approach helps reduce the cost with greater sharing and reusability. Enterprise data integration platforms are instrumental to maximize return on data.
SandHill.com: What is the risk in a company waiting until more solutions are available in the marketplace or waiting until solutions are more mature?
Sohaib Abbasi: The risk in waiting for mature solutions is losing an important competitive advantage: time to market. The secular technology mega-trends – cloud computing, social computing and mobile computing – represent tremendous advantages for the early adopters. Early adopters are leveraging these new computing platforms for customer-centricity and social commerce initiatives.
As an example, a leading retailer is measuring not just the purchases by their top customers but also the influence of their customers on the purchases by their network of friends identified by social media sites like Facebook.
SandHill.com: What are the risks for early adopters?
Sohaib Abbasi: The risk for the early adopters is being locked into technologies or vendors that may not be the long-term winners. As an example, in the social media segment, the early pioneer MySpace lost their lead to Facebook. However, by relying on a neutral, vendor-agnostic approach, the early adopters can leverage the wealth of data without being locked in to any one vendor or being locked out of any data.
With near-universal connectivity, the leading data integration platforms enable organizations to gain a competitive advantage by leveraging the wealth of data both within the enterprise and beyond, including cloud computing sources and social computing services.
SandHill.com: How does the Informatica platform support the three Big Data trends?
Sohaib Abbasi: The latest Informatica 9.1 platform uniquely supports all three Big Data trends:
- For big transaction data, Informatica supports near-universal connectivity to traditional OLTP and new analytic databases
- For big interaction data, Informatica delivers connectivity to access interaction data from popular social media services like Facebook, Twitter and LinkedIn
- For Big Data processing, Informatica 9.1 features connectors to move data in and out of the Hadoop file system, a highly scalable open source technology stack.
The promise of Big Data is to gain business value in four ways:
- Analyze the transaction data
- Relate the social interaction data
- Correlate the device sensor data
- Expedite processing of large volumes of data.
For each of these four, the Informatica platform increases the value of data:
- Trustworthy and secure transaction data to analyze
- Authoritative and relevant interaction data to relate
- Timely and actionable data to correlate
- Holistic and accessible data to process
SandHill.com: Please share some examples of the business value companies are gaining from innovation around Big Data.
Sohaib Abbasi: Innovations in OLTP and analytic databases deliver impressive business value. Our customer, DirectEdge, dramatically increased the velocity of their transactions by reducing the latency of an individual trade by 83 percent to only 340 microseconds. By processing five times more transactions, DirectEdge is now one of the top five stock exchanges in the U.S.
Another of our customers, Stations Casino, showcases the value of business analytics. With better analysis of transaction data across 500 data sources, they extended the player times at the slot machines, increasing slot revenues by four percent. Even more impressively, with targeted promotions, they increased new members by 160 percent. Companies are beginning to realize the promise of Big Data by analyzing transaction data.
SandHill.com: Do you have examples of business value gained in innovations around social computing and mobile computing?
Sohaib Abbasi: Sure. With Twitter, Dell is crowdsourcing ideas for new products and services while increasing sales. And a top telecommunications leader is leveraging billions of call data records to reduce maintenance costs by 60 percent and increase revenues by 30 percent.
Companies are beginning to realize the promise of Big Data by relating social media and correlating device sensor data. Big Data will live up to the lofty expectations only by maximizing return on data to deliver business value.
SandHill.com: Do midsize companies approach decisions around this type of solution differently than larger enterprises?
Sohaib Abbasi: All midsized and large companies share the same challenge of integrating silos of data. The opportunity for both is to maximize the return on their data by increasing the value of their data and decreasing the cost of their data.
With timely and trustworthy data, organizations are able to compete more effectively. Even though the opportunity is identical, the processes are influenced by both the size of the organization as well as the maturity of the IT organization. In these times of uncertain change, data matters more than ever, and organizations in all segments aspire to become data-centric enterprises.
Sohaib Abbasi is Chairman and CEO of Informatica. Under his leadership, Informatica is now the largest independent leader in the data integration software category. Fostering a culture of continuous innovation, Abbasi has built on Informatica’s success in data integration and led Informatica into new software market categories. In addition to growth in new product categories, Informatica is gaining significant contributions from international regions through global expansion of field operations. Prior to Informatica, he spent 20 years at Oracle where he was a member of Oracle’s executive committee and, as senior vice president, led two major divisions, Oracle Tools and Oracle Education. He joined Oracle in 1982 when it was a 30-person startup and was instrumental in growing the business from $4 million to more than $9 billion in annual revenues. He envisioned and launched the Oracle Tools business.