On any Friday at my #BIWisdom tweetchat, you’ll find a tribe of followers who are keenly interested in discussing how to achieve better outcomes from business intelligence solutions. In chatting last week about what will be hot in business intelligence in 2013, the tribe focused on analytics and a prediction that Big Data will shift next year from tech talk to talking about business value. And ETL (the Extract, Transform, Load process) comes into play.
Some of the tribe tweeted that their clients and colleagues want more intelligent systems so that data doesn’t have to travel to the data center or ETL to the data warehouse. Many want to bypass data warehouses altogether as they can’t keep up with business need and users have little tolerance for DW latency. As one of our group pointed out, we still need the Transform of ETL today, but we don’t need the Extract or Load if the dataset and speed are reasonable. Hence the discussion of ELT (Extraction, Load, Transform), which is more suitable to end-user-based approaches.
Some of the tribe concluded that business people need their own data transformation, and some BI products are hinting at end-user ETL. Will it be separate from data warehouse efforts, or will it be a complementary “snap alongside” component that enhances the data warehouse and allows IT to serve the business better? One person tweeted that the new bottleneck in information is not just data quality and reliability but also semantics, which is a user issue.
So we’ve reached the “seriously-considering-the-need-to-get-information-management-into-the-business-person’s-hands”stage. We had an interesting chat with two opposite viewpoints.
The yea viewpoint: Users know the business relevance of the information and giving users the choice of how they want to integrate and view the data should be priority #1.
The “nays” punctured that balloon with:
- Users need to be shielded from semantics.
- Users should guide rather than handle the info management. They should have some freedom, but with parameters.
- Business users always have untapped sources that allow them to see the data their way.
- With user choice, we’ll have shiny new BI tools and devices replacing Excel; how will we be able to avoid a return to the 1990s?
- Business users dominating BI is as bad as IT dominating it.
And I asked the group: Is the infrastructure sufficient to handle putting information management (data transformation) into the hands of the users?
The quick verdict was that issues would arise because some data sources are “noise heavy” and don’t really merit tapping. Plus there are many BI distractions in the market now that are right for some organizations but not for all. The noise-to-signal ratio in data is increasing by orders of magnitude. How will this be managed, especially with users in charge? How will we surface the most valuable signals to users? And will the bulk of info remain untapped until someone nails this?
The tribe tweeted that the noise would cause users wearing a biz hat to “focus on low-hanging fruit every time.” That’s not a new problem in the business world, but it could jeopardize the value outcomes from business intelligence.
So perhaps we need a Low-Hanging Fruit Guidebook. The #BIWisdom tribe brainstormed with me on some rules to guide users:
- Avoid shiny objects.
- Think “valley” (departmental). There’s no low-hanging fruit on the mountaintops. Put another way: ensure contextual decision making; solve the tactical problems, and then boil the ocean.
- Broadcast success when it happens, and don’t be shy about identifying outliers.
Bottom Line: Rule #3 above is very important when it comes to value. Not all outliers are noise. Vendors are trying to automate every BI process. But as a Dilbert cartoon points out, the fact is most business decisions need a human. Computers can handle the labor surrounding the biz intelligence data, but so far we still need humans because we fundamentally think differently than computers. History bears out that predictors and trends were often mischaracterized as outliers. Sure, they’re noise, but some outliers turn out to be great opportunities. For now, we need to rely on humans’ internal systems to catch exceptions among BI data.
Howard Dresner is president, founder and chief research officer at Dresner Advisory Services, LLC, an independent advisory firm. He is one of the foremost thought leaders in Business Intelligence and Performance Management, having coined the term “Business Intelligence” in 1989. He has published two books on the subject, The Performance Management Revolution — Business Results through Insight and Action, and Profiles in Performance — Business Intelligence Journeys and the Roadmap for Change. He hosts a weekly tweetchat (#BIWisdom) on Twitter each Friday. Prior to Dresner Advisory Services, Howard served as chief strategy officer at Hyperion Solutions and was a research fellow at Gartner, where he led its Business Intelligence research practice for 13 years.