Skip to main content

11 Tips for Ensuring Your Big Data Initiative Succeeds

By May 7, 2013Article

The ability to easily extract meaning from complex big datasets has become something of a Holy Grail for business. Whether it is analyzing millions of investment trades per day, monitoring online search terms or correlating Facebook “Likes,” the super abundance of digital data (i.e., Big Data) offers tantalizing opportunities for competitive advantage. Everyone wants in.

Until recently Big Data was too costly or too difficult to extract and process. Google might be able to do it, but not the rest of us. Today’s commodity hardware and cloud server architectures change that. Big Data processing is available to everyone from small firms to large enterprises. There are both commercial and open source BI / data mining software that can integrate, “clean up” and parse these large datasets, then present them in a dashboard.

But adoption/usage rates for these tools across organizations tend to be low. The culprit is not poor data crunching; the culprit is poor user experience design. The tools make it hard and cumbersome to interactively drill down into the underlying data, re-query, or untangle and visualize the connections between variables and key performance indicators (KPIs).

This article offers 11 tips to improve the user experience design of Big Data applications. Designing around these tips will enable an organization to ask better questions, get better answers and make better decisions.

Design data experiences, not just visualizations

Data visualization is one of the most talked about UI components in the Big Data picture. These are the charts, graphs, maps, infographics, indices, etc. that help users to uncover a greater number of trends or patterns within the data. There is an abundance of information written on visualization design patterns and techniques. There are also many comprehensive data visualization and dashboard tools. But visualizations are only a small part of the user experience design required for Big Data analysis, answers and actions.

Simplify query/report UI to reduce bottlenecks 

Researchers and analysts (or their support staff) spend significantly more time searching for and organizing data than they do analyzing, visualizing and interpreting it. They spend even more time creating reports and presentations about their findings. The design of interactive and consistent query filters and faceted search matched to validated workflows can significantly reduce bottlenecks in iterative report/query cycles. Automated, templates or guided reporting UI can facilitate both novices and experts to publish documents and be more responsive to ad-hoc requests. 

Design rich but flexible search 

Good data visualization acts as a lens that lets users explore and filter their data intelligently and easily drill down or flexibly respond to ad-hoc queries. It provides the context for analysis. Faceted search is one approach we often use.

Facets enable users to flexibly break down a large set of results into bite-size chunks and then create their own custom search by combining multiple dimensions rather than forcing them through a specific, pre-determined path. They navigate based on the questions they are trying to answer and what the data reveals. When implemented well, faceted search provides insights into data and helps to clarify intent and direct next steps.

To implement faceted search it is necessary to understand the users’ mental models, as well as key use cases and the kinds of questions they might entertain. Then users validate how they interpret the taxonomies under real workloads. Poor choices in the design can just add more complexity and frustrate users, so user testing is essential.

Design for self-service 

With today’s consumer SaaS, a user can set up and use applications completely on their own. This level of usability and self-service is similarly expected for professional BI data discovery applications. The UI needs to enable any level of user from novice to expert to set up, understand and manipulate the application without significant training or support. This is particularly true for query and reporting where the most bottlenecks occur.

The value of self-service also applies to executive dashboards. Senior-level executives need the ability to interact and experiment with the data in real time instead of relying on static reports or PowerPoints created by staff analysts. Once the data is fixed into a report, the types of questions that an executive can ask are inherently constrained. Dynamic, easily understood, self-service dashboards solve this problem and increase confidence in any decision made.

In his talk “Mining the Oil of the 21st Century,” Arnab Gupta, CEO of Opera Solutions, said “The real battlefield ultimately, is how you will make it [data and visualization] usable to people on the front line, to a person who is not technical, who does not understand statistics, and yet this helps them do their job a lot better.”

Design for delight 

Dashboards and reports also need to be designed to elicit and encourage curiosity. Successfully exploiting the value in Big Data requires experimentation and exploration. C-Level executives are probably not used to thinking of BI software as being “fun to use,” but that is exactly how it needs to be designed.

Design around workflows, not data 

Most BI and data analytics/visualization applications are designed around data rather than user workflows. A user can get completely engrossed with collecting new forms of data, integrating data sets, building new reports or dashboards, and so on. Data and visualizations are presented as columns of numbers or charts to be put in a report, rather than as business questions that must be answered and decisions that must be made to move the organization forward.

A workflow-centric discovery and visualization application UI supports a complete workflow process from insight to action. Users should be guided to identify any problem and take action, all in just a few steps. The query UI itself should work to clarify intent and direct next steps.

Use suggestive visualization 

One approach to guiding users through data visualization is to present them with a UI that makes it easy to create variations based on selecting changes to the dataset, the transfer function, viewpoint or degree of interest. The interface then automatically generates a new set of sample visualizations.

These “suggestions” for exploration exploit the domain knowledge of the user without requiring statistical or visualization knowledge. For example, nothing may stand out when first looking at conversion funnel reports; but if one of the suggestions for visualization is a geographic map, insights based on location may start to appear. Ultimately, it’s all about visual experimentation — a combination of play, pattern recognition and domain insight.

Provide information in multiple modalities 

People comprehend information optimally in different learning styles. Some people are auditory, some are visual, some are kinesthetic, while others are mathematical.  In a Big Data BI/data visualization application, alerts can be an auditory signal directing the user to a particular action. Visual charts can give a quick snapshot of summary information. Numeric details can be an easy drill-down, etc.

A UI design that addresses multiple modalities makes it much easier for a broader range of people with different comprehension styles to use and manipulate the software.

Build in tools for annotation

This is not a complex idea, but any BI/data visualization application needs an easy way for people to add in their own annotations and comments. This can be critical for remembering “aha” moments or establishing bread crumbs for further discovery.

Build in tools for collaboration 

The best analysis is one that is easily shared and facilitated by different perspectives and viewpoints. Analytics software needs collaboration capabilities built in — anything from IM to share reports and annotations via email. This should be integral to the UI.

Mobile data analytics and visualization 

Mobile analytics applications are unique from desktop software both in terms of design and types of information. Tailor any mobile data visualization solution to the mobile device instead of trying to replicate or port the desktop or SaaS solution. Desktop browser-based solutions enable broad, thoughtful analysis. Mobile solutions are more “in-the-moment.” They need to be quick, agile, interactive and enable users to actually do something on the go. Build prototypes and test how users interact and what kinds of functionality enable them to make decisions on the go in real-world scenarios.

There are many factors that go into successful UI design for Big Data analysis and visualization. Whether you are developing your own in-house solution or customizing off-the-shelf BI/visualization tools, the 11 tips presented in this article will offer a solid foundation for turning raw data into actionable insights.

Paul Giurata is the managing partner for Catalyst Resources, a user experience and application design firm headquartered in Silicon Valley. He and his teams have worked on more than 450 software projects in Financial Services, SaaS, Life Sciences / Biotech and mission-critical systems.  For more information, contact

Copy link
Powered by Social Snap