Big Data

Q&A with Sumo Logic on Big Data Analytics and Cloud Trends

  • author image

Editor’s note: Most organizations’ cloud footprint is growing, including SaaS apps the organization may not be aware of, while the existing data center infrastructure is still in place. They need an analytics solution that logs machine data across cloud and on-premises infrastructure. Christian Beedgen, Sumo Logic’s co-founder and CTO, discusses differentiation in software solutions addressing challenges in monitoring cloud environments. He also predicts big changes coming in the next 18 months to Big Data analytics. 

SandHill.com: What are the two biggest mistakes organizations typically make when seeking an analytics solution? 

Christian Beedgen: We’re strong believers that the best analytics combine both machine learning and human interaction with those analytics. Often, organizations rely solely on the patterns without providing any input as to whether or not those patterns are relevant to their particular organization.  

A second big mistake is not fully understanding the most relevant use cases for an analytics solution. Do they need a big batch-based reporting tool? Do they need a solution that is optimized for finding “needles in a haystack?” Do they care more about analyzing data that is fresh off the wire or doing real-time queries on data that might be a few days old?  

SandHill.com: What are some of the comments your company has heard over the past 6-12 months from potential customers regarding the Big Data log challenges they have with their cloud environments, which are driving them to consider your company? 

Christian Beedgen: We’ve seen a few different patterns over the past year.

  1. Companies want to be able to monitor and troubleshoot logs that cover the full application life cycle, from development to production.
  2. They need to support both internal and regulatory compliance needs (e.g., PCI), which means they need to centralize and manage logs across their entire infrastructure.
  3. Finally, they see opportunity in providing a better customer experience by tracking Web/access logs that provide insight into how customers are interacting with their organization. 

SandHill.com: Please give an example of how Sumo Logic helped a customer determine the root cause of poor performance of an application. 

Christian Beedgen: We have a customer whose business is based on sponsored ads via its search application. Its search application was throwing off 404 and 403 errors that led to both a poor customer experience and diminished revenue.  In addition, its lack of visibility into what was happening with its logs led to either overpaying or underpaying royalties to its search partners. 

With Sumo Logic they were able to centralize all their logging into a single infrastructure and use our LogReduce™ technology to detect patterns across both Web and application server tiers. This enabled the company to more quickly identify and resolve issues which, given its business model, had an immediate impact on its bottom line. 

SandHill.com: Please explain how the Sumo Logic Elastic Log Processing engine is a market differentiator in achieving greater ROI for customers. 

Christian Beedgen: Log data can “burst” at any time, whether driven by seasonality, new product launches or very specific infrastructure issues. In a typical on-premises solution, companies need to provision hardware, storage and software for the maximum log volume that might come through the system.  

In contrast, the patent-pending Sumo Logic Elastic Log Processing engine allows companies to provision for the mean so that the compute capacity for any particular client will increase and decrease as their volume requirements change, which optimizes overall performance.  

For example, we have an e-commerce customer whose business is extremely seasonal around the winter holidays. By using Sumo Logic they have the capacity to handle the 5x growth in logs they see during the months of November and December while still ensuring the same customer experience in terms of response time, etc. 

SandHill.com: You and your co-founder, Kumar Saurabh, have a background in software security. Kumar has stated that security at scale is difficult; it increases the exposure area if it’s not designed correctly. Please explain how Sumo Logic helps customers address this cloud issue.   

Christian Beedgen: Sumo Logic is uniquely positioned to centralize log and machine data across cloud and on-premises infrastructure. This is what we see in many enterprises today: there’s a cloud footprint that is growing, while at the same time the existing infrastructure in the data center is still in place. We also see increased use of SaaS platforms, often by lines of business without direct knowledge of IT.

In other words, there simply isn’t anything like a perimeter anymore behind which to hide. The enterprise is doing business in the cloud, knowingly or unknowingly. We’ve completed the security certifications and examinations that assure we keep data secure whether it’s at rest, in use, on the premises or in the cloud. Sumo Logic can collect and analyze data from everywhere because the data is everywhere. Doing it securely helps our customers deal with modern realities. 

SandHill.com: What changes do you believe we’ll see in Big Data analytics capabilities over the next 12-18 months? And how will these changes affect the vendor landscape? 

Christian Beedgen: Hadoop has clearly set the bar in terms of available and affordable batch processing. We think there is tremendous value there simply because there is so much more data being generated everyday that doesn’t get covered by the enterprise data warehouse. Agility is a must, and Hadoop shines and will continue to be the torchbearer. 

We predict several trends around this. First, efforts such as Cloudera’s Impala will get Hadoop closer to the classic enterprise data guys because it can speak SQL. 

Secondly, more and more efforts are being made to move from batch processing closer to real-time processing; Twitter’s Storm is a good example for this approach. We will see more developments in this space that are not Hadoop based but will have to have an integration story with batch-based Big Data. 

Finally, even with Hadoop’s relative accessibility, it still requires the provisioning of hardware in the widest sense. More and more companies will build solutions on top of the Big Data ecosystem that are delivered as services, further democratizing the access to analytics of large scale. Amazon EMR is an obvious example for broad use case; Sumo Logic is a perfect example for delivering a Big Data analytics service that focuses on domain-specific use cases, such as operational intelligence. 

Sumo Logic was a collaborator in the 2013 Future of Cloud survey hosted by North Bridge Venture Partners, 451 Research and GigaOM. Click here to view the results of the survey. 

Christian Beedgen is CTO and co-founder of Sumo Logic. Previously he was chief architect at ArcSight where he led teams to deliver the core ArcSight product line across server infrastructure and user interaction. Christian was integral in designing and developing core product features including a flexible framework for real-time query processing. Before ArcSight, Christian co-founded Gigaton, an infrastructure software company enabling a distributed file management solution over IP-enabled networks. Contact him at christian@sumologic.com  and follow him on Twitter: @raychaser. 

Kathleen Goolsby is managing editor of SandHill.com

Comments

By Patrick Taylor

In my view, layers of services will definitely evolve as part of Big Data. As described by Christian Beedgen, a layer of services that provide analytic capabilities makes sense. That will serve the people that understand how to perform analysis, how to correctly apply analytic techniques enabled by the services Christian described. This evolution allows more people to leverage the power. The next level of evolution will be to develop services that deliver analysis. This will serve the larger set of people who know they need to ask the questions but have neither the time nor the skills to perform the analysis – they just want the answers. As we move up these levels of services we lose flexibility, you can ask more questions at the analytics layer than at the analysis layer, provided you know how to do the analysis. With the well documented shortage of “data scientists” it’s important to provide the leverage of these higher layers. There are many places where there are known questions that can be answered with analytics and these are the places that can be best served with an analysis service.

Post Your Comment




Leave another comment.

In order to post a comment, you must complete the fields indicated above.

Post Your Comment Close

Thank you for your comment.

Thank you for submitting your comment, your opinion is an important part of SandHill.com

Your comment has been submitted for review and will be posted to this article as soon as it is approved.

Back to Article

Topics Related to this Article