Skip to main content

How to Evaluate SaaS Vendors: Five Key Considerations

By February 6, 2012Article

How important is infrastructure in a SaaS solution? It’s crucial. A cloud hosting provider’s outage, for instance, prevents companies from having access to their SaaS services. Yet, many enterprises and midsize companies fail to pay adequate attention to the solution’s underlying infrastructure when evaluating SaaS vendors. They fail to ask questions that help in making infrastructure tradeoff decisions that are necessary when selecting a SaaS solution.
I have met with many software companies that are in the process of converting their applications to SaaS solutions or are really ramping up in deploying their SaaS solutions. They tell me that security, service level agreements (SLAs), disaster recovery, and other important infrastructure aspects are questions that seldom come up when potential customers evaluate them.
Buyers tend to evaluate SaaS differently from other technology solutions because of its subscription model. Uppermost on their minds is the fact that all they have to do is point, click and buy SaaS for a specific number of users, and they’re off to the races. SaaS is so easy to get up and running quickly, and companies view it as a simple procurement that doesn’t need in-depth evaluation or considerations of the underlying infrastructure.
But what happens if the SaaS vendor goes out of business? What if their site goes down? At Savvis, we use services from several small SaaS companies, so we have back-up plans and other solutions in place should a company go out of business. It’s easy to get a SaaS solution up, but buyers need to think about the impact to their business if that solution is no longer available.
In addition, there are five aspects of underlying infrastructure that buyers should evaluate before selecting a SaaS solution.
1. SaaS buyers’ biggest pitfall: disaster recovery
I am amazed at how many software vendors don’t have a disaster recovery site. And many that have a DR site don’t know how long a disaster will affect their business. Buyers should ask potential SaaS vendors these questions:

  • How do you test your disaster recovery process and procedures?
  • How often do you test?
  • What is your recovery time?
  • Is your infrastructure dispersed; are your primary site and your disaster recovery site geographically separated?

2. Security
It’s not unusual to find that a SaaS vendor does not utilize enterprise-grade infrastructure to deploy SaaS applications. As an example, the firewall may be a software firewall or may be a custom implementation. Security protecting the SaaS solution needs to be holistic.
Another factor to consider is that many SaaS solutions involve multiple providers. There may be an Internet provider, a firewall provider and four or five others in the mix. When a problem occurs, there will be finger-pointing. It’s the classical issue that on-premise IT has always had. There is no easy way to address this up front. The solution is to limit the number of vendors so they take more responsibility over performance rather than having the opportunity to point a finger in any direction when there are many vendors. Alternatively, buyers can work with companies like Savvis that manage these issues for them.
Questions to ask the SaaS vendor in the solution evaluation phase include:

  • Is your company SAS70 compliant? (and PCI compliant for credit cards, HIPAA compliant for healthcare)
  • What security guidelines and audits does the colocation or hosting provider follow?
  • What security is in place at the colocation or hosting provider’s facilities?
  • Who manages network connectivity, firewalls, log file management, web application firewalls and access and identity management?
  • Does your company have an Internet backbone network giving you visibility into emerging threats, thus facilitating quick protection of the SaaS solution?

3. Flexibility and quality of service
Buyers that don’t carefully evaluate the infrastructure aspects of a SaaS solution are at risk for selecting a provider with limited capabilities and ending up with a solution that is too rigid to integrate with back-end legacy systems or offer additional services or more storage.
Many SaaS providers do their integration and data exchange from Web server to Web server over public Internet on a secured or unsecured port. The quality of service via public Internet is “best effort” at the network level and thus the provider cannot ensure high quality.
In contrast, if the buyer and the provider use hosted services, such as from a company like Savvis, the data exchange takes place over a “cross connect” or network cable between both companies in the Savvis data center. The advantage: they’re not on the Internet and can accomplish the data exchange at a higher quality of service and with service level guarantees.
Buyers should ask potential SaaS providers: “who provides the infrastructure for your SaaS solution?” If the answer is a company using the Internet for data exchange, the quality of service will be best effort and not guaranteed even by paying a higher price.
Buyers that are already using a SaaS vendor with limited flexibility can work with their provider to purchase a private VPN line and move the integration off a public network.
4. SLAs
Buyers usually believe that they’re getting an SLA from a SaaS company (three, four or five 9s of availability), only to learn later that their SaaS provider did not work with an infrastructure provider to correlate SLAs so that it can meet the buyer’s requirements.
They may have only one network connectivity or only one firewall established with the infrastructure provider, and in the case either one fails, the SaaS application goes down. Buyers need to evaluate the architecture to be sure the SaaS solution has adequate redundancy built in for load balancers and routers in case of a failure.
Typically, SLA discussions occur during negotiation and finalization of the agreement, but this is really too late. Buyers need to identify SLAs earlier in the evaluation process and also ensure the SaaS SLAs are also being supported by the cloud provider hosting the solution.
5. Global reach
Global companies need to ensure their SaaS provider has a solution in the countries that meet the buyer’s existing and expanding geographic needs. This is especially important where users need fast response times or where data storage and access is restricted in some countries.
Eliminating the procurement mindset for SaaS
As I pointed out earlier, enterprises tend to overlook the infrastructure aspects of a SaaS solution and view it as a simple procurement that doesn’t need in-depth evaluation. How can companies change this mindset and prevent the risks I’ve described?
When it’s easy for companies to point, click and buy a SaaS solution, they take IT out of the picture. IT’s legacy is dealing with all these infrastructure issues. Bringing IT back into the picture in a new cadence of how to evaluate SaaS solutions going forward would be a much better way of purchasing SaaS because it’s not throwing the baby out with the bathwater, so to speak.
IT can then say, “Let’s look at the security, the flexibility, the underlying cloud infrastructure and how it will fit the business requirement.” Involving IT in the evaluation is far better risk mitigation than just having the business requirements person, head of sales or marketing or the operations person clicking and buying.
Larry Steele is technical vice president, software-as-a-service, at Savvis, a leading provider of Saas infrastructure services.

Copy link
Powered by Social Snap