Today we use mobile applications for almost every task – whether it is our alarm clock, watching Netflix or reviewing the latest report your boss sent. When companies develop a mobile application, it’s imperative customers find it easy to use. However, data from the World Quality Report 2016-17 showed that mobile testing is still “viewed as a relatively new skill” despite the ubiquitous use of mobile. The report found that the biggest challenge that organizations face with mobile testing is not having the right process or methodology in place, followed by a lack of mobile testing experts.
Mobile testing best practices will help you surge past mobile challenges
Mobile testing best practices can overcome these difficulties. The quality assurance (QA) team is often the forgotten part of developing mobile applications, but their work is critical in ensuring user expectations are met. This article highlights some typical challenges organizations face when developing a mobile app, how you can overcome them and details of what to include in your mobile testing plan.
Overcoming device and OS fragmentation
Device and operating system (OS) fragmentation is a major challenge facing QA teams. (Android Marshmallow, for example, is the OS most widely installed as of December 2016 but is found on just 26.3 percent of devices). This fragmentation is compounded by frequent device changes and OS updates. QA teams need to compare in detail how applications will behave on multiple devices with different versions of operating systems.
You can sacrifice coverage across device/OS combinations to an extent; but when you reduce the number of device types that you test against, you take a chance that your application might not work for some customers.
To handle the device challenge, you have three options: 1) test exclusively using real devices, 2) test exclusively with emulated devices, 3) use a combination of both.
The first step to define when you create your plan is to prepare the list of targeted devices that you need to test. It should include different screen sizes and characteristics (memory, processor, etc.).
In the test plan, you can include real devices and emulators. Using an emulator versus a real device has its respective benefits and advantages. Emulated devices are easy to manage, but their big disadvantage is that they lack characteristics that only a real device can provide.
Analyze performance under different network conditions
In addition to making sure the application works on different devices and operating systems, your QA team needs to verify how the application responds with different network conditions. Here are five main scenarios to test:
- Connection on Wi-Fi
- Only 4G connection
- Only 3G connection
- In airplane mode (or all connections deactivated)
- And, most interesting, intermittent network test scenarios that a user can find in the real world.
Four additional elements to include in your testing plan
Beyond assessing the application with different devices and/or emulators and analyzing the performance under different network conditions, we recommend reviewing the following four aspects.
1. Interruptions. Smartphones are all about multitasking. A QA engineer needs to verify how the app responds in a variety of situations such as when the device receives an incoming call, SMS, application alert, low-battery alert, calendar event, social network notification, alarm clock, or the user switches between applications or application switch.
2. Localization. Is the app localized? Testers need to test the app in different languages. Make sure that every string has been translated into the required language.
3. Battery consumption. Verify that the application does not consume excessive battery life.
4. Performance. It’s important to compare the released version with the current version to see if the performance is the same, better or worse. Install the app on older devices to see if the app still runs on them. Key performance aspects to review include:
- Test how the app reacts to incoming phone calls, SMS, MMS, tweets or other notifications. Check the battery drain while using the app.
- Verify the battery usage every 10 minutes to see if the app consumes too much power.
- Install the app on devices with a very low battery level and check what happens.
- Validate the app’s memory usage. If the app stores data on the local file system, test the usage of different memory cards. Consider what will happen when the local storage is nearly full – will the app crash or notify the user with a proper error message?
- Use the app on different carriers and network connections like GPRS, Wi-Fi or LTE.
Your QA team plays a critical role in ensuring the success of your mobile application.
When you develop a mobile app, your QA team is a critical part of the process and will help you define the right test plan. The QA team plays the crucial role of making sure your customers and users feel comfortable with the application and that they have a good user experience. Only with the right test plan can you ensure the successful release of your app.
Carina Valdez is mobile QA team leader and a certified scrum master at Belatrix. She has more than 12 years of experience in developing and testing software. She started her professional career as a functional analyst and system auditor. In 2007, Carina started leading QA projects at Belatrix and certified as a scrum master. Since 2011, Carina has been specializing in mobile device testing, focusing on iOS.