Testing on Mobile and Smart Products
March 28, 2016

Kevin Dunne
QASymphony

Mobile and Internet of Things systems are a popular yet nascent industry subject to the traditional growing pains of innovative sectors, requiring distinct testing approaches.

When it comes to testing, software is software, right? Not exactly. Mobile apps keep growing, and now with the predicted growth in Internet of Things applications and technologies, testing organizations have new considerations and limitations. If your company is involved in developing software for the Web, mobile devices and IoT platforms, you may need a few different types of testers, tools, and process to succeed. This blog outlines some top considerations for testing on both mobile and IoT applications.

Mobile Everywhere

Mobile apps succeed when they have a simple, elegant design, run fast and meet customer expectations. Behind the scenes, however, developing effortless mobile applications is not easy or simple. Compared with Web applications, mobile apps are affected by a wide range of variables, including: network diversity, battery life, location and the speed at which users are traveling. The more variables, the more testing scenarios a company must account for in testing. Real-world testing is generally accepted as imperative at some point in the process, given the array of devices and networks that people are using when they open a mobile app. Mobile apps primarily focus on two core issues: the user interface/experience and the quality, security, and speed of the connection from the app to the server.

Automated tools for mobile application testing have become much more reliable, making the job of the tester a bit easier today. Even so, testers must take into consideration all the versions of iOS, Android and Windows-based phones and tablets in their user base; it’s not economical to thoroughly test them all. Focus your testing efforts on the most popular devices and use cases in your user base and from there, the most-often used and critical path features in the application.

Due to the need for real-world testing — across all those devices, networks and locations mentioned earlier — beta testing is used heavily to get economical coverage across the potential mobile variants. Where companies often fail, however, is in properly managing the beta testing process. Beta users need guidance so that they will interact with the desired features and tasks, and in turn, provide useful feedback. To fill the gap in incomplete beta testing, companies can take videos of users testing an app to track where they often get stuck or confused. Crowd-testing services can help by managing the process of crowd sourced testing, cost-effectively increasing device test coverage.

Internet of Things on the Rise

Testing IoT or hyper-connected applications requires knowledge of both hardware and software — and the intersection between the two. IoT applications incorporate the hardware of the smart product and the software that controls it along with the user applications that interact with the product. That’s a lot of dependencies and a lot of pieces to test. When troubleshooting an issue or researching a vulnerability, a tester must investigate the mobile and enterprise applications that collect, analyze and display the IoT data, the various network connections across which that data travels, the IoT appliance’s hardware and its software.

Therefore, testing for IoT systems usually requires more manpower and time—along with more specialized resources. Individuals who develop and test products such as smart home security systems or thermostat don’t often have the same skills as the consumer app testers, and vice versa.

What IoT testing also requires is deep investigation into the “what-ifs” and corner cases, or those problems which occur outside of normal operating parameters. Take a home security system: if there is a power outage and the appliance doesn’t restart automatically when the power comes back on and no one is around to do it manually, that could cause a problem.

If a smart product breaks, it’s not a simple matter of uninstalling and reinstalling a new piece of software on the fly. If there is a suspected hardware problem, the customer usually needs to return the device for repair or replacement by the manufacturer. Developers and testers should aim to battle test IoT products more stringently than a mobile app, to avoid these situations. The goal for testing is to not only eliminate or minimize these risks but also create self-healing capabilities within the products.

As with mobile apps, beta testing is important in IoT to gain the real world feedback that’s not as credible in a lab test environment. Beta testing smart products is considerably more expensive than releasing cheap software code, due to the cost of the appliances, which means the beta community will likely be much smaller. Companies may opt to use internal employees as beta testers, allowing them to more easily collect, diagnose and repair troublesome devices.

As with mobile apps, IoT testers should identify and target the highest-risk scenarios for their testing plan: those which focus on user experience and stability. IoT apps should also be configured to collect and deliver more alerts and logging data than mobile apps, due to the higher number of components involved. This will make it easier to prevent, troubleshoot and fix problems when they arise.

It’s an exciting time to be a software tester today, due to the ever-growing array of software types and products that run software, all of which need quality assurance expertise. Testers have the ability to grow their careers quickly, through developing a deep understanding of these new applications and smart products which tightly integrate hardware and software. Companies hiring and managing these testers have the opportunity to create new best practices, which once finessed, can push their product to the top of the heap. After all, no matter how cool an app or smart product – if it’s clunky, users and customers won’t stick around for long.

Kevin Dunne is VP of Strategy & Business Development at QASymphony.

The Latest

May 17, 2018

The top two business priorities for CIOs of midsize enterprises (MSEs) in 2018 are growth and digital transformation. However, 57 per cent of MSEs are not yet delivering digital initiatives, according to findings from Gartner Inc.'s 2018 CIO Agenda Survey ...

May 15, 2018

Almost every company is facing the challenge of digital transformation today. This means rethinking and retooling your company to compete and succeed in an increasingly digital world. While digital transformation is not only about technology, the right tools can help. To find out what these right tools are, APMdigest asked experts from across the IT industry for their opinions on the essential tools to support digital transformation ...

May 08, 2018

With data breaches consistently being in the news over the last several years, it is no wonder why data privacy has become such a hot topic and why the European Union (EU) has put in place General Data Protection Regulation (GDPR) which will become enforceable on May 25, 2018, which is less than a month away ...

May 03, 2018

The prospect of increased workloads, combined with shrinking mainframe skillsets, has huge implications for mainframe DevOps. The only way for organizations to solve this skills gap crisis is by optimizing developer productivity. Drilling down a level further, what does this all mean for mainframe DevOps? ...

May 02, 2018

When it comes to operations and development, DevOps has changed the traditional compartmentalized style of development by eliminating silos. But what about the security team? Security is largely still siloed from operations and development. No doubt, many DevOps teams have some security controls baked into their automation processes, but a recent survey shows there are still alarming gaps ...

April 30, 2018

According to the 2018 Global Security Trends in the Cloud report, 93 percent of respondents faced challenges when deploying their current on-premises security tools in the cloud, and 97 percent lacked the tools, cross-functional collaboration and resources to gain proper insight into security across the organization. These numbers indicate a big problem in DevSecOps that needs to be addressed ...

April 26, 2018

Moving more workloads to the cloud is a top IT priority, so eventually it will be time to consider how to make those critical legacy applications cloud ready. In Part 1 of this blog, I outlined the first four of eight steps to chart your cloud journey. In addition, consider the next four steps below ...

April 25, 2018

Clearly, moving applications to the cloud delivers significant advantages. So what's standing in the way of full cloud adoption? For many companies it's those burdensome (but critically important) legacy applications. Moving more workloads to the cloud is a top IT priority. So, eventually it will be time to consider how to make those critical legacy applications cloud ready. Consider the following eight steps to chart your cloud journey ...

April 24, 2018

Developers and engineering teams are under increasing pressure to release higher quality software faster. Continuous testing has proven to be central to these efforts as it helps eliminate bottlenecks and ensures that automated testing is a constant throughout the development process, not an exercise relegated to the "last mile." The value of automated testing is more evident than ever before, with nearly half the respondents reporting that management is fully committed to automated testing and with plans to increase spending, according to the recent Sauce Labs Testing Trends for 2018 report ...

April 19, 2018

As development speed has become a competitive advantage, the DevOps team has sought to enable continuous integration and continuous delivery (CI/CD). For the CI/CD process to be successful, it must be fast and efficient. Any potential roadblocks that delay any part of the process increase cycle times and slow down delivery ...

Share this