Exploratory Testing Cuts Risk of Software Failures
July 31, 2017

Wayne Ariola

Agile development compresses software testing cycles, jeopardizing risk coverage and opening the door for software failures. Here's what you can do:

The adoption of Agile and iterative development processes is forcing testing teams to conduct and complete testing more rapidly than in the past. Teams that previously had weeks or months to test must now accelerate testing to deliver even more comprehensive test results in a matter of hours or days. Today, testing must be performed under intense time pressure — often with reduced resources and budget. And that spells R-I-S-K. After all, how comprehensive can your testing be under such duress, and what is the risk of failure once the software goes into production?

A recent survey of over 2,400 respondents revealed that many enterprise testing teams are adopting exploratory testing in response to these challenges and risks.

Among these respondents, exploratory testing is increasingly being used to evaluate how an application performs from the perspective of the end user. Exploratory testing is often contrasted with formal testing, which focuses solely on verification (i.e., whether the acceptance criteria outlined in requirements specifications have been met). That's called validation. As such, formal testing monitors known risks, whereas exploratory testing focuses primarily on analyzing potential risks.

Verification and validation are independent procedures used together to check and confirm that a product meets the requirements and specifications and that it fulfills its intended purpose.

Because exploratory testing does not require laborious upfront planning, teams commonly apply it to start testing new software functionality as soon as it's completed. This promotes rapid defect detection within the compressed development cycles that are the norm today. And, because exploratory testing encourages branching and exploration of different testing ideas in a way that simulates the end user's perspective, it tends to uncover, and therefore snuff out, more critical defects than formal testing.

But why has exploratory testing become so widely adopted? And how are testing teams using it? Take a look at selected findings from the survey for more insight.

1. Agile Has Become the Primary Driver for Exploratory Testing Adoption

87 percent of respondents use exploratory testing to accelerate agile development cycles by providing feedback as quickly as possible to all parties concerned (e.g., development, business, and operations). Agile processes require teams to react quickly to changes and adapt accordingly. This is valuable since rapid feedback enables teams to “fail early,” when the failure can be remedied before a system goes into production.

2. Exploratory Testing Supplements Test Automation

91 percent of the respondents who are actively adopting or practicing DevOps consider exploratory testing a critical practice for risk reduction. More than 9 out of 10 respondents state that it is crucial to combine test automation and exploratory testing in a fast-paced development environment. To prevent process bottlenecks, teams use risk coverage criteria to select the most powerful set of automated tests to run at the various stages of the software delivery pipeline.

3. Exploratory Testing Accelerates Defect Detection

Respondents who practice exploratory testing estimate that, by exposing defects earlier (when defects are easier to eliminate), they accelerate delivery by approximately 20 percent. Exploratory testing exposes many defects that would otherwise be overlooked until real users encountered them in production.

4. Exploratory Testing Uncovers Types of Defects Overlooked by Formal Testing Techniques

Respondents practicing exploratory testing report that the top three issues exposed by exploratory testing are (95 percent) usability issues such as confusing interfaces or inconsistent usage patterns (95 percent); missing requirements (for example, functionality that is critical for the end user experience, but was not originally specified (87 percent); and problems with functionality that was implemented beyond the boundaries of specification, and thus not covered by specification-based tests (85 percent).

5. Exploratory Testing is Popular for Testing Usability

The most frequently software characteristic tested by exploratory testing is usability (93 percent), followed by performance (77 percent), security (62 percent), stability (54 percent), and safety (40 percent).

6. Exploratory testing is Geared for User Acceptance Testing, Regression Testing, and Smoke Testing

95 percent of respondents actively practicing exploratory testing state that exploratory testing is applied during user-acceptance testing, followed by 72 percent during regression testing, and 37 percent during smoke testing (testing that comprises a non-exhaustive set of tests to determine if a build is stable enough to proceed with further testing).

In any approach to software testing, the objective is to eliminate risk of software failure. With exploratory testing, software testers now have a potent addition to their testing regimens.

Wayne Ariola is CMO at Tricentis

The Latest

September 21, 2017

There are many options when it comes to container orchestration platforms and services. Figuring out which one is best for a particular organization’s needs and applications can be a challenge. Which platform for containerization you choose can significantly influence your business success, so the selection process should be carefully considered ...

September 20, 2017

A powerful tool for simplifying DevOps is containerization, which delivers a convenient form of application packaging, combined with the opportunity to automate certain IT provisioning processes. With containerization, DevOps teams can focus on their priorities — the Ops team preparing containers with all needed dependencies and configurations; and the Dev team focusing on efficient coding of an application that can be easily deployed ...

September 18, 2017

Web development and web design are intertwined in such a way that there is not one without the other — not anymore at least. The following outlines 5 benefits of collaboration ...

September 14, 2017

Mastering modern software development by building a "Modern Software Factory" is at the heart of business success in the digital economy, according to the results of a survey of over 1,200 IT leaders released today by CA Technologies ...

September 13, 2017

IT-Business convergence is needed to deliver continuous change, but many of the current tools add complexity and fail to merge the two, according to the Panaya 2017 State of Functional Testing Report ...

September 11, 2017

Application Program Interfaces (API’s) represent an effective way to build and manage mobile services. By using APIs — a set of routines, protocols and tools for building software applications — application developers no longer have to buy technology software or hardware. Instead, they can simply plug into a growing open ecosystem of API-driven services. It is simple to integrate, and saves time and money for new developers ...

September 07, 2017

More than a quarter of enterprises globally have not built, customized or virtualized any mobile apps in the last 12 months, according to the latest mobile app survey by Gartner ...

September 06, 2017

The number of malware breaches (to use a generic term) are rising in near exponential numbers and, unless there are radical changes, this is set to continue unabated. Most pundits agree with this forecast ...

September 01, 2017

DevOps encourages communication and collaboration between development and operations teams. Achieving greater synergies between the Dev and Ops teams doesn't happen overnight, but it is possible to fast track the process with the right technologies in place. One such technology is IT automation ...

August 29, 2017

Newly released data shows that distributed denial of service (DDoS) and web application attacks are on the rise once again, according to the Second Quarter, 2017 State of the Internet / Security Report released by Akamai Technologies ...

Share this