DEVOPSdigest

IT-Business convergence is needed to deliver continuous change, but many of the current tools add complexity and fail to merge the two, according to the Panaya 2017 State of Functional Testing Report.

Panaya commissioned Spiceworks to survey 154 IT testing decision-makers in the US and the UK, in order to gain insights regarding current practices, trends, challenges and perceptions of business process testing.

According to the survey, the majority (53 percent) of testing leaders believe that making their organizations more agile is critical to defining their testing methodology and two out of three testing leaders surveyed stated that QA and testing are highly important to their organization's overall strategy. Yet the lack of visibility into testing processes and reliable production data is creating a disconnect between IT and the business that is challenging IT's ability to keep up with rapidly changing requirements.

The survey shows that testers' key performance indicators (KPIs) have shifted with "Understanding Business Requirements" seen as most critical to defining test methodology, followed closely by "Improving the Customer Experience" by reducing the amount of software defects. This trend is expected to intensify in the future, as over one third of testing leaders believe that business domain expertise will become a requirement for testers with the growing effects of agile and continuous delivery.

Despite the importance of understanding the business domain, there are gaps in IT organizations' readiness for continuous delivery, as almost half (47 percent) of IT leaders surveyed believe that their tests do not accurately reflect real-life production scenarios. In fact, creating testing scenarios that accurately reflect business processes is currently the top test planning challenge.

The survey findings further demonstrate that while testing decisions and processes are driven by the understanding of business requirements to improve customer experience, 40 percent of respondents claim to be challenged by the lack of business oriented testing reports. In fact, when asked about their top test execution challenges, most testing leaders reported tools that are unfriendly to non-technical testers and require intensive training are hindering successful completion of functional testing.

To make up for lack of preparedness, testers often overtest resulting in wasted resources and unnecessary project delays. Close to one third of testers admit to always testing more than needed to ensure quality.

Dissatisfaction with current solutions and the need for better alternatives can account for the low percentage of organizations (12 percent) relying solely on existing centralized testing solutions, in contrast with the percentage (42 percent) of organizations who have purchased these solutions.

These results are consistent with Gartner's assessment with the state of test automation tools. According to the Gartner Magic Quadrant for Software Test Automation, “There is a growing number of vendors in the test automation market as technology evolves (cloud, mobile, Internet of Things [IoT]), practices change (agile, lean, DevOps) and business moves to utilize technology as part of a digital business disruption to the market. These create opportunities for new and small vendors and new offerings to automate different layers of the release pipeline. Many solutions currently don't meet the criteria of this evaluation."
Gartner, Magic Quadrant for Software Test Automation, 15 November 2016