DEVOPSdigest

Sauce Labs announced the launch of Failure Analysis, a new analytics solution that applies sophisticated machine learning to customers’ pass/fail data in order to surface the most common reasons a given set of tests fail.

Failure Analysis gives developers, testers, and QA managers insight into how often the same type of failure repeats itself across a given test suite, enabling them to move quickly to address the most pervasive issues and drive rapid improvement in test quality.

“Testing is critical to delivering flawless applications and creating great user experiences, yet the task of maintaining a high-quality test suite has only become more difficult as organizations accelerate the pace of development and automate testing at scale,” said Matt Wyman, CPO, Sauce Labs. “With Failure Analysis, developers finally have a machine learning-driven tool to help balance the demand for faster deployments with the heightened need to maintain a reliable test suite. We’re excited to deliver this new resource to aid in the testing community’s collective efforts to improve test quality, increase developer productivity, and build digital confidence.”

The release of Failure Analysis comes as many development teams reach an inflection point in their digital journeys. Though more organizations than ever have shifted to agile development methodologies and implemented continuous testing throughout the software development cycle, poor test quality continues to undermine many efforts to deliver quality applications at speed. Poor test quality also increases the occurrence of false failures, frustrating testers and developers responsible for manually following up on those failures, and eroding confidence in the overall automated testing process.

Failure Analysis from Sauce Labs addresses this increasingly urgent need by helping testers and developers better understand why tests are failing and enabling them to prioritize remediation accordingly. Leveraging powerful machine learning algorithms to analyze an organization’s pass/fail data and its Selenium and Appium command logs, Failure Analysis gives testers and developers visibility into precisely where and how often failure patterns recur within a test suite, all from a single dashboard view within the Sauce Labs UI. Using the learnings provided by Failure Analysis to improve test quality, organizations can also dramatically lower the occurrence of false failures, ensuring that failed tests carry a high signal-to-noise ratio and developers feel confident that a failed test has truly detected a breaking change as opposed to the flaky noise of an unreliable test suite.

Failure Analysis is available now as a standard capability for enterprise customers as part of the Insights package within the Sauce Labs Continuous Testing Cloud. Sauce Labs provides a host of analytics capabilities that provide intelligent views into test data, including a new test overview and trends dashboard, interactive visualizations that enable customers to drill down on granular data sets, and a powerful REST API that allows users to aggregate data into their home-grown quality dashboard or test analytics tools.