DEVOPSdigest

Testing automation is in need of a revolution and so, we have published a manifesto for change. This manifesto makes some powerful statements about the direction we believe the test automation industry should be heading in. It's our case for change in the conservative world of testing. So, let's banish the QA winter and usher in the QA spring. A review of problems throughout a test lifecycle cements the need for autonomous testing.

The Test Lifecycle

■ Test creation: Scripts are still the main way of generating automated tests. Test recorders exist, but they just dumbly generate a new script from your interactions with the site under test. Usually, scripts have to be refined manually. In order to test cross-browser, you need to update your scripts. And testing responsive pages is even harder. As for nested DOMs(link is external), this is simply impossible for most systems.

■ Test execution: The majority of companies still maintain their own in-house test infrastructure, consisting of old, underpowered, end-of-life servers. Test teams use clunky systems, such as Selenium Grid, for managing orchestrations. So, tests can take hours to run, simply because of the inefficiency and age of the infrastructure. When (inevitably) things go wrong, the QA team finds itself relying on the DevOps team for help. Because infrastructure is local, backups of test results are probably not done effectively. And if anything goes wrong with the hardware you have no Plan B.

■ Test maintenance: In the depths of the QA winter, test maintenance is the bane of every test automation engineer's existence. Before Selenium, test maintenance just referred to updating test plans to keep up with changes to your site. Nowadays, every minor UI or styling change can break all your tests. This is a result of how brittle the selectors are in any script-based test system like Selenium. As a result, changing the border on a button can break tests; and, moving elements on the page is guaranteed to break things. We often hear of test teams spending 50 percent of their time just updating existing tests to reflect changes in the UI.

Testing Must be Strategic - Don't Leave it as an Afterthought

It's all too easy for management to overlook the importance of testing, until something breaks. At which point the test team becomes the target of everyone's ire. The problem with this view is it is rather short-sighted. If your test team failed to spot a bug it may not be a reflection on their performance. It is more likely to be a reflection on your lack of strategic focus on testing and quality management.

At the heart of this approach must come intelligent use of autonomous testing. We won't pretend autonomous testing can solve everything — sometimes you need manual and exploratory testing. Especially, if you are suffering from the curse of the unreproducible bug. But, used strategically, autonomous testing can save you from a lot of heartache and pain. It shouldn't be just an afterthought but should be one of the first things you think of when planning a new product.

Agile DevOps Requires Autonomous Testing

At its heart, DevOps is part of the Agile world. DevOps experts Len Bass, Ingo Weber and Liming Zhu have defined it as a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production while ensuring high quality.

In other words, DevOps teeters between developing and running. It's what you get when you apply the Agile approach to software development to deploying services in production.

A key part of this definition of DevOps is "ensuring high quality." As we all know, that is (or is meant to be) the definition of what a test engineer does. So, it's clear that there is a significant overlap between testing and DevOps. The second key part is the aim to "reduce the time" for a change to reach production. This is where autonomous testing becomes key.

In addition to the above considerations, our industry should apply these methodologies when it comes to the future of testing:

■ Plain English is the new test language

■ Tests should never break

■ Shifting testing left is not enough. Shift right

■ Software testing is about driving revenue, not just finding bugs

■ Quality is in the DNA of every great company

■ Autonomous testing is your brand's shield and protector

■ Intelligent automation just works

■ Testing must be cloud-first

The QA winter has endured for too long. It's time automated testing powered into the modern world. Here at Functionize, we are well on the way to creating a QA spring. A future where an intelligent test agent handles all your testing needs while freeing up resources, saving time, and driving revenues. A future where testing is a function everyone on your team can contribute to. Above all, a future where you can embed quality in everything you produce.