DEVOPSdigest

DevOps thrives on automation, and it's clear why: manual processes are slow, error-prone, and inconsistent. In practice, automation rates, especially for testing, are very low, so achieving effective test automation can be challenging, but there are steps that you can take to increase test automation in your organization.

Continuous Integration Needs Continuous Testing

Before diving into what you can do to increase testing automation, it's important to understand the intersection of continuous integration and continuous testing as both affect automation implementation.

Continuous Integration (CI) is a DevOps practice whereby code changes are pushed to a common code base many times every day, and are automatically integrated with the rest of the code base. This frequent merging activity carries the risk of destabilizing the code base and introducing defects into the software delivery pipeline.

To find out quickly if a change causes problems, teams use a Continuous Testing (CT) approach, whereby a code change automatically runs a set of tests, designed to provide speedy feedback to the developers.

Checking if a recent change breaks functionality that worked earlier is known as regression testing, and is an essential part of the CI pipeline, triggered whenever a change is committed, which can be many times each day.

In addition to regression testing, CT should check that new functionality works as expected, that it is easy to use, and is resilient to unexpected circumstances. It should also cover aspects such as application performance and security.

Continuous Testing Needs Test Automation

The team must get feedback on their changes as quickly as possible, but with so many tests to run, it is not practical to test manually. The solution is to automate the tests, so that they can run unattended, and complete tests quickly and reliably.

However, automating everything is not always possible. For example, evaluating the user experience of a new feature requires a subjective examination. We need to study and explore the new feature before we can determine if the user experience is satisfactory. Many teams turn to Exploratory Testing, which involves manually playing with the system, learning about it as you go, and applying the knowledge gained to conduct further testing.

Although the goal should be to automate whatever can be automated, there's no magic wand to "automate all the things."

Some of the challenges with test automation are:

■ Applications change frequently, requiring automation scripts to be constantly updated

■ Difficulty of provisioning stable test environments with appropriate test data

■ Lack of skilled automation engineers

Nevertheless, organizations are facing up to these challenges and looking to grow their automation. A word of caution though: Jez Humble and David Farley, in their book Continuous Delivery(link is external), recommend a gradual approach to automation, rather than attempting to automate everything at once.

Be Smart About Test Automation

Here are some tips to help you increase your test automation.

1. Start with tests that are small, focused, and run quickly

Tests should be small, independent, self-contained, check a very specific area of the code, and complete within a very short amount of time. These tests, known as unit tests, should be the first tests to run. You can remove dependencies on external or unavailable services and components by emulating them using a service virtualization tool. Once these tests have passed, more extensive testing can be performed.

2. Automate security testing and performance testing early on

Security testing and performance testing are often run late in the cycle, increasing the risk of serious issues being detected when they're more difficult to fix. To get quick feedback, design short load tests that run with each build, and include static application security tests that focus on the code that changed. You can run more comprehensive tests after the initial tests have run, or in parallel. The key is to provide valuable feedback as early as possible.

3). Put developers and testers on the same team, with the same tools

Automation rates increase when testers and developers work together. When they use the same tools and technologies for development and testing, communication becomes more efficient, automation becomes aligned with objectives, and issues can be identified and resolved more quickly. Consider practicing Test Driven Development (TDD), whereby tests are written first, and only then code is written to pass the test. This can reduce ambiguity, and ensure that developers and testers are on the same page.

4. Make your metrics visible to the team through a real-time dashboard

Establish a dashboard showing each build and pipeline, and what stage they are at. Include the status of tests that are running, which ones have passed and failed, and how long they are taking to run. Visibility into the pipelines through simple-to-understand, real-time graphs and charts on a central dashboard provides critical information that helps teams proactively identify and resolve issues.

What's Next for Test Automation?

The assertion that test automation isn't suited for in-depth explorations of a system is increasingly being challenged by artificial intelligence. AI and machine learning (ML) will help computers to "learn" how a webpage behaves, and develop, optimize, and maintain scripts that can test the page and how it interacts and behaves with the rest of the system — all without human intervention.

AI and ML are pushing the boundaries of automation. There will be significant advances over the next few years as the technology matures, blurring the lines between automated checking and intelligent testing.