DEVOPSdigest

In the fast-paced world of software development, development organizations are under significant pressure to evolve how they build, test, and deliver new products. Internal changes, like a growing reliance on APIs and other third-party components, are increasingly impacting the customer experience, threatening to hurt user retention if quality doesn't evolve.

The question is: how are these teams coping under such pressure, and how does that ultimately translate to customer delight? The 2022 Testing in DevOps Report, which surveyed nearly 600 DevOps and QA professionals, wanted to understand these challenges as well as how the role of software testing and quality engineering (QE) are changing the adoption of DevOps practices.

Cultural Issues Stymie DevOps Progress

The report discovered that cultural issues such as leadership, management, and the ability to foster collaboration within software development teams are the most vexing DevOps challenges, a repeat of the 2021 findings. 65% of respondents cited cultural challenges as their primary obstacle. Interestingly, budgetary and technology limitations were not among the main sticking points, but rather "human factors" such as the slow process of change (26%), and a lack of prioritization (15%) dominated respondents' most significant frustrations.

But as software development teams struggle with change and leadership, adding to these pressures is another reality: customer satisfaction is directly related to the speed of development cycles. The study found that customer satisfaction for development teams deploying rapidly was twice as likely to be rated as good or excellent. This bodes well for any organization battling to acquire new customers and build lasting relationships — but also keeps the pressure on for many teams struggling to accelerate development cycles.

As we enter 2023, there will be a greater emphasis on connecting DevOps progress to improved customer experiences in ways that not only help teams sustain transformation momentum, but also help organizations navigate economic uncertainty with greater efficiency and higher customer loyalty. Therefore, the need to understand what processes and behaviors can help these teams succeed is increasingly important.

Quality Engineering and Automation Are Key Pillars to Success

The study iterated the importance of quality engineering (QE) as a strategic pillar of development. QE, a discipline that includes quality testing throughout the development lifecycle to deliver positive user experiences, has been shown to help software development organizations build faster while also improving customer happiness.

The report found that teams with high test coverage were more likely to have smoothly running development pipelines. For example, teams with good/excellent test coverage were 2.5 times more likely to find bugs in the earlier stages of development and three times as likely to resolve bugs in 24 hours or less. These streamlined development processes result in lower stress deployments, which indicates that teams are accelerating product velocity with confidence.

Test coverage was also highly correlated with pipeline automation. Teams with fully automated pipelines were seven times more likely to rate test coverage as good/excellent, aligning with overall the overall trend of successful DevOps adopters being those who matured their technological, quality, and cultural practices in tandem. The connection between test coverage, pipeline automation, and customer happiness showcases how quality professionals can lead critical transformations that serve the users, the product, and their team.

Time to Resolve Bugs

The time to resolve bugs captures how well an organization can collaborate and respond to change. Not surprisingly, the report found that the time needed for a team to resolve defects was a strong predictor of test coverage, pipeline automation, and customer satisfaction. Not only were teams with good or excellent test coverage 2.5x more likely to find bugs in the earlier stages of development, they were also three times more likely to resolve bugs in 24 hours or less. Once again, respondents reported improving collaborative processes and cultural challenges were closely connected to high test coverage.

Accelerating Deployment Frequency

75% of respondents reported that their organization increased deployment frequency in the past year, a key measure of DevOps progress. But how significant that increase was strongly correlated with DevOps transformation and automation adoption. Those with heavily automated pipelines with mostly DevOps development practices were far more likely to report doubling deployment frequency in the past year than those with few automated workflows, suggesting that automated pipelines support faster deployments.

But as deployment speeds accelerate, maintaining an optimal level of test coverage becomes a challenge. Every time a new feature is introduced, quality teams need to adapt and expand their testing efforts to ensure all likely customer journeys are functional. Efficient testing is key to maintaining quality as deployments happen more frequently. This allows teams to move faster with confidence and ensures that new features will improve the user experience.

Teams that accelerated deployments by 50-100% were more than 3x as likely to have good or excellent test coverage. Despite the fact that these organizations deployed more frequently — potentially increasing the risk of bugs slipping into production — they were able to exceed customer expectations for quality.

Tackling Shared Challenges Improves DevOps Processes

A common theme through the report: DevOps leaders are those who can successfully navigate shared challenges to improve collaboration, quality, and product velocity. Few components of the development pipeline illustrate this better than testing. 48% of teams said that testing and QA are very important, while 25% said that testing is a shared responsibility across the organization. Those teams were more likely to have high test coverage and the ensuing benefits of faster, more confident development cycles. Organizations who can evolve their toolset, processes, and people continue to be DevOps leaders and those best positioned to retain customers in an increasingly competitive digital economy.

When quality is elevated, the entire software ecosystem feels the impact. This means that deployments happen more frequently, with less stress, and fewer bugs. Developers can build new products with more confidence, quality teams are empowered to improve the user experience, and consumers benefit from delightful, inclusive digital experiences that make daily life easier.