Testing Trends in 2017
Global Study of Developers and Testing Professionals Reveals Increased Desire to Speed Deployment Times, Lack of Progress in Speed of Fixing Bugs
February 16, 2017

Developers and testing professionals intend to speed up build deployment times even further — as 28 percent of those surveyed say they want to deploy hourly (up from 18 percent in 2016) — according to an independent survey, Testing Trends in 2017: A Survey of Software Professionals, conducted by Dimensional Research for Sauce Labs. Last year only 8 percent surveyed said they were not deploying as fast as they wanted to, but that percentage has nearly doubled to 14 percent this year.

Per the survey, teams are deploying builds faster, with 14 percent reporting hourly up from only 10 percent last year.

The survey also revealed a lack of process in fixing bugs once they have been identified. No progress was made at all in the number of development teams that can fix bugs immediately once they are identified, which stayed flat at 23 percent in both 2016 and 2017. Only minor progress (51 percent, up from 48 percent last year) was made in those who fix bugs within a few working days after they are identified.

"Faced with increased market pressure to compete in today’s digital economy — where every business is a software business — it's no surprise that more development teams want to release software faster,” Charles Ramsey, CEO of Sauce Labs concluded. "What is surprising from this report is the revelation that the speed in fixing identified bugs has remained stagnant. There is huge opportunity for teams to leverage automated testing to identify bugs earlier in the release cycle, immediately evaluate the bug with video playback and other assets, and share those test results across teams to fix them as quickly as possible.”

Additional key findings from the report include:

Still Ample Room for Testing to Improve

■ 87 percent use test automation tools, but 42 percent are still mostly or entirely manual testing

■ Some good news for automation with 32 percent being mostly or entirely automated with their testing, up from 26 percent in 2016

■ 74 percent fix bugs within a few working days, up only a tiny amount from 71 percent in 2016

■ Only 1 in 4 report tests are written each time new code is checked in

Decrease in Breadth of Cross Browser Testing

■ Testing on Chrome and Edge increased while testing on all other browsers decreased

■ 49 percent test on only the most recent browser version, up from 27 percent in 2016

Other Trends: Cloud, DevOps, Agile and More

■ 52 percent are using public or private cloud for testing, up from 43 percent in 2015

■ Use of cloud for spillover testing (19 percent up from zero percent) and specific types of tests (54 percent up from nine percent) increases dramatically

■ Only 10 percent have fully embraced DevOps although 73 percent have started down the path and an additional 15 percent are considering it

■ 89 percent have adopted agile in some form but only 32 percent are fully agile

“In the past few years, the way development teams use cloud services for testing has changed dramatically,” said Diane Hagglund, Principal Researcher of Dimensional Research. “Teams are becoming more efficient and prescriptive when it comes to how they are using third party cloud services for testing. The report further shows adoption of Continuous Integration (CI) continues to increase, as development teams invest in new methodologies like CI to meet the demands of today's market needs.”

Survey Methodology: A global database of technology professionals responsible for testing web and mobile applications was emailed an invitation to participate in a Web survey on the topic of testing trends. A total of 732 individuals participated in the survey. Participants included a variety of roles, company sizes, industries and regions. A copy of this report was offered as an incentive for participation. To allow for trend analysis, certain questions were repeated from similar surveys conducted to the same audience in early 2016 and early 2015.

The Latest

September 20, 2018

The latest Accelerate State of DevOps Report from DORA focuses on the importance of the database and shows that integrating it into DevOps avoids time-consuming, unprofitable delays that can derail the benefits DevOps otherwise brings. It highlights four key practices that are essential to successful database DevOps ...

September 18, 2018

To celebrate IT Professionals Day 2018 (this year on September 18), the SolarWinds IT Pro Day 2018: A World Powered by Tech Pros survey explores a "Tech PROactive" world where technology professionals have the time, resources, and ability to use their technology prowess to do absolutely anything ...

September 17, 2018

The role of DevOps in capitalizing on the benefits of hybrid cloud has become increasingly important, with developers and IT operations now working together closer than ever to continuously plan, develop, deliver, integrate, test, and deploy new applications and services in the hybrid cloud ...

September 13, 2018

"Our research provides compelling evidence that smart investments in technology, process, and culture drive profit, quality, and customer outcomes that are important for organizations to stay competitive and relevant -- both today and as we look to the future," said Dr. Nicole Forsgren, co-founder and CEO of DevOps Research and Assessment (DORA), referring to the organization's latest report Accelerate: State of DevOps 2018: Strategies for a New Economy ...

September 12, 2018

This next blog examines the security component of step four of the Twelve-Factor methodology — backing services. Here follows some actionable advice from the WhiteHat Security Addendum Checklist, which developers and ops engineers can follow during the SaaS build and operations stages ...

September 10, 2018

When thinking about security automation, a common concern from security teams is that they don't have the coding capabilities needed to create, implement, and maintain it. So, what are teams to do when internal resources are tight and there isn't budget to hire an outside consultant or "unicorn?" ...

September 06, 2018

In evaluating 316 million incidents, it is clear that attacks against the application are growing in volume and sophistication, and as such, continue to be a major threat to business, according to Security Report for Web Applications (Q2 2018) from tCell ...

September 04, 2018

There's a welcome insight in the 2018 Accelerate State of DevOps Report from DORA, because for the first time it calls out database development as a key technical practice which can drive high performance in DevOps ...

August 29, 2018

While everyone is convinced about the benefits of containers, to really know if you're making progress, you need to measure container performance using KPIs.These KPIs should shed light on how a DevOps team is faring in terms of important parameters like speed, quality, availability, and efficiency. Let's look at the specific KPIs to track for each of these broad categories ...

August 27, 2018

Protego Labs recently discovered that 98 percent of functions in serverless applications are at risk, with 16 percent considered "serious" ...

Share this