DEVOPSdigest

A new tool by Buildkite launches, Test Analytics, promise to fix one of the biggest and most expensive problems in software development — flaky tests.

Buildkite's new product Test Analytics speeds up delivery pipelines for DevOps engineers by eliminating waste in the development process. The tool works seamlessly with Buildkite's CI/CD platform and integrates with any CI system, including CircleCI, GitHub Actions and Jenkins, providing continuous performance monitoring and real-time insights for test suites. These insights help software development teams identify, optimize, and resolve flaky and unreliable tests faster than ever before, freeing up teams to focus on fixing bugs and rolling out new features.

Buildkite's CI/CD platform helps developers run software quickly and reliably, with testing and deployment tools that work for all software projects, whether for building mobile apps, backend applications, or custom hardware.

Tim Lucas, Co-founder and Co-CEO of Buildkite, said, "Flaky and unreliable tests are a nightmare for the developer world. There are three traditional ways of dealing with the issue. The first is to spend an inordinate amount of time and energy identifying the root cause of a flaky test. The second is to run your test repeatedly until it passes, wasting compute capacity and increasing costs. And the third is to ignore the failure altogether, which makes for unhappy developers and erodes trust in your entire testing and delivery process. With Test Analytics, developers can performance-tune their test suites to identify problems more quickly off the back of clear insight and analysis. Development teams not only have access to creating highly performant and stable build infrastructure, they now have the tools for making their own test suites faster and less flaky."

A clear dashboard within Test Analytics shows the slowest and least reliable tests, prioritizing the most critical fixes. Historical views and trend graphs provide further insight into diagnosing recurring problems. User-defined thresholds provide granularity in defining slow or unreliable tests and creating custom alerts. Integrations with commonly-used testing frameworks allow for deep inspection of where tests are spending time, from long-running database queries to API calls timing out.