DEVOPSdigest

Since the inception of DevOps practices, technology providers in this field have generally focused on a single aspect of the Software Development Lifecycle. Some expanded their product lines with additional discreet offerings, but for the most part, a vendor provided either Planning, Orchestration, or Testing. They didn't combine the three in a single platform.

Some companies have taken a very different approach and realized from the very start that User Stories were the foundation of Agile-based DevOps. The User Story can be the container for change and associated the list of changes directly with the business goal of the story. Linking the User Story to a Feature Branch in the Version Control system enables the system to know what changes are required to the underlying code and metadata, but it also enables the synthesis of a release branch at any stage of the Pipeline, just by providing a list of approved user stories.

Likewise, Orchestration and Planning companies have not historically offered testing tools. Testing companies specialize in testing only, even though their products typically include a minimum orchestration capabilities to automate tests. Furthermore, "testing companies" generally focused on either functional testing, performance testing, or security testing, but not all. When they do provide all, they tend to be a ragtag collection of products acquired over time that are never fully integrated.

Here's why the combination of Planning, CICD Orchestration, and Functional Testing is like Reese's Peanut butter cups. The combination of flavors is just too good to resist. (For those of you who are detail oriented, add a crushed nut topping to make it three ingredients. Thanks for counting.)

Testing + Planning

Let's start with the benefits of deep integration between Testing and User Stories.

A well written User Story includes not only a complete description of the requirements, but should also include the business impact and what if any impact this has to existing processes. This provides 3 key opportunities for testing:

■ Automated Generation of Happy Path Tests using GenAI (aka Intent Based Testing)

■ Automated Identification of existing tests which are impacted by the change

■ Test association and Test Run History

Intent Based Test Generation relies on Generative AI to create the scripts for testing a feature. If the User Story is written well, there is enough information to create both the manual testing steps as well as a test script to validate the happy path.

Since many organizations conduct User Acceptance Tests (UAT), the ability to provide a well written description of a new feature is key to preparing the business users to test, including a summary of the feature and the manual test steps. It's recommended to take that one step further and to write a test script and perform a test while capturing screen shots and a video. So the manual testers receive high quality documentation to understand what the feature is all about.

Most new features in business applications impact existing processes. There aren't a lot of net-new processes. If this process has automated test coverage in place already, then this new feature may break the existing test. It would not be a failure per se, just a new way of conducting the process. The existing test should either be retired or modified to account for the new feature. But how do you identify the existing tests which may be impacted by the new feature? AI can search your previous User Stories for similar processes and link those to the existing tests. This relies on the third benefit of User Story Integration, Test Association with User Stories.

Deep integration with User Stories directly associates tests with the business capability documented by the User Story. The verbal description can be fed into the GenAI tool to identify which User Stories are associated with the same processes. This in turn enables you to flag tests which may be impacted and can be used to automatically select those related tests for execution before allowing this change to be promoted. (See the quality gates and test selection discussion below.)

One final word on planning concerning User Story quality. These benefits are only achievable if the User Story is complete and well documented. GenAI can be used for User Story enrichment to ensure this is the case. Part of the Planning process should include quality checks on the User Story itself. This has not been the case in the past, where agile teams practice "just enough, just in time" documentation. GenAI can be a very powerful assistant, but only when intentions are well documented.

Testing + Orchestration

The combination of Testing and Planning is further enhanced when CICD orchestration is added to the recipe. It can contain an explicit list of the code and metadata changes required for each user story.

The major benefits are:

■ Intelligent Test Selection

■ Timing of multi-story Feature testing

If compute power were free and tests ran in zero time, then we could perform every automated test each time a check-in was done to a new environment. Unfortunately, compute time costs real money and complete automation suites take a long time to execute. For minor changes that only touch a small portion of the solution, many tests are unnecessary. They can't possibly be impacted by the changes. But which tests are required and which are not?

Intelligent Test Selection is a process whereby the DevOps platform determines which tests are important and which are unnecessary. This is made possible by the integration with planning as discussed above, and the association of component changes with the User Story. If a given story changes a piece of code, data model, page layout, or API, then it can impact any other User Story which also touched these items in the past. Identifying which User Stories are impacted gets us the list of tests to run, since they are associated with the User Story.

Finally, the Timing of the test runs is often a critical factor. In most agile practices, product owners are taught to keep User Stories small. This means that the work to implement the story can be completed in a single sprint. Many new features are more complex than that though, so developers and product owners are taught to break up a complex feature into a collection of small User Stories that are dependent on each other. But this means that the new functionality cannot be tested until all of the dependent User Stories have made it into a given environment. This gets even more complicated when the feature includes an API change on a different system. The API User Story is likely managed in a different pipeline by a different team of developers and served from a different environment.

To properly test dependent User Stories, the tests must be triggered when all of the dependent User Stories are aligned in the same stage. This is not practical when using the Test tool orchestration engine to run tests. This can only work when the CICD system triggers the test case after it has confirmed that all of the stories are aligned.

Conclusion

In the new age of Generative AI, DevOps tools will be able to perform much of the mundane tasks and automate much more of our work. But that requires a much tighter integration between the User Story, CICD Orchestration Engine, and Functional Testing tools. While all of this is possible using disparate tools and hand crafted integrations, the true value comes when a single DevOps platform combines all of these capabilities out of the box.