DEVOPSdigest

Start with Top 5 DevOps Challenges & Solutions Part 1: Focus on Business

Start with Top 5 DevOps Challenges & Solutions Part 2: Focus on Technology Leadership

Vision without execution is delusion. (iSpeak Cloud 2016)

The success of DevOps depends on both the cultural shift around people and process as well as the technical implementation skills of the team across Dev and Ops. For example, one DevOps initiative was struck down by the CIO because of technical impact issues to the business of the continuous integration and continuous delivery process. The reality did not point to either process or leadership flaws, but improperly written and executed tests. At the end of the day, the damage was done. The executives only heard the sound bite that the system issues were due to the “DevOps” process. The official initiative was cut to placate the source of the issues as a result. Ironically, the final implementation of their cloud strategy was around DevOps, but to this day, the technology team will not call it that.

In Part 1 of this 3-part series on Challenges and Solutions for implementing DevOps, I provided an overview of the Top 5 Challenges and Solutions the Business faces when the technology team implements a DevOps Strategy. Part 2 focused on technology leadership challenges and suggested solutions. Finally, in this post, Part 3, I will focus on the top 5 technology implementation mistakes.

Challenge 1: Not Understanding Security and Access Management

Security was the number one concern of every person I interviewed for the iSpeak Cloud series. Why? Because implementation of both DevOps and Cloud solutions has often been in a vacuum, with security requirements as an afterthought. As a result, companies have discovered that they are vulnerable in areas that could significantly impact not only their DevOps implementation, but also their company overall.

One critical success factor is as simple as ensuring that the security team has a seat at the table when you are creating your overall strategy. From all perspectives, the team should understand the security features of the identity and access management solutions and the best routes to implement them to reduce risk and ensure compliance.

Another success factor is to ensure that your team is trained on the security requirements for your company from both a Control Objective Baseline for IT Standards (COBIT) and Cloud Security Alliance perspective. This will reduce the need to re-do later because solutions will be architected for security and compliance from the start.

In addition to extending the security tool sets to the DevOps team, the team should map out a plan to use them. In other words, before they begin, the entire team should have mapped out a strategy to secure all the applications, services, microservices and/or APIs.

Challenge 2: Not consistently managing all environments

The second challenge is that every environment throughout the DevOps process must be consistently managed across teams.

Environmental differences can cause differences in development and deployment. One large retailer overcame its environment inconsistency issues by leveraging an orchestration tool as the framework for their DevOps platform. This enabled the company to create a library of repeatable run books to automate.

An IT leader of another company reported addressing consistency issues by automating redundant processes such as change management. This company reduced time to value as well as the traffic associated with the previous layers of approval by automating the approval process for sign off to roll into production.

Finally, top performers standardize every stage and enforce those standards across teams based on a bonus pool for quality. These top performers hold everyone accountable to the agreed upon standard throughout the process from the business leaders to the DevOps engineers. Not only did this reduce shadow IT, but it also helped to improve the quality and consistency of releases as part of Continuous Integration Continuous Delivery process.

Challenge 3: Ignoring the importance of monitoring and/or metrics from the business perspective

One top performer I interviewed hit quality issues across multiple business lines in different regions. In post mortem, the company discovered that it was caused by an undetected latency in the system. It turns out that the services were only being monitored if they were up or down, not if they were actually working. Although the users could log into the application because it was up, they could not use it because it would hang or take too long to load. The team realized that in order for their DevOps practice to be successful, they had to provide value to the business.

Top performers implemented monitoring against service level agreements for all of the components in the stack. This ensured that they were measuring what was important to the company overall, rather than what was important to IT.

Metrics should be defined not only for the services, but also the underlying systems and team members throughout the process. For example, understanding the velocity issues of your scrum team can assist in identifying bottlenecks or critical skills gaps that will impact quality and agility.

A big part of DevOps, along with Continuous Integration and Continuous Delivery, is Continuous Improvement. The team can build a proactive (versus reactive) process by monitoring and measuring the systems, team members and users interactions. In this way, the team is set to embrace change and be in tune with the business.

Challenge 4: Onboarding Codebase or Service to the DevOps Process Too Soon

Not every service or application codebase will be a good candidate for frequent releases or the type of agile development environment prescribed by DevOps.

One mistake cited by many early adopters was not categorizing their services and/or applications prior to migrating them onto the new cloud platform and/or DevOps Process. As a result, they hit issues with the continuous improvement and continuous delivery process.

Legacy services and applications, and even some new ones, should be reviewed. It is important to identify areas in the codebase that should be simplified and documented. Part of the documentation should be calling out key files that need refactoring to function properly on the new platform with the new process. Another key success factor to reduce the refactoring risk is to perform unit tests.

Challenge 5: Not Designing for Scale around Development & Testing

Be sure to consider the scalability of each unit, from interdependency design of the codebase, to the automated test. Otherwise, the entire system will start to break down.

Interdependencies should be kept at a minimum through the individual codebase and test cases.

Whenever possible, test casts should not be written to be dependent on the outcome of other tests.

Another best practice is to use temporary tests and/or QA pods for parallel testing of features.

Regardless of your strategy, the key take away is to ensure that reliability, availability, supportability, serviceability and scalability are an integral part of the solution.