Financial services organizations are high value targets for cyber criminals all over the world. Because of this, it is imperative that the keys and certificates used by financial service DevOps teams are properly protected. If not, bad actors can easily exploit cryptographic assets and wreak havoc on sensitive corporate data, all while remaining undetected ...
Downtime has always been a fact of life for IT and operations teams.
But for all the headaches it caused, you at least had ownership of it. It was your downtime. Your team, your servers, your problem.
Now, with cloud and DevOps in the mix, downtime has become a shared hassle. It could be code causing outages for end users, or a problem with the myriad of cloud providers used to host and run services.
Sometimes it's a problem IT can fix. But more and more, it involves sitting and waiting for a vendor to fix the problem. Meanwhile, customers grow increasingly frustrated while waiting to regain access. When building on the cloud, a lot of things are simply out of the IT team's control.
Shared downtime is a major reality for businesses building and using cloud tools.
It's easy to ignore downtime. Especially if you're not a cloud-first shop. Just ask the thousands of businesses affected by recent outages with Dyn or AWS S3. But ignoring downtime is a surefire way to upset your customers and your colleagues. More and more, teams need to think about shipping stellar experiences. Proper incident response is a great place to start.
Atlassian and xMatters joined forces on a survey of 1000+ organizations to gain insights on current approaches to incident management and response.
The findings highlight the need for better incident response and management processes. A large percentage of the teams surveyed admitted they don't have strong incident management processes in place. For example:
■ Half of the respondents indicated that the tools, processes and steps vary from incident to incident.
■ 50 percent of developers said they usually wait for the operations center to declare a major incident before taking action.
■ 43 percent use manual processes to keep customers and internal stakeholders up to date.
When it comes to closing the loop between the developers and end users, a lot of teams are still unnecessarily repeating work and answering the same questions multiple times. For example:
■ 29 percent said duplicate tickets were created while the incident was being resolved — leading to an unnecessary backlog of tickets.
■ Almost a quarter (23 percent) said that tickets are routed without proper assignments being made during incidents.
■ More than a third (39 percent) of respondents said incident resolution is delayed by time spent waiting for subject matter experts.
Business stakeholders are equally frustrated. The lack of timely communications during a major incident frustrates them more (56 percent) than the actual incident (44 percent).
Incident Management in a DevOps World
DevOps advocates a "you build it, you ship it, you run it" model. This leads to more agility in delivering solutions to market and better resilience. In fact, 65 percent of those respondents reported that their DevOps initiatives are producing the benefits they expected to see — which is great news.
But to achieve resilience, teams must adapt to new challenges. With an increasing pace of changes, incidents happen more often.
The good news is that the same DevOps practices and tools can be applied to incident management. Combine that with automated testing, which has become foundational to DevOps, teams should become more empowered to swiftly respond.
The survey made one thing clear: smart teams are proactive about downtime and make a point to practice incident management. Customers and colleagues don't care who's responsible for downtime, they just want to know what's going on. They want businesses to take ownership of it.
Scott Klein is StatusPage Product Manager at Atlassian.