DEVOPSdigest

Electric Cloud recently hosted a special episode of Continuous Discussions (#c9d9), featuring Gene Kim and speakers from the upcoming DevOps Enterprise Summit San Francisco (DOES17). In light of the recent Equifax breach, Kim and the speakers dissected the situation and discussed the technical leadership lessons learned while offering their own expert advice for handling crisis situations.

The panel included:

■ Carmen DeArdo, Technology Director at Nationwide Insurance

■ John Allspaw, (former) CTO of Etsy

■ John Esser, Senior Director of IT and Data Center Operations at AdvancedMD

■ Mik Kersten, CEO of Tasktop

■ Scott Nasello, Senior Manager of Platform and Systems Engineering at Columbia Sportswear

■ Anders Wallgren, CTO of Electric Cloud

The following is a list of the 17 most memorable takeaways:

1. Failures and breaches can happen at any moment – leaders should take an investigative approach, explains Allspaw: "How much attention did vulnerability patching get up until the day before (the breach)? If I was the leader of this organization, part of my job is to create the conditions such that the organization can bring their attention proportionally and understand what the trade-offs are."

2. If you can't responsibly protect a certain part of your business, then you have no business trying to monetize it, per Kim: "I think the other thing in this scenario is, I would actually want business leaders to truly understand that this is an existential risk. The outside threat has never been higher, it's always escalating and it is actually left unaddressed. This is an existential threat to the business model. If we can't responsibly hold PHI, then our ability to make money from it should be put into jeopardy."

3. Kersten is particularly peeved by the narrative around the breach: "What's so disturbing about the narrative here is that these leaders of these companies are not understanding that they have an organizational responsibility to managing their IT stack. That stack is how they're delivering value to their customers and how they're exposing their customers' data or safety."

4. It's important for the person responsible for a system failure to step up and take responsibility, says DeArdo: "You have to accept responsibility not just because it's the right thing to do, but because it allows you to start talking in a way of not only what we can do for our customers but what we can do to our culture and our systems. If you don't take that responsibility, not only does this send the wrong message, but it doesn't let you move towards fixing things."

5. Nasello on the finger-pointing nature of the response to the breach: "[The ownership component] figures quite prominently in the DevOps transformations we're trying to do in our companies. It rings hollow if you're not really willing to own the consequences and outcomes of the transformation that you're trying to drive, and I think that lack of authenticity is going to hurt companies when they're trying to attract and retain and grow their organizations."

6. The Equifax breach was an organizational-wide failure, per Esser: "What happened here was truly an organizational failure all the way up and all the way down. Any security auditor would pick up on these things in a basic audit. The question would be is how long some auditor was saying, ‘We have a problem.'"

7. Elaborating on what he calls "normalization of deviance," Wallgren says: "We've been running with struts for so long and nothing bad has happened – maybe we'll be able to keep doing. It really is a systems failure and maybe there needs to be a NTSB-like function for these kinds of problems where you have an independent fact-finding, probable cause finding situation. We're never going to find out what the real problem was at Equifax."

8. Lead by example – what you do as a leader will become the new standard, explains Nasello: "As leaders of organizations, the things that we tolerate become standards as well. Our organizations and the teams that we lead are very observant in terms of what we tolerate and the examples that we set. We undermine ourselves a lot by acting differently than what our words are, and so authenticity and really being clear on what the principles of the organization are and what you're really trying to achieve is primarily a leadership function."

Read 17 Tech Leadership Lessons Learned from the Equifax Breach - Part 2 for more highlights from the discussion.

Watch the full discussion below