1700 DevOps Monitoring Experts Agree: Too Many Alerts from Too Many Tools Put Customers at Risk
May 18, 2016

Dan Turchin
BigPanda

We're all technology companies. Every second of downtime hurts. Monitoring at scale is hard. And that's just the beginning of what you shared in our recent survey.

We invited you to tell us about the state of monitoring. Tales of woe and glory from more than 1,700 ops experts provided the most articulate, profound, comprehensive summary of IT Ops life ever assembled.

We thought you'd all benefit from what you shared so we published the results. You represent five continents, large and small companies (modal reply: more than 10,000 employees), large and small teams (modal reply: less than 10 members), and both traditional IT and DevOps organizations.

Here's what fascinated me...

You rely on many tools to monitor your infrastructure.

■ Each team member is responsible for triaging between 10 and 50 alerts per day.

■ In an eight-hour shift, that means you're each working about 10 issues simultaneously assuming you don't inherit orphans from previous shifts (which you do!).

■ Translation: there are fire-swallowing, tightrope-walking, lion tamers working the e.coli route for Carnival Cruise Line with easier jobs than yours.

The more you've invested in agility and velocity, the more effective you are at reducing downtime.

■ Self-described "DevOps" organizations are more than twice as likely to deploy code and/or infrastructure changes at least a few times per day (31% for DevOps orgs vs. 15% overall).

■ They're also more than twice as likely to have cloud-based infrastructure (32% of DevOps orgs vs. 13% overall).

You're dissatisfied with the current reliability of your monitoring and incident management process.

■ Nearly 80% of you say the most challenging part of your job is suppressing alert noise.

■ The problem's not going away: more than 55% are dissatisfied with the current monitoring strategy. Your comments also indicate the problem won't improve in the next 12 months without a better way to manage the growing workload.

A bleak picture perhaps best summarized by Carlos from a midwestern credit union who says if he could change one thing about his organization's current monitoring strategy it would be "to focus on the only thing that matters: reducing noise." Carlos, you're right. Human beings alone can't fix a problem created by machines. We've been in this position before … before there was client-server, TCP, DNS, virtualization, cloud.

We've approached each challenge with the same tenacity, the same passion, the same commitment to solving problems with technology. We'll do it again. This time, with better automation and collaboration. Soon, machines and people will speak a common language. And when they do, we'll be the first to share how great technology plus your ingenuity makes life better for everyone.

Dan Turchin is VP Product at BigPanda.

The Latest

July 17, 2018

In my first blog in this series, I highlighted some of the main challenges teams face with trying to scale mainframe DevOps. To get past these hurdles, the key is to develop an incremental approach that enables teams to capture value along each step of the journey ...

July 16, 2018

The key to mainframe DevOps success is in quickly identifying and removing major bottlenecks in the application delivery lifecycle. Major challenges include collaboration between mainframe and distributed teams, lack of visibility into the impact of software changes, and limited resource flexibility with scaling out necessary testing initiatives. Now let's take a closer look at some of these key challenges and how IT departments can address them ...

July 11, 2018

How much are organizations investing in the shift to cloud native, how much is it getting them? ...

July 10, 2018

In the shift to cloud native, many organizations have adopted a configuration-as-code approach. This helps drive up application deployment velocity by letting developers and DevOps teams reconfigure their deployments as their needs arise. Other organizations, particularly the more regulated ones, still have security people owning these tools, but that creates increased pressure on the security organization to keep up. How much are organizations investing in this process, and how much is it getting them? ...

June 28, 2018

More than a third of companies that use serverless functions are not employing any application security best practices and are not using any tools or standard security methodologies to secure them, according to the State of Serverless Security survey, conducted by PureSec ...

June 27, 2018

The popularity of social media platforms and applications is spurring enterprises to adopt "social business" models to better engage with employees and customers and improve collaboration, according to a new study published by ISG ...

June 25, 2018

The previous chapter in this WhiteHat Security series discussed Codebase as the first step of the Twelve-Factor App and defined a security best practice approach for ensuring a secure source control system. Considering the importance of applying security in a modern DevOps world, this next chapter examines the security component of step two of the Twelve-Factor methodology. Here follows some actionable advice from the WhiteHat Security Addendum Checklist, which developers and ops engineers can follow during the SaaS build and operations stages ...

June 21, 2018

DevSecOps is quickly gaining support and traction, within and beyond information security teams. In fact, 70% of respondents believe their culture can embrace the change needed to fuse Security and DevOps, according to a new survey of 80 security professionals by Aqua Security ...

June 20, 2018

The larger the company size, the higher the proportion of low IT performers, according to the State of DevOps: Market Segmentation Report from Puppet, based on the 2017 State of DevOps Survey data ...

June 18, 2018

An overwhelming 83 percent of respondents have concerns about deploying traditional firewalls in the cloud, according to Firewalls and the Cloud, a survey conducted by Barracuda Networks...

Share this