How Artificial Intelligence is Revolutionizing IT Operation Analytics
July 20, 2016

Akhil Sahai
Perspica

After many science fiction plots and decades of research, Artificial Intelligence (AI) is being applied across industries for a wide variety of purposes. AI, Big Data and human domain knowledge are converging to create possibilities formerly only dreamed of. The time is ripe for IT operations to incorporate AI into its processes.

IT infrastructures today are increasingly dynamic and agile but at the same time extraordinarily complex. Humans are no longer able to sift through the variety, volume and velocity of Big Data streaming out of IT infrastructures in real time, making AI—especially machine learning—a powerful and necessary tool for automating analysis and decision making. By helping teams bridge the gap between Big Data and humans, and by capturing human domain knowledge, machine learning is able to provide the necessary operational intelligence to significantly relieve this burden of near real-time, informed decision-making. Industry analysts agree. In fact, Gartner named machine learning among the top 10 strategic technologies for 2016, noting “The explosion of data sources and complexity of information makes manual classification and analysis infeasible and uneconomical.”

IT administrators, IT operators for TechOps and Site Reliability Engineers (SRE) for DevOps are tasked with manually gathering this disparate information and applying their domain expertise in an attempt to make informed decisions. While these professionals are great at what they do, trying to analyze so much data from multiple tools leaves the door wide open for human error. On the other hand, analytics that are based on machine learning are quickly becoming a necessity to ensure the availability, reliability, performance and security of applications in today's digital, virtualized and hybrid-cloud network environments.

The traditional approach centered around using multiple monitoring tools for IT siloes that provided IT operations teams with information about their virtual and physical infrastructure, application infrastructure and application transaction performance. While these tools provide pieces of the puzzle, they offer a narrow view of the IT infrastructure and, therefore, only one aspect of the tool chain. The other aspect is service desk tools that manage tickets and change management. Humans more often than not bridge this gap between the siloed monitoring tools of yesterday and service desk applications with their domain expertise.

What Analytics Can Do Now

Today, the entire application infrastructure stack is overflowing with Big Data. TechOps and DevOps environments need to automate, learn and make intelligent, informed decisions based on real-time analysis of all that data. Following are key analytics for IT operations:

1. Anomaly Detection: Machine learning algorithms should have the ability to look at contextual, historical and sudden changes in the behavior of objects to detect anomalies. Understanding when there is a real anomaly and more importantly, when there is not, is critical to avoid generating false alarms. This is the bedrock of what is typically referred to as diagnostic analytics.

2. Topology Analysis: This type of analytics understands the hierarchal, peer-to-peer and temporal relationship between hybrid cloud elements. Topology is something every IT administrator or SRE should be aware of. This type of analysis should be able to self-learn the inter-relationships of objects and the impact of their performance on one another. Learning those relationships and maintaining that understanding in order to spot trouble in time is extremely important for both TechOps and DevOps environments.

3. Behavior Profiling: This is about understanding the behavior profile of every metric, how that is incorporated into the object behavior and then how the object behaviors relate to other object behaviors across the hybrid cloud environment. It is a multi-dimensional problem, and understanding and adapting to “normal” behavior is extremely important.

4. Root Cause: By finding the specific cause and impact of an incident, root-cause analysis is able to fast-track the resolution and reduce mean time to repair substantially.

5. Predictive: These analytics help operators identify early indicators and provide insights into looming problems that may eventually lead to performance degradation and outages. Predictive analytics are also good at providing early insights into anomalies to better plan for what's ahead.

6. Prescriptive: When you are looking for intelligent and actionable recommendations to remediate an incident, prescriptive analytics are the way to go. These recommendations should capture tribal knowledge gathered over the years in the organization and best practices in the industry, and may even be crowd-sourced to capture state-of-the-art knowledge. These analytics provide the opportunity to finally close the loop in automated IT Operations Management.

Embracing Machine Learning

It's been tough for a while now to be in IT operations, having to constantly react to incidents as well as trying to resolve them after they have spun out of control. Instead, AI provides technologies to help automate many of these tasks in order to handle incidents in advance. The whole notion of automating IT operational tasks, as well as preventing outages in the first place, and getting to the root cause quickly and in an automated way is the next frontier in remediating these issues.

As Gartner so eloquently put it, manual classification and analysis is infeasible and uneconomical. Not even an army of IT staff could review monitoring data quickly and thoroughly enough to identify incidents. Fortunately, AI has the capacity to enable real-time decision making by using multiple analytics capabilities simultaneously to see what's going on across the application stack.

Akhil Sahai, Ph.D., is VP Product Management at Perspica.

The Latest

June 25, 2018

The previous chapter in this WhiteHat Security series discussed Codebase as the first step of the Twelve-Factor App and defined a security best practice approach for ensuring a secure source control system. Considering the importance of applying security in a modern DevOps world, this next chapter examines the security component of step two of the Twelve-Factor methodology. Here follows some actionable advice from the WhiteHat Security Addendum Checklist, which developers and ops engineers can follow during the SaaS build and operations stages ...

June 21, 2018

DevSecOps is quickly gaining support and traction, within and beyond information security teams. In fact, 70% of respondents believe their culture can embrace the change needed to fuse Security and DevOps, according to a new survey of 80 security professionals by Aqua Security ...

June 20, 2018

The larger the company size, the higher the proportion of low IT performers, according to the State of DevOps: Market Segmentation Report from Puppet, based on the 2017 State of DevOps Survey data ...

June 18, 2018

An overwhelming 83 percent of respondents have concerns about deploying traditional firewalls in the cloud, according to Firewalls and the Cloud, a survey conducted by Barracuda Networks...

June 14, 2018

Despite the vast majority of cloud management decision-makers believing that DevOps and microservice enablement are important, very few believe that their organizations are capable of delivering them today — a gap that is costing the average enterprise $34 million per year, according to new report from the Ponemon Institute ...

June 12, 2018

Dev teams are doing their best to give the customers what they want, but oftentimes find themselves in between a rock and a hard place. Teams are struggling to get up to speed with new tools that are meant to make their lives easier and more realistic to hit deadlines. With spring cleaning season upon us, take time this season to tune up agile processes and continue the work of advancing the shift towards DevOps ...

June 11, 2018

The ability to create a culture of DevOps is critical to any organization's ability to deliver applications and services at a high rate of speed, but can we clearly and concisely answer the question: What exactly is DevOps? Despite the best intentions, some large companies are struggling to understand what DevOps actually is, and what it takes to fully implement its concepts and reap its benefits ...

June 07, 2018

The Twelve-Factor App is a methodology that offers a 12-step best practice approach for developers to apply when building software-as-a-service apps that are both scalable and maintainable in a DevOps world. As software continues to be written and deployed at a faster rate and in the cloud, development teams are finding there is more room for failure and vulnerabilities. This blog series will discuss how to build a Twelve-Factor app securely ...

June 05, 2018

Everyone understands the importance of code quality for applications, particularly when DevOps results in releases becoming faster and faster, reducing the room for error. The same issues increasingly apply to databases, which are a vital part of DevOps workflows. Fail to integrate the database into DevOps and you'll face bottlenecks that slow down your processes and undermine your efforts ...

June 04, 2018

DevOps and security traditionally have been siloed functions and security is often seen as a policing function by DevOps team members. However, more mature business leaders are trying to bridge the gap between the two functions to achieve business excellence. This theme was evident from our recent survey where 39% of respondents cited that DevOps and development teams care greatly about their cybersecurity posture, showing that the silo between security/IT and development teams is diminishing ...

Share this