Big Data and Cloud Problems Are an Employment Problem
February 16, 2016

Brad Kolarov
B23

Regardless of the industry, we are finding these days all businesses are becoming data businesses. To keep up with competition, companies need to collect and analyze data, gain insight on that data and act on that insight.

That's easier said than done. At the moment, many companies can't keep up with their data, let alone their competition. That's not a technology problem; it's an employee skillset problem.

The proliferation of data and the pace of innovation within the open source software ecosystem are driving the convergence of Big Data, Cloud Computing, and DevOps/Automation. Proficiency across these functional areas is essential in this rapidly evolving, converging IT environment. At the moment, however, there's a lack of qualified talent with these cross-disciplinary skills.

To an extent, the emergence of DevOps itself has led to a skills imbalance across disciplines.

In the recession of 2007–2009, business cost cutting led to many technical efficiencies. This included the migration from waterfall to agile development – which created the need for DevOps, bridging software engineering, technology operations and quality assurance. DevOps, therefore, emerged as an active and growing skillset (DevOps is the convergence of software development, IT Operations and Quality Assurance). Many technologists immersed themselves in the skills related to DevOps.

In fact, though, DevOps is only a subset in the much larger convergence of technical skills needed in today's data-centric business environment.

Here's the real need:


The center of these overlapping circles is where the real action is. But it's also a real challenge.

Finding qualified candidates who can work with Big Data (Hadoop, NoSQL, Storm, Kafka, etc.) tools is very difficult. Understanding Cloud (AWS, Azure, Google, etc.) is just as hard. Those with experience with DevOps/Automation (Chef/Puppet/Ansible/Salt, etc.) is a little easier but nonetheless challenging.

And it's not enough to have a certification in certain technology skills. Certification and experience are not synonymous. Of course, certifications are a great prerequisite, and a start towards a true cross-disciplinary skillset, but certifications are not necessarily an indicator of real world experience.

■ In Big Data, what's important is having built a production Big Data system. What's not is something like Hadoop Certification (nice but not essential).

■ In Cloud Computing, what's important is being able to use SDKs and APIs, and to implement with best practices in your coding. What's not is something like AWS Certification (nice but not essential).

■ In DevOps/Automation, what's important is being able to work with full stack automation. What's not is working with software installation scripts (do we even need to explain why that's not important?)

It's quite uncommon to have a great developer come to the table with all three of these skill sets. Softer skills are often a good indicator of future capability. For example, when our company hires new technology talent, the most desirable traits among our qualified candidates are  proven aptitude and motivation to learn.

As rare as it is to find people with the ability to work in today's converging technologies, it is still vitally important. Businesses with a shortage in these skills are definitely at a technical disadvantage compared with their better-heeled competitors.

For most companies, hiring individuals with these cross-disciplinary skills is not scalable. To level the playing field, businesses should create opportunities to gain these technology benefits.

Collaboration with integrators and consultants that have invested specifically in product and service offerings across the three disciplines of Big Data, Cloud and DevOps/Automation can go a long way to building real world capability in your own organization, without having to scale up in either internal cost or human resources.

Brad Kolarov is a Co-Founder and Managing Partner at B23.

The Latest

July 16, 2018

The key to mainframe DevOps success is in quickly identifying and removing major bottlenecks in the application delivery lifecycle. Major challenges include collaboration between mainframe and distributed teams, lack of visibility into the impact of software changes, and limited resource flexibility with scaling out necessary testing initiatives. Now let's take a closer look at some of these key challenges and how IT departments can address them ...

July 11, 2018

How much are organizations investing in the shift to cloud native, how much is it getting them? ...

July 10, 2018

In the shift to cloud native, many organizations have adopted a configuration-as-code approach. This helps drive up application deployment velocity by letting developers and DevOps teams reconfigure their deployments as their needs arise. Other organizations, particularly the more regulated ones, still have security people owning these tools, but that creates increased pressure on the security organization to keep up. How much are organizations investing in this process, and how much is it getting them? ...

June 28, 2018

More than a third of companies that use serverless functions are not employing any application security best practices and are not using any tools or standard security methodologies to secure them, according to the State of Serverless Security survey, conducted by PureSec ...

June 27, 2018

The popularity of social media platforms and applications is spurring enterprises to adopt "social business" models to better engage with employees and customers and improve collaboration, according to a new study published by ISG ...

June 25, 2018

The previous chapter in this WhiteHat Security series discussed Codebase as the first step of the Twelve-Factor App and defined a security best practice approach for ensuring a secure source control system. Considering the importance of applying security in a modern DevOps world, this next chapter examines the security component of step two of the Twelve-Factor methodology. Here follows some actionable advice from the WhiteHat Security Addendum Checklist, which developers and ops engineers can follow during the SaaS build and operations stages ...

June 21, 2018

DevSecOps is quickly gaining support and traction, within and beyond information security teams. In fact, 70% of respondents believe their culture can embrace the change needed to fuse Security and DevOps, according to a new survey of 80 security professionals by Aqua Security ...

June 20, 2018

The larger the company size, the higher the proportion of low IT performers, according to the State of DevOps: Market Segmentation Report from Puppet, based on the 2017 State of DevOps Survey data ...

June 18, 2018

An overwhelming 83 percent of respondents have concerns about deploying traditional firewalls in the cloud, according to Firewalls and the Cloud, a survey conducted by Barracuda Networks...

June 14, 2018

Despite the vast majority of cloud management decision-makers believing that DevOps and microservice enablement are important, very few believe that their organizations are capable of delivering them today — a gap that is costing the average enterprise $34 million per year, according to new report from the Ponemon Institute ...

Share this