Big Data and Cloud Problems Are an Employment Problem
February 16, 2016

Brad Kolarov
B23

Regardless of the industry, we are finding these days all businesses are becoming data businesses. To keep up with competition, companies need to collect and analyze data, gain insight on that data and act on that insight.

That's easier said than done. At the moment, many companies can't keep up with their data, let alone their competition. That's not a technology problem; it's an employee skillset problem.

The proliferation of data and the pace of innovation within the open source software ecosystem are driving the convergence of Big Data, Cloud Computing, and DevOps/Automation. Proficiency across these functional areas is essential in this rapidly evolving, converging IT environment. At the moment, however, there's a lack of qualified talent with these cross-disciplinary skills.

To an extent, the emergence of DevOps itself has led to a skills imbalance across disciplines.

In the recession of 2007–2009, business cost cutting led to many technical efficiencies. This included the migration from waterfall to agile development – which created the need for DevOps, bridging software engineering, technology operations and quality assurance. DevOps, therefore, emerged as an active and growing skillset (DevOps is the convergence of software development, IT Operations and Quality Assurance). Many technologists immersed themselves in the skills related to DevOps.

In fact, though, DevOps is only a subset in the much larger convergence of technical skills needed in today's data-centric business environment.

Here's the real need:


The center of these overlapping circles is where the real action is. But it's also a real challenge.

Finding qualified candidates who can work with Big Data (Hadoop, NoSQL, Storm, Kafka, etc.) tools is very difficult. Understanding Cloud (AWS, Azure, Google, etc.) is just as hard. Those with experience with DevOps/Automation (Chef/Puppet/Ansible/Salt, etc.) is a little easier but nonetheless challenging.

And it's not enough to have a certification in certain technology skills. Certification and experience are not synonymous. Of course, certifications are a great prerequisite, and a start towards a true cross-disciplinary skillset, but certifications are not necessarily an indicator of real world experience.

■ In Big Data, what's important is having built a production Big Data system. What's not is something like Hadoop Certification (nice but not essential).

■ In Cloud Computing, what's important is being able to use SDKs and APIs, and to implement with best practices in your coding. What's not is something like AWS Certification (nice but not essential).

■ In DevOps/Automation, what's important is being able to work with full stack automation. What's not is working with software installation scripts (do we even need to explain why that's not important?)

It's quite uncommon to have a great developer come to the table with all three of these skill sets. Softer skills are often a good indicator of future capability. For example, when our company hires new technology talent, the most desirable traits among our qualified candidates are  proven aptitude and motivation to learn.

As rare as it is to find people with the ability to work in today's converging technologies, it is still vitally important. Businesses with a shortage in these skills are definitely at a technical disadvantage compared with their better-heeled competitors.

For most companies, hiring individuals with these cross-disciplinary skills is not scalable. To level the playing field, businesses should create opportunities to gain these technology benefits.

Collaboration with integrators and consultants that have invested specifically in product and service offerings across the three disciplines of Big Data, Cloud and DevOps/Automation can go a long way to building real world capability in your own organization, without having to scale up in either internal cost or human resources.

Brad Kolarov is a Co-Founder and Managing Partner at B23.

The Latest

May 24, 2018

DEVOPSdigest asked experts from across the IT industry for their opinions on the top tools to support DevSecOps. Part 3 covers security and monitoring ...

May 22, 2018

DEVOPSdigest asked experts from across the IT industry for their opinions on the top tools to support DevSecOps. Part 2 covers DevOps and development ...

May 21, 2018

While DevSecOps, much like DevOps itself, is more about changing IT culture than employing certain types of technology, some tools can be an important support. To find out what the right tools are, DEVOPSdigest asked experts from across the IT industry for their opinions on the top tools to support DevSecOps. Part 1 covers the testing phase ...

May 17, 2018

The top two business priorities for CIOs of midsize enterprises (MSEs) in 2018 are growth and digital transformation. However, 57 per cent of MSEs are not yet delivering digital initiatives, according to findings from Gartner Inc.'s 2018 CIO Agenda Survey ...

May 15, 2018

Almost every company is facing the challenge of digital transformation today. This means rethinking and retooling your company to compete and succeed in an increasingly digital world. While digital transformation is not only about technology, the right tools can help. To find out what these right tools are, APMdigest asked experts from across the IT industry for their opinions on the essential tools to support digital transformation ...

May 08, 2018

With data breaches consistently being in the news over the last several years, it is no wonder why data privacy has become such a hot topic and why the European Union (EU) has put in place General Data Protection Regulation (GDPR) which will become enforceable on May 25, 2018, which is less than a month away ...

May 03, 2018

The prospect of increased workloads, combined with shrinking mainframe skillsets, has huge implications for mainframe DevOps. The only way for organizations to solve this skills gap crisis is by optimizing developer productivity. Drilling down a level further, what does this all mean for mainframe DevOps? ...

May 02, 2018

When it comes to operations and development, DevOps has changed the traditional compartmentalized style of development by eliminating silos. But what about the security team? Security is largely still siloed from operations and development. No doubt, many DevOps teams have some security controls baked into their automation processes, but a recent survey shows there are still alarming gaps ...

April 30, 2018

According to the 2018 Global Security Trends in the Cloud report, 93 percent of respondents faced challenges when deploying their current on-premises security tools in the cloud, and 97 percent lacked the tools, cross-functional collaboration and resources to gain proper insight into security across the organization. These numbers indicate a big problem in DevSecOps that needs to be addressed ...

April 26, 2018

Moving more workloads to the cloud is a top IT priority, so eventually it will be time to consider how to make those critical legacy applications cloud ready. In Part 1 of this blog, I outlined the first four of eight steps to chart your cloud journey. In addition, consider the next four steps below ...

Share this