Improving DevOps Efficiencies by Building an Agile Data Tier
August 10, 2016

Michelle McLean
ScaleArc

The ability to react quickly to market changes and launch new features and functionality is critical to maintaining a competitive edge.
 
Yet today, DevOps teams must spend substantial time and resources dealing with database scalability. If they need to consider scalability implications every time they roll out new code, the agile process breaks down. Software rollout cycles can get bogged down in application complexity that requires applications to be changed to function with the various roles of database servers and to regulate traffic to ensure data being read from secondary servers is current; database provisioning requirements which can be time-consuming and inefficient, limiting agility; and a lack of visibility – not understanding what application behaviors cause databases to slow down, which increases the development time required to address those issues.
 
Using database load balancing software, DevOps teams can solve these problems by creating an agile data tier that guarantees scalability of their database stack. An agile data tier can slash development time from months to just weeks by eliminating the need to build database scalability functionality into each separate application, enabling database clusters to be easily built and shared at scale and ensuring visibility into database traffic.

Enable Automated Database Failover and Scalability Without Increasing App Complexity

To scale structured data, DevOps teams follow a very specific set of rules. Functionality such as replication-aware load balancing, read/write split, and caching must be programmed into applications for them to handle and take advantage of multiple database servers. Applications also need to understand how to fail over to the right server within a cluster for high availability (HA).

A lot of code must be created within an app for it to be able to handle high availability (HA) –taking up valuable development time. Likewise, building an app that understands database topology – to do read/write split and assist in automated failover – adds to application complexity.

While typical caching solutions – such as Memcached – can increase database speed and scalability, they add complexity at the application layer. With these solutions, developers must spend time writing code that can create, manage, and expire cache in NoSQL. They must also write the individual business rules that specify which type of data can be cached. Similarly, a lot of code must be created within an app for it to be able to handle high availability (HA) – again, taking up valuable development time.

Remove the Need for Application Modification

Database load balancing software transparently provides replication-aware load balancing and read/write split to any application. By monitoring the replication lag between primary and secondary servers, database load balancing software can intelligently distribute queries, making sure an application gets the freshest data from the secondary servers. DevOps teams can easily set the appropriate replication lag threshold for a particular application.

Database load balancing software also allows teams to analyze database traffic and find read queries that are good candidates for caching. They can then choose which queries they want to cache and for how long. The software will then cache query responses and serve them to the application from a transparent in-memory cache without any changes at the application layer.

DevOps teams can also enable automated failover – without changing a single thing at the application layer. The database load balancing software understands which queries are reads and which are writes/transactions. It load balances the reads across all available servers and routes the writes to the primary. If the primary fails, the database load balancing software can enable seamless cross-server or cross-data center failover, maintaining state and reducing app errors with no changes to your applications.

By removing database scalability from the application and using database load balancing software, you can:

■ Roll out applications faster, since they are simpler to develop

■ Scale  database stack without requiring any new application coding

■ Focus staff attention on adding features and functionality that serve the business, instead of programming for data distribution

Provision Databases Seamlessly – Internal DBaaS

With database load balancing software, they can easily carve up a large database cluster into smaller pieces, giving each user the ability to run a certain amount of concurrent load on that database cluster. This approach provides a regulated, controlled load management stack – similar to a resource governor. They can slice and dice a cluster into smaller pieces, giving users access to their own sets of resources without disturbing anyone else on the cluster, essentially creating an internal DBaaS for their organization.

In this architecture, provisioning a new database endpoint takes only a few seconds compared with minutes or even hours when using an individual VM for each developer – similar to Amazon’s RDS that essentially provides a database ‘instance’ as a service.

With an internal DBaaS, teams can provide their application developers with an already highly redundant environment, ensuring that developers and QA are not interrupted due to database downtime. Teams also get access to very deep SQL analytics that help them test their applications better and identify potential application problems before moving into production.

Build New Applications, Scale and Solve Problems Faster and More Effectively

By decreasing application complexity, database load balancing software lets DevOps teams scale more effectively, benefit from DBaaS, and get the control and visibility needed to achieve results faster. Because database scalability functions are now out of the application, application design is simpler, making it easier to develop and build new applications. DevOps teams effortlessly provision databases to test your application’s functionality and benchmark performance better, with greater visibility into what the application is doing with the database.

With increased application resiliency, better use of database resources and enhanced system performance, organizations can speed the development process, improve overall efficiencies and gain business advantage.

Michelle McLean is VP of Marketing at ScaleArc.

The Latest

August 15, 2018

Microservices are a hot topic in IT circles these days. The idea of a modular approach to system building – where you have numerous, smaller software services that talk to each other instead of monolithic components – has many benefits ...

August 13, 2018

Agile is expanding within the enterprise. Agile adoption is growing within organizations, both more broadly and deeply, according to the 12th annual State of Agile report from CollabNet VersionOne. A higher percentage of respondents this year report that "all or almost all" of their teams are agile, and that agile principles and practices are being adopted at higher levels in the organization ...

August 09, 2018

For the past 13 years, the Ponemon Institute has examined the cost associated with data breaches of less than 100,000 records, finding that the costs have steadily risen over the course of the study. The average cost of a data breach was $3.86 million in the 2018 study, compared to $3.50 million in 2014 – representing nearly 10 percent net increase over the past 5 years of the study ...

August 08, 2018

Hidden costs in data breaches – such as lost business, negative impact on reputation and employee time spent on recovery – are difficult and expensive to manage, according to the 2018 Cost of a Data Breach Study, sponsored by IBM Security and conducted by Ponemon Institute. The study found that the average cost of a data breach globally is $3.86 million ...

August 06, 2018

The previous chapter in this WhiteHat Security series discussed dependencies as the second step of the Twelve-Factor App. This next chapter examines the security component of step three of the Twelve-Factor methodology — storing configurations within the environment.

August 02, 2018

Results from new Forrester Consulting research reveal the 20 most important Agile and DevOps quality metrics that separate DevOps/Agile experts from their less advanced peers ...

July 31, 2018

Even organizations that understand the importance of cybersecurity in theory often stumble when it comes to marrying security initiatives with their development and operations processes. Most businesses agree that everyone should be responsible for security, but this principle is not being upheld on a day-to-day basis in many organizations. That’s bad news for everyone. Here are some best practices for implementing SecOps ...

July 30, 2018

While the technologies, processes, and cultural shifts of DevOps have improved the ability of software teams to deliver reliable work rapidly and effectively, security has not been a focal point in the transformation of cloud IT infrastructure. SecOps is a methodology that seeks to address this by operationalizing and hardening security throughout the software lifecycle ...

July 26, 2018

Organizations are shifting away from traditional, monolithic architectures, with three-quarters of survey respondents delivering at least some of their applications and more than one-third delivering most of their applications as microservices, according to the State of DevOps Observability Report from Scalyr ...

July 24, 2018

What top considerations must companies make to ensure – or at least help improve – Agile at scale? The following are key techniques and practices to help accelerate Agile delivery rollouts and scale Agile and DevOps in the Enterprise ...

Share this