Breaking Down Deployment Frequency

The software development process is in a state of constant evolution. Expectations of development teams continue to increase even as practices become increasingly complex and more decentralized. Today we’re seeing proliferation of tools, languages, and strategies across organizations that comprise teams and projects scattered all across the world. And so it’s critical for  teams to track data points to understand where blockers are in the software delivery process in order to identify and remediate underperforming efforts and keep software releases flowing smoothly. DORA metrics help navigate this massive puzzle of optimizing software delivery. 

The DevOps Research and Assessment (DORA) group has shown us that elite performing teams will measure and optimize around four key metrics:

• Lead time for changes
• Deployment frequency
• Time to recover
• Change failure rate

While DORA metrics are best taken in context with one another, this post will be a deep dive into deployment frequency. We will give deployment frequency some context and define it, discuss how it’s measured, and why it’s a critical metric for modern software development organizations to track. 

What is deployment frequency?

Deployment frequency is simply the tempo by which an organization successfully releases software to production. This metric supports the fundamental value proposition of agile methodology, that the goal of a modern software delivery organization is to deliver software continuously to maximize value to the end user. This consistent, iterative change associated with continuous deployment aligns with the concept that smaller, frequent software deployments enable teams to:

• Remediate issues faster, course correct, or pivot nimbly in light of new information or updated strategy
• Reduce downtime associated with large, bulky deployments.

How to measure deployment frequency

Measuring deployment frequency is simple enough. Your engineering management platform or tracking tool should be able to collect deployment data from any pipeline tool (Jenkins, GitLab CI, Circle CI, etc.) and measure the numbers of successful deployments (or releases as they may be called) to production. The trick here is to determine the threshold for “successful” releases and/or what constitutes a release. 

Most organizations define deployments as when code is released into production or when the feature/update is released to end users. But the exact “time” when a deployment or release is determined often varies from organization to organization.The key is to keep the definition consistent across the organization so that context is consistent and comparisons are apples to apples.

Why is deployment frequency important, and what does “good” look like?

Deployment frequency is a great measure of how consistent your software delivery practices are. The ability to more frequently deploy code in smaller, albeit more consistent iterations is a fundamental tenet of DevOps. This underpins one of the primary goals of DevOps; to create an environment of continuous integration and continuous delivery. The metric itself is an excellent gauge of a software engineering organization’s capability to respond to requests and its workflow efficiency. 

The goal of any DevOps-aligned software development team should be to improve their deployment frequency. However it’s also important to contextualize the velocity of a team’s frequency within the context of their work; what may be “high frequency” for a team dedicated to fixing bugs or KTLO work may not be the same as a team focused on developing new features. 

Broadly speaking though, the common ranges for deployment frequency fall into the following subjective spectrums:

• Elite performers: deployments can take place on-demand, or at least multiple times a day or week.
• High performers: teams will deploy at least weekly up to once a month
• Medium performers: deployments will take place anywhere from once a month to once every six months.
• Low performers: deployments occur fewer than once every six months.

These performance categories may be a little subjective, but the rationale here is strong. Imagine a few scenarios: 1) you identify a security bug that compromises customer data and cannot address this issue for months, or 2) you fail to beat competitors to market with a new feature. Unfortunately, these scenarios are all too common, with organizations realizing too late that contemporary software delivery strategies are at the crux of enabling growth and ensuring security.

Getting started with Deployment Frequency and Jellyfish

Deployment frequency is foundational to DevOps strategy and is a key clue for evaluating the overall performance of a software engineering organization through the lens of DORA. Getting started with tracking and utilizing the deployment frequency metric is tough and requires a deliberate effort to ensure that the organization’s overall goals are being met. Here are some helpful tips as you monitor deployment frequency:

1. Remember that faster does NOT always equal “better.” While deployment frequency is foundational to DevOps/agile metrics tracking, context is of utmost importance. If deployment frequency is improving, congratulations(!), but be sure to question “why” that is the case. Don’t skip proper testing or negatively impact code quality for the sake of “releasing faster.”
2. Visibility and evaluating a baseline measurement is paramount. You can’t figure out where you’re going and how you’re doing unless you understand where you came from. Measure and communicate this baseline state to ensure that the entire team is on the same page in regards to HOW deployment frequency will be used within the organization, reducing potential sources of fear, uncertainty, or doubt (FUD.)
3. Intra-organizational competition one the basis of one metric is detrimental to the overall success of any organization with aspirations to be better at continuous integration and delivery. Breaking down these metrics based solely on individual performance can introduce unwelcome “gaming” of the system and may impair team cohesion, ultimately reducing the effectiveness of the team and organization as a whole.
4. Consistency is key when assessing deployment frequency. Regularly pulling the metrics in context to the other factors such as the other DORA metrics or situational objectives will align all stakeholders with a consistent definition of what deployment frequency means to the entire organization. 

And of course make sure that any tool your teams use to measure deployment frequency gives you that flexibility in how you define deployments. If you’re looking for a tool to help get you started in measuring deployment frequency or other DORA metrics, check out Jellyfish or request a demo today!