As both a school of thought and a software engineering practice, DevOps emerged over the last decade as a way workplaces - from developer teams to security testers, product managers, operations staff, and other workplace departments - can become cross-functional and work to improve collaboration and communication. DevOps aims to streamline processes, reduce outages and downtime, and improve workflow.
You can monitor and measure the effectiveness of DevOps practices through data-driven metrics. The DevOps Research and Assessment (DORA) team identified four key metrics that can be used to assess the performance of an organisation: lead time, deployment frequency, mean time to restore (MTTR), and change failure percentage. This article takes a deeper look at deployment frequency.
What is deployment frequency?
Deployment frequency is the frequency of code deployment. This can include bug fixes, improved capabilities and new features. The frequency of deployment can differ across organizations that may deploy code anything from bi-yearly to monthly, weekly, or multiple times each day. In the DevOps bible Accelerate by Forsgren, Humble, and Gene, the authors state:
"We settled on deployment frequency as a proxy for batch size since it is easy to measure and typically has low variability. By ‘deployment,’ we mean a software deployment to production or an app store."
Why measure deployment frequency?
Central to DevOps is the idea of continuous delivery and shipping code as fast, small, frequent deployments. These smaller deployments make it easier to test and release code. By adopting DevOps, companies aim to maintain or increase their rate of deployment over time. Measuring deployment frequency correlates with continuous delivery and the comprehensive use of version control, meaning it offers an insight into the efficacy of DevOps practices within a company. Organizations can measure deployment frequency to compare their deployment speed over an extended period, helping to map a company's velocity and growth. By identifying specific periods where code deployment is delayed, teams can determine if there are problems in the workflow that are causing delays. Delays in workflow could result from a process that includes unnecessary steps or by a team not using the right tools, for example.
Ideally, by monitoring your metrics, you'll see an increase in deployments, faster deployments, reduced failures after deployment, and quicker recovery from failures.
Potential impacts of measuring deployment frequency
Measuring deployment frequency offers an opportunity to understand how well your existing processes function. For example, monitoring the frequency of deployment to QA and pre-production environments can help identify broader issues such as staff shortages, inefficient processes, and the need for longer testing time. Finding bugs in QA can reduce your defect escape rate (how often defects are uncovered in pre-production versus during the production process). Some teams opt to measure daily deployments per developer or per project to gain deeper insights into the team's functionality and productivity.
Measuring the frequency of deployments can also reveal the wider impacts of change to your organizational structure, personnel, or process within the same timeframe. For example, if a senior engineer leaves and is not replaced, this may reduce the speed of deployment and suggest a need to hire more experienced team members. Deployment frequency may also change when a team tries to improve other aspects of their workflow. For example, a concerted effort to reduce technical debt and remove bugs may increase subsequent deployments. In an effective organization, analyzing and responding to the bigger company priorities and practices will help lead to faster deployments.
Deployment frequency as a barometer of high performing teams
The annual Google "State of DevOps Report" surveys organizations to determine the health of DevOps' in action. It provides actionable guidance for organizations of all sizes and in all industries to improve their software delivery performance. Google categorizes teams as low, high, and elite according to their responses. The latest findings reveal that high performers will make multiple deployments per day while lower-performing IT and DevOps teams will deploy anywhere between once per month or once per six months.
According to the research, an elite group routinely deploys on-demand and performs multiple deployments per day.
The normalized annual deployment numbers range from 1,460 deploys per year (calculated as four deploys per day x 365 days) for the highest performers to seven deploys per year for low performers (average of 12 deploys and two deploys). Extending this analysis shows that elite performers deploy code 208 times more frequently than low performers.
The researchers note that four deploys per day are a conservative estimate when comparing large enterprises. Companies such as CapitalOne report deploying up to 50 times per day for a product, and Amazon, Google, and Netflix deploy thousands of times per day (aggregated over the hundreds of services that comprise their production environments).
Research by Atlassian into large enterprises found that deployment frequency is the most common method of DevOps success measurement. 75% of respondents cited improved deployment frequency as their biggest indication of success.
Conclusion
This article looked at DevOps' effectiveness across organizations by measuring deployment frequency. Deployment frequency has become a means to differentiate between low and high performing teams. When deployment frequency is regularly assessed, it can provide an organization with critical insights into their workflow and offers data-driven goals to improve upon.
What is your deployment frequency? How mature is your DevOps setup? Check it out and take the test.
