Metrics in Software Engineering

Metrics in software engineering contribute to the development, maintenance, and overall success of a software project. These quantitative measures serve as helpful tools for assessing specific aspects of a project, including its efficiency, effectiveness, quality, and general performance.

In a world where software engineering has become the foundation for modern business, the Engineering team is often left without the data or actionable insights to make business decisions. Instead, these decisions are often informed by simple traffic lights and bullet points.

For modern Engineering leaders, it’s clear that better measurement will foster alignment with product and go-to-market teams, improve the overall product experience and value for customers, and speed time to market. But exactly which data should you pay attention to? What metrics should you be tracking?

Types of Metrics in Software Engineering

Let’s put our software engineering metrics into 4 buckets:

Metrics for Investment & Capacity: For many organizations, the engineering team is the largest investment your company is making, so it’s important to ensure that this investment is being pointed in the right direction. Two metrics to track here are:

• Allocation
• Hiring & Ramp Time

Metrics for Quality: helps ensure that the software being developed can provide that value to your customers consistently. For quality, track these metrics: 

• Bugs
• Time to Resolution
• Uptime

Metrics for Process: helps teams to set proper expectations, drive alignment across functional teams, and allow for better execution and better penetration of the market. For process metrics, track:

• Cycle Time and Lead Time
• Deployment Frequency
• Task Resolution Rate Over Time

Metrics for Deliverable Progress & Forecasting: measure and report on the progress your team is making toward that value creation in order to drive alignment across all the teams in your company. Two metrics to track are:

• Completion / Burndown Percentage
• Predicted Ship Date

Now, let’s take a closer look at the 10 metrics that software engineering leaders should track. 

10 Metrics For Software Engineering Leaders to Track

1. Allocation

In order to inform decisions around investing in new products or capabilities, you must first understand how your team’s efforts are split right now, and how much capacity they have

to take on new projects. The priorities of the engineering team should align with those of your company. Allocation is a way of visualizing how close your team is to that goal by breaking down the work they do across axes that matter to the business. Most teams will want to track the category of engineering investments they are making. That way they can see how much work is currently going into innovation vs. managing infrastructure or tech debt vs. fixing bugs or customer support issues, etc. If the team is spending most of its time tracking down bug fixes or managing maintenance work, that should inform your product and engineering strategies to either manage down the amount of time on these categories and/or manage the expectations of how much new feature work can feasibly be done with the current team.

2. Hiring and Ramp Time

It’s no mystery why understanding your team’s hiring progress is important. You simply cannot plan engineering work, allocation, and delivery without first knowing the size and composition of your team. But as we all know, hiring is a long and expensive process, and it does not always go according to plan, so tracking this will help you adjust plans and expectations accordingly.

Ramp time might be a surprising metric to see on this list, but it’s one that is often overlooked, and it can have a big impact on the capacity of your team, especially in growth mode as you hire. No one starts a new job and is completely productive right away. If you’ve linked hiring to the ability to build and deliver a new product or feature (and most of us do), it’s important to know approximately how long new hires will need to ramp after hiring is done. Bake this into your assumptions during product and roadmap conversations, for operational planning sessions, and as you plan throughout the year.

3. Bugs

For the most part, bugs are an inevitable part of building software. Of course having some bugs does not inherently imply your product will not satisfy customers, but it can certainly have a big impact. By measuring bugs as a KPI, you are not trying to prevent them, but rather simply surfacing where they exist so you can catch, prioritize, and fix them in a timely manner.

Therefore, the key thing to monitor here is a breakdown of bugs by product or feature. By understanding the number and severity of bugs that exist per product or feature, and comparing that with product or feature usage among your customer base, you will have a better understanding of which bugs to prioritize fixing, and therefore where to devote your resources.

4. Time to Resolution

Since you cannot fix every incident, bug, or failure immediately, it’s important to keep track

of how long quality issues hang out in the product before being addressed. Of course, some incidents (security breaches for example) require immediate attention and should have shorter resolution times, while you can afford to wait on others. On the whole, measuring the time it takes to resolve reported bugs, failures, and other incidents will give you a sense for the team’s ability to be responsive to customer problems. Combined with a metric like net bugs, or the number of bugs reported vs. fixed which monitors how bugs are accumulating, this will give you an understanding of how well your team is managing the constant inflow of issues and how fast they can fix high priority issues in the product.

5. Uptime

In today’s constantly connected world, your customers expect technology (especially SaaS) products to be available to them at all times, whenever they need it. Unplanned downtime, or even slower delivery of services during certain times can threaten the relationship with your customers.

The importance of monitoring and maximizing uptime is especially true in industries like e-commerce where failing to deliver services can equate to loss of revenue and is an immediate threat to the business. But across industries, and especially in the modern world where just about the only way to engage customers is remotely, it’s important

to ensure your products perform well and are delivered reliably to your customers.

6. Cycle Time and Lead Time

Cycle time is a common metric touted by Agile aficionados and for good reason. It measures the amount of time that elapses from the start of work on a particular task until that task is complete and ready to be shipped. Simply put, cycle time measures the time it takes to complete work on a task. By keeping track of cycle time, you can compare planned work with similar tasks that the team has completed in the past and provide an estimate for the delivery of that functionality. It will help you better predict how long features will take to build and therefore when they should be expected to ship.

Lead time is one of the key metrics that DORA (DevOps Research and Assessment group) espouses for optimizing the speed of value delivery to customers. Most often it is measured as the time between first commit and deployment, but that definition may be increasing in scope with newly available technologies and methods of measuring.

The point of Lead time is to understand the amount of time it takes between the initiation of a change request and when that change is running in production – or how quickly we can deliver value to customers.

7. Deployment Frequency

Tracking how often your team does deployments can be extremely useful for understanding and improving on the speed with which you can deliver value to customers. Deployment Frequency is another DORA metric, and in true DevOps fashion, the goal is to do smaller deployments as often as possible. Small deployments will make testing and releasing easier, which will in turn drive down the time it takes to get new functionality in the hands of users.

8. Task Resolution Rate Over Time

As most projects are broken down into smaller bits of work that can be handled by and assigned to an engineer, it can be useful to measure how many of those pieces of work are being completed versus the number that have been created (resolution rate) over time. Many teams use a common framework in which an issue is the basic unit of work, a group of issues is an epic, a group of epics is an initiative, and so forth. In that terminology, measuring issue, epic, or initiative resolution rate gives a sense for how well your team is handling the work being assigned to them, how fast they can generally complete this work, and whether changes need to be made.

It’s important to recognize that these tasks are not a standard size. Some issues will take a day to resolve, while others will take two weeks simply due to their scope. That’s why we suggest measuring the resolution rate, and monitoring it over time to identify trends. By understanding how your resolution rate is trending you can be quicker to respond to problems that arise in the development process, and measure changes in efficiency when making changes.

9. Completion / Burndown Percentage

Burndown is another one of those common Agile concepts that can be extremely useful to track from a leadership position. Burndown measures the trend of work that has been completed vs. what remains to be done over a certain period of time.

By understanding how many hours have been worked on resolved items, what percentage of the total project those make up, and how many items remain, you’ll have a fairly accurate understanding of how much work each team member has ahead of them, how likely the team is to complete the work in the next sprint, whether the team is likely to complete the project on time, or if not, what timeline is more reasonable to expect.

10. Predicted Ship Date

Being able to provide an estimate as to when a given release, project, feature, or product will ship has its obvious benefits. It helps all interested parties in the company (that means pretty much everyone) plan around the work that they need to do to either support your team or bring new functionality to market. In many companies, these predictions can be handwavy, especially when they are done by intuition and experience alone. But it’s better to provide an estimate and need to change it than to provide nothing at all.

How to Track Software Engineering Metrics

As engineering has become the core of modern businesses, the fulcrum from which value is delivered to customers, engineering leaders will continue to gain more influence, and the need for a data-driven approach to engineering will continue to grow. The KPIs reviewed here are only a subset of the metrics engineering teams can and should be measuring. They are not an exhaustive list. But they will help guide your decision-making process and drive more effective conversations and alignment within your executive team and your business.

As the engineering organization scales or adapts to changing business objectives, the importance of measuring and aligning only grows. Aggregating these metrics can quickly become a challenge. Engineering Management Platforms (EMPs) like Jellyfish automatically aggregate engineering signals along with contextual business information in order to surface the metrics necessary to maintain a data-driven approach to engineering leadership, providing visibility into engineering investments and operations, product quality, delivery speed, and deliverable progress of your organization.

Interested in learning more about Jellyfish? Take the product tour or request a demo today.