October 08, 2024
Using data to make decisions is far from groundbreaking – it’s usually standard operating procedure. But for software companies, where R&D teams make up both the bulk of expenses and revenue, data and insights have long been inaccessible. Instead, engineering and finance leaders have been left with simple traffic lights and bullet points to make major strategic decisions.
Thankfully that’s changed with the emergence of engineering management software that can surface key insights. And there’s an appetite for this kind of data. According to the 2024 State of Engineering Management Report, metrics are a crucial piece of the puzzle for engineering leaders. While a significant majority (84%) believe they’re using metrics effectively to inform decision making today, there’s an opportunity to close the gap between those who believe data-driven insights are valuable and those who are actually realizing that value. At companies with more than 500 engineers, only 75% of leaders said they’re using metrics effectively, highlighting a need for unique approaches at scale.
For modern engineering leaders, it’s clear that better measurement will foster better alignment with product and go-to-market teams, improve the overall product experience and value for customers, and speed time to market. But figuring out exactly which data to pay attention to is less clear and determining the top engineering management KPIs is the key to success.
Let’s take a look at five top engineering management KPIs to track:
1. Allocation
Are we investing in the right areas?
In order to inform decisions around investing in new products or capabilities, you must first understand how your team’s efforts are split right now, and how much capacity they have to take on new projects. The priorities of the engineering team should align with those of your company. Allocation is a way of visualizing how close your team is to that goal by breaking
down the work they do across axes that matter to the business.
The simplest approach is the most effective way to start measuring allocation. Focus on measuring the amount of effort the engineering team spends on things that are intended to grow the business (Growth), versus the amount of effort the engineering team spends on things that are meant to continue operating (KTLO), versus the amount of effort spent on unavoidable, unplanned work (Support). Where the team is spending most of its time should inform your product and engineering strategy. For example, if the majority of time and effort is going towards tracking down bug fixes / dealing with outages (Support) or managing maintenance work (KTLO), the strategy either needs to include a way to manage down the support and KTLO work, or acknowledge that there is a limited amount of capacity to spend on Growth work.
Allocation does not stop there – there are many additional axes to measure Allocation on. When measuring allocation by product line, you may uncover legacy products that consume outsized amounts of effort. Splitting by market segment, you may decide to focus more time and attention on enterprise features rather than those built for SMB customers, or vice versa. Other common Allocation breakdowns include Product Themes, and Business Objectives, or even more granular categories such as Projects, and Releases.
How to Measure Allocation
Measuring Allocation can be quite cumbersome, but it doesn’t have to be. Some teams roughly guess where each engineer has spent their time in simple self- or manager- reported spreadsheets. Others count story points or jira issues, by fill out detailed time cards (we don’t recommend either of those), or track actual work done by aggregating PRs, Commits and the like and associating them to the specific issue or project they belong to in an issue tracking system like Jira.
When using self reported data (like spreadsheets or timecards), or pre-work estimates like story points, or issue counts, Allocation is a high level estimate at best. An Engineering Management Platform like Jellyfish can reduce the degree of difficulty in tracking allocation for you, by automatically ingesting signals from SCM tools, Issue Tracking systems, and any strategic planning or roadmapping tools to visualize your chosen Allocations on an ongoing basis and in an accurate manner.
2. Time to Resolution
How fast can we fix problems when they arise?
Since you can’t fix every incident, bug, or failure immediately, it’s important to keep track of how long quality issues hang out in the product before being addressed. Of course, some incidents (security breaches for example) require immediate attention and should have shorter resolution times, while you can afford to wait on others. On the whole, measuring the time it takes to resolve reported bugs, failures, and other incidents will give you a sense of the team’s
ability to respond to customer problems. Time to resolution pairs well with quality centric metrics (such as the number of defects resolved) to paint a full picture of how your team deals with quality challenges – from the frequency at which they need to respond to the speed at which they can.
How to Measure Time to Resolution
Time to resolution is best measured by identifying the time between a quality issue beginning to be experienced by customers
3. Deployment Frequency
How fast and iterative are we at delivering value?
Tracking how often your team does deployments can be extremely useful for understanding
and improving on the speed with which you can deliver value to customers. Deployment Frequency is another DORA metric, and in true DevOps fashion, the goal is to do smaller deployments as often as possible. Small deployments will make testing and releasing easier, which will in turn drive down the time it takes to get new functionality into the hands of users.
This also allows for a more iterative approach to delivering value by giving more opportunities
for faster feedback from your users which you can then incorporate into future deployments. Ultimately, measuring deployment frequency will help drive this process to increase the overall value that you are able to provide customers, and make the delivery of that value faster.
4. Completion / Burndown Percentage
Are we tracking to deliver on time?
Burndown is another one of those common Agile concepts that can be extremely useful to track from a leadership position. Burndown measures the trend of work that has been completed vs. what remains to be done over a certain period of time.
By understanding how many hours have been worked on resolved items, what percentage of the total project those make up, and how many items remain, you’ll have a fairly accurate understanding of how much work each team member has ahead of them, how likely the team is to complete the work in the next sprint, whether the team is likely to complete the project on time, or if not, what timeline is more reasonable to expect.
5. Predicted Ship Date
When can we start selling?
Being able to provide an estimate as to when a given release, project, feature, or product will ship has its obvious benefits. It helps all interested parties in the company (that means pretty much everyone) plan around the work they need to do to either support the team or to bring new functionality to market. In many companies, these predictions can be handwavy, especially when they’re done by intuition and experience alone.
But if you can take a data-driven approach that factors in things like scope creep over time, metric averages from other projects of similar size and scope, and burndown percentage through the life of the project, you’ll be able to give a more confident estimation. Advanced Engineering Management Platforms like Jellyfish can even incorporate these factors into a prediction or forecast for each of your deliverables.
Data-driven Engineering
As more companies rely on engineering as a primary value driver, the need for a data-driven approach to engineering grows. The KPIs mentioned here are just a subset of the key performance indicators engineering leaders can and should measure. To view more engineering management KPIs, you can access the full guide here.
