November 04, 2021
It’s 2021 and the importance of data-driven decision making across every function of modern organizations cannot be overstated. For modern Engineering leaders, studies show that better measurement will foster alignment with product and go-to-market teams, improve the overall product experience and value for customers, and speed time to market. But exactly which data should you pay attention to? What metrics should you be tracking as your team’s key performance indicators (KPIs)?
In this series of posts, we will present some of the KPIs that can help technical leaders answer these questions. In this post we’ll talk about engineering quality KPIs. Part 1 covered investment KPIs, in this post we’ll talk about engineering quality KPIs.
Engineering Quality Metrics and KPIs
Once you know your engineering team is building the right products and features, it’s important to ensure that the software being developed can provide that value to your customers consistently. Quality metrics measure that consistency. Any quality problem, whether that is a bug, a glitch, or something else unforeseen, is a potential threat to successful delivery of value. Unhappy customers are not simply a problem for the revenue team. Quality issues will eventually come back to the engineers – your team. It’s important to monitor quality metrics in order to minimize customer impact and ultimately to maximize customer satisfaction and retention.
Three important KPIs to keep a close eye on are: Bugs, Time to Resolution, and Uptime.
Number and Severity of Bugs
For the most part, bugs are an inevitable part of building software. Of course having some bugs does not inherently imply your product will not satisfy customers, but it can certainly have a big impact. By measuring bugs as a KPI, you are not trying to prevent them, simply surfacing where they exist so we can catch, prioritize, and fix them in a timely manner.
Therefore, the key thing to monitor here is a breakdown of bugs by product or feature. By understanding the number and severity of bugs that exist per product or feature, and comparing that with product or feature usage among your customer base, you will have a better understanding of what bugs to prioritize fixing first, and therefore where to devote your resources.
Time to Resolution
Since you cannot fix every problem or bug immediately, it’s important to keep track of how long quality issues hang out in the product before being addressed. Of course, some issues (security breaches for example) require immediate attention and should have shorter resolution times, while you can afford to wait on others. On the whole, measuring the time it takes to resolve reported bugs, glitches, and other issues will give you a sense for the team’s ability to be responsive to customer issues. Combined with a metric like net bugs, or the number of bugs reported vs. fixed which monitors how bugs are accumulating, this will give you an understanding of how well your team is managing the constant inflow of issues and prioritizing quality in the product.
Uptime
In today’s constantly connected world, your customers expect technology (especially SaaS) products to be available to them at all times, whenever they need it. Unplanned downtime, or even slower delivery of services during certain times can threaten the relationship with your customers. The importance of monitoring and maximizing uptime is especially true in industries like e-commerce where failing to deliver services can literally equate to loss of revenue and is an immediate threat to the business. But across industries, and especially in the world of 2021 where just about the only way to engage customers is remotely, it’s important to ensure your products perform well and are delivered reliably to your customers.
The metrics discussed here do not represent an exhaustive list of KPIs to track, but if you can track all of them, you’ll be well on your way with the data you need to provide continuous value to your customers and your company. In our next post, we’ll look at KPIs for optimizing process.
Want to learn more about which metrics and KPIs you should be tracking? Check out our eBook: 10 KPIs Engineering Leaders Should Track
