On recent webinars and research projects, we’ve discussed some metrics that engineering teams should consider measuring. It can be tricky to advise companies on this topic because it’s been our position that there is no single “correct” list of productivity metrics that work across all companies. Every company has its unique values, challenges, and approach to managing its engineering teams. That’s why it was a breath of fresh air when GitHub, The University of Victoria, and Microsoft Research released the result of their collaborative efforts and announced the SPACE Framework. If you haven’t checked out the original publication, take some time to do so; it’s a must-read for engineering leaders.
The SPACE framework was needed
The SPACE framework is a key moment for engineering management because it validates through extensive research that there is no “silver bullet” group of metrics that should be used by all to measure developer team productivity. This isn’t to say you should measure everything either, as they debunk that notion almost immediately: doing so will put your teams quickly into a state of analysis paralysis. And a “laissez-faire” attitude to your metrics strategy is not going to cut it either: just because it’s hard to figure out your approach to metrics doesn’t mean you won’t see benefits from your pursuits. What’s left after debunking these two beliefs is a middle-ground approach, and that’s exactly what SPACE is.
Our opinion of the SPACE framework
Based on our work with various company types, we at Jellyfish recommend the approach to productivity metrics in the SPACE Framework. Modern software engineering organizations need to improve their productivity tracking, and our profession needs research to debunk long outdated myths and misconceptions about metrics. This framework assists with both of those things while complimenting many ideas found in prominent works on the topic of leading software engineering teams, such as Accelerate.
More specifically, the idea of focusing on three dimensions at multiple levels of organizational hierarchy feels right to us. It allows for flexibility that accommodates the priorities of the team while providing parameters that focus teams over time. We could write several posts just acknowledging all the statements made where we agree. But instead of that, let’s address one area where we think we can supplement the conversation.
Prioritizing Business Alignment (and the Performance Dimension)
The SPACE Framework provides several examples of metrics that you could measure across each dimension (Satisfaction, Performance, Activity, Communication and Collaboration, and Efficiency). These dimensions align well to the key practices of engineering management that we believe are essential, as demonstrated with the visual below.
Specifically, we’ve come to think of engineering management as consisting of three different practices. The bottom two within the diagram, Engineering Operations and People Management, are covered by two dimensions each within the SPACE Framework. However, the top one, business alignment, was only addressed by one dimension, Performance.
Why does this matter? It means that if your team does not select Performance as a dimension, you do not measure for what we consider is a critical practice of engineering leadership. Performance might be the dimension of heightened importance amongst the framework because all companies need to ensure that engineering work is driving impactful outcomes for the business. In our review of the framework, this appeared to be the only notable point where we slightly deviate from SPACE Framework recommendations, so it’s worth discussing why we advocate for the Performance dimension and accounting for business alignment.
Why We Place Importance on the Performance Dimension
The Performance dimension is the one that considers if work is driving desired business outcomes. Additionally, what your team is working on influences the metrics of the other dimensions in a similar way that Satisfaction influences other dimensions. Depending on what your team is working on, the individual, team, and system metrics across all five dimensions will vary week over week, month over month, or quarter over quarter.
Consider a few examples. Would your Activity metrics at a team level change if you had to spend 20% more time on customer support work during a given week? Most likely, because the team behavior responding to customer incidents is much different from that of roadmap work. Let’s consider another example, Satisfaction. Could engineers feel unfulfilled if most of their work over a long period of time is spent on work that doesn’t impact the business? Based on our conversation with various engineering teams, we think it’s reasonable to assume that many engineers will be more satisfied with their work when it has an impact or priority.
One perceived detraction from choosing Performance, and (more precisely) measuring business alignment, is that you cannot easily measure it very well. However, with Engineering Management Platforms (EMPs), you can measure business alignment in an automated way through a metric called Allocation. We work with many engineering leaders who have leveraged EMPs to obtain deeper levels of understanding and insight into the productivity of their teams. So when considering the productivity dimension of the SPACE Framework, Allocation could be a metric to prioritize at the team level and within the Performance dimension.
The bottom line, we highly recommend engineering leaders begin slowly adopting the SPACE framework within their teams while also strongly considering prioritizing Performance to ensure engineering alignment to your business. But getting started and forming your metrics strategy even with this new guidance is a daunting task. So in our next post on the SPACE Framework, we’ll discuss how you can make SPACE more accessible to teams just getting started.