August 11, 2021
The SPACE Framework is a breath of fresh air for engineering leadership. Leaders can now find clear guidance on how to measure productivity, with a robust and thorough approach that’s backed by Microsoft Research, Github, and the University of Victoria.
But putting this guidance into action isn’t going to be so simple for many engineering organizations. At Jellyfish, we endorse the SPACE Framework’s approach; but the reality is some organizations simply have not reached the level of maturity with regards to a metrics strategy for their engineering leaders to put SPACE into action. Frankly, even though it focuses on prioritization, the framework may appear daunting, even overwhelming to implement.
It’s for this reason that we’re leaning into our experiences working with hundreds of software engineering teams to provide additional guidance on how to get started on SPACE’s approach. For most organizations, getting started in SPACE usually boils down to a few preliminary steps.
Start with self-reflection
Before you even begin to think about metrics, you need to define explicitly what you value as an engineering organization (and really as an organization broadly speaking).That may require some serious self-reflection. If you had no limitations, what would the perfect engineering organization at your company look like?
As the framework guidance specifies, what you prioritize measuring speaks volumes about your values and the culture you’re building. If you cannot explicitly define them, you have no idea what to begin measuring. Only after you know your values can you begin to stack rank the five dimensions, focus on measuring your top three, and measure them at the individual, team, and organizational levels.
This may seem obvious, too often when we ask engineering leaders about their priorities and what they want to measure, the two answers don’t align. Listing your values will set you up to compare your long-term vision to your current realities. Which brings us to our next step…
Take inventory
Once you understand your org’s long-term team vision, take a quick inventory of what your teams track today and make sure they are purposeful and consistent. For example, suppose your teams count story points as a general measure of their activity levels. Are you certain all of your teams are valuing story points in the same way? We’ve worked with enough customers to know that most teams aren’t. In those cases, story points are not the best standardized metric for your Activity dimension at the team level.
As you take inventory, ask these types of critical questions before assuming you already “check the box” for a specific dimension. Once you know what you currently can measure, you can form a plan for how to build on your tracking capabilities.
Focus on continuous improvement
When it comes to adopting new metrics, remember that some will be naturally easier to measure than others. Given multiple options of where to invest your time/energy measuring, we’d recommend pursuing the metrics that are easier to begin tracking, at least early on, as long as you’re not straying from those three focus dimensions, and your long-term goals.
This approach prioritizes continuously achieving short-term gains while slowly fulfilling your long-term vision. Over time, it’s going to be important to show tangible incremental progress on your metrics plan. If you and your teams do not feel you’re making progress on the metrics front, it can have negative impacts on morale, and will likely hinder momentum.
Our recommendation is to prioritize continuous improvement and frequent quick wins over rolling out a robust approach all at once. Get started, feel good about your progress, and keep pushing forward.
Leverage technology to streamline operations
Assuming you’ve followed the previous steps, we recommend you begin to evaluate the sample metrics listed within the framework. Many of these and other metrics, which could be ideal for your teams to measure, are attainable through Engineering Management Platforms (EMPs). EMPs can help provide a metrics baseline for organizations getting started in SPACE, covering metrics from nearly all dimensions. They remove almost all of the time, resources, and costs needed for internally developed metrics tracking systems.
In Jellyfish, for example, viewing metrics across multiple hierarchical levels is built into the platform’s design. They can then be broken down further to the individual level with just one click.
Other views present a number of productivity metrics that you choose for the team level. In the screenshot below taken from the Jellyfish platform, metric examples from the framework such as PRs merged, PR cycle time, PR comments, and coding days are readily accessible. With the right EMP, the metrics you’re prioritizing will be measured automatically and seamlessly with minimal to no disruption to your daily engineering operations.
Fair warning: EMPs will make more metrics available to you than you’ve prioritized in the framework, and that’s okay. Leverage them for the metrics you care about, and stonewall the urge to analyze all the others. Your team does not want to become a victim of analysis paralysis or metrics overload!
Refine & customize your approach incrementally
EMPs will help provide an early metrics baseline but if you’re truly leaning into the guidance, you can’t compromise on measuring all the required metrics across the three dimensions. There might be metrics that aren’t in a technology platform like an EMP, so you’re going to need to dedicate some time and effort to measure them.
For example, consider how you might measure Satisfaction. If you select this as one of your primary dimensions, aggregating sentiment and feedback about parts of the software development process should be built into engineering operations. This could require polling, survey work, or other methodologies. These methodologies should be selected by what’s most comfortable to your teams, so they are reported accurately and within a trusted, safe, and productive environment. As you might expect, your methods for measuring satisfaction will most likely be unique to your organization, and they probably should be.
The most important thing is to get started
There isn’t a one-size-fits-all approach to a sound engineering metrics strategy. The one “silver bullet” metric does not exist. Finding the right approach for you and your team will be a journey, so don’t put off getting started. We now have clear guidance on how to measure productivity, and it’s on us to find ways to put it into action. If you get stuck along the way, don’t hesitate to reach out to one of us here at Jellyfish. We’re here to help.
