April 26, 2022
Accelerate and the SPACE Framework, two highly regarded research projects from the DORA team, speak to the value and promise of data-driven engineering management. These foundational works have left a significant impact on the engineering community by fostering a desire amongst engineering leaders to adopt a more data-driven approach to decision-making. But recent research suggests that there’s a deep disconnect between leaders’ intentions to embrace a data-driven approach and their actual progress toward that goal.
The 2022 State of Engineering Management Report found that 67% of leaders today strongly desire to measure team performance metrics, but only 27% can successfully measure them. Naturally, this begs the question: what’s keeping them from getting started? Are there barriers to entry, and if so, what are they?
In this post, we’ll examine the perceived barriers to entry of the data-driven engineering management approach, and what progress we’ve made in recent years to drastically reduce them.
Access to Engineering data has been difficult or limited
Until very recently, the primary obstacle to data-driven engineering management has been the difficulty in obtaining accurate engineering metrics in a sustainable way. Today, this is only a perceived obstacle that teams can overcome thanks to a few key breakthroughs in technology.
Just as cloud computing enabled the DevOps movement, Engineering Management Platforms (EMPs) help realize the promise of accurate and seamless data-driven insights. An EMP is a software platform that provides data-driven visibility into the engineering team’s investment of time and resources, operational efficiency, and deliverable progress in order to inform leaders to make better business decisions and deliver results. You can see a tangible example of one within Jellyfish’s product tour. Prior to EMPs, some engineering organizations like Gainsight, were forced to build their own engineering management platforms internally. Teams dedicated significant time and resources to these projects because the problem was pervasive, worth solving, and without a clear solution.
The 2022 State of Engineering Management Report revealed that nearly 53% of organizations tracking engineering metrics are still taking this approach. Other teams (42%) have developers manually track metrics in siloed spreadsheets. Most managers will admit this approach is strongly resisted by teams, and few would ever recommend it. It’s widely considered inaccurate, time-consuming, and detracting from team morale. In short, there is still a lack of awareness that Engineering Management Platforms can drastically reduce time, resources, and total effort required to adopt data-driven practices.
The Value of Data-Driven Decision Making Within Engineering Teams
The second barrier to entry is a valid question that all leaders should be asking: how do our teams use data to lead and manage more effectively? Put simply, now that we have this data, what do we do with it? What data is most important to my teams and how do we use it to improve?
These questions are part of an ongoing discussion in the engineering leadership community, and the answers are specific to each organization that asks them. For years, there was a prevailing notion that what matters cannot be measured; today, leaders are beginning to understand that you can reap the benefits of data-driven management without sacrificing the benefits of continuous, direct feedback loops. They are a complement to one another, not a detraction. Data helps identify the topics or potential challenges to ask your teams about, while the direct feedback fills in the inevitable gaps that the data cannot immediately explain. We recently published a case study that demonstrates a perfect example of this. Our own team used our Engineering Management Platform to run an engineering operations experiment that ended up cutting issue cycle time by 21%.
The findings from the 2022 State of Engineering Management Report clearly reflect the impact of data-driven decision-making within engineering teams. The types of work that data-driven teams spend time on vary drastically from those that don’t. Teams spend 29% more time on innovation work, while spending 48% less time on what team’s considered “unplanned work.” Additionally, engineering teams using EMPs saw benefits across every operational metric analyzed as a part of the study. Engineers made nearly 10% more commits last year than the average team, merged 10% more PRs, and resolved more than 14% more issues. These metrics are even more impressive when you consider that cycle times also decreased by 17% during the same period. EMPs are enablers of these operational efficiency gains, and they’re still a little-known secret to most software teams across the globe.
Why wasn’t Engineering “all-in” from the start?
So why was Engineering the last department to adopt a data-driven approach? Aggregating, compiling, and making sense of all the data created by engineering teams has always been an enormous task. The sheer volume of data output across the tools that engineers use can be tremendously overwhelming, and it’s difficult for most teams to bring it all together to tell a cohesive story about what’s happening in their software organization. This is why the red, yellow, and green traffic lights have been the de facto standard of every engineering leader’s executive update presentation for the past 30 years. Only recently have we introduced a better solution than dedicating an entire division of your organization to manually compiling engineering data.
Engineering might be the last to adopt a data-driven approach, but that’s only because software engineering is the most complicated (yet vital) part of almost all successful modern businesses. Now that technology has caught up to the need, it’s never been easier for engineering leaders to get started. And as the age of software continues to progress, the day will come when data-driven management will not be just an engineering aspiration, but rather a business imperative.
