July 12, 2023
How to Evaluate an Engineering Management Platform (EMP)
Putting it lightly, engineering is a challenging function.
In a world which has come to rely on the instant value of SaaS, the demands of engineering are becoming exponentially complex. That’s why tools that support leaders in grappling with these demands, such as an Engineering Management Platform (EMP), are in the ascendance. Contrary to the ever-growing expectations placed upon technical leaders to drive this value, the market for such support tools is still in its infancy.
And so the task of evaluating the platforms itself is a challenge. Perhaps not on the level of engineering, but another hurdle nonetheless – toss it on the pile, right?
In the face of such a task, I find it helps to structure the evaluation process.
That’s why I’m going to utilize the handy Who/What//Where/Why question format in order to pose various queries you might ask when evaluating Engineering Management Platforms. In doing so, hopefully you’ll become more able to discern what characteristics stand to provide the most immediate value to your engineering organization.
Starting with “Who do I talk to to get it to run Doom?”
Just kidding.
Who’s going to benefit from the platform?
When you’re in the market for an Engineering Management Platform, rather than looking for a stop-gap solution to a single process, it pays to understand whose life stands to be made easier with its implementation. Maybe you’re an Engineering VP on the receiving end of boardroom pressure around data. Perhaps you’re an Engineering Manager who needs more insight into your team’s work and sprint predictability. You could be in Engineering Operations and looking for ways to inform your process and workflow decisions with data. There are plenty of role-specific tools on the market, you just need to understand who’s most in need of a solution.
In any case, the best EMPs will be able to accommodate an entire array of technical profiles, offering functionality that will provide insight into business objective alignment for leaders whilst also offering operational insight for all levels of decision makers.
What dev-tools/methodologies are supported?
As the data generated by Source Control Management tools (SCMs) and Issue Trackers has become easier to wrangle, the SDLC tool market has exploded. Whatever your current development footprint may be, it’s key that the EMP you choose ingests data from the tools where engineering effort is being spent – be it writing source code, managing issues, planning product launches or just ‘meeting time’ on the calendar.
It’s not enough to evaluate solely on your existing footprint though; in a space as cutting edge and competitive as software engineering, future-proofing is a must.
The on-going industry shift from pure Agile to DevOps and beyond is a useful illustration of this. Some industry voices say DevOps is just an evolution of Agile; they’re not wrong. But undertaking DevOps doesn’t just mean tracking a new set of metrics – it’s upheaving your entire process, workflow and tooling philosophy. EMPs need to be adaptable to progressive, continuous changes such as this.
Technical flexibility is key here; “What tools/methodologies are supported?” really means “Does this tool have the ability to adapt quickly to my ever expanding set of tools and processes?”.
Where will it provide value?
Technical Leaders have been conditioned to make a little go a long way when it comes to data-driven engineering analysis. The level of data and insights provided by an EMP can be overwhelming for those who’ve never had this depth of analytics before.
It’s helpful to ask where a tool like this will provide value. As I just alluded to, this could be in the boardroom – what questions do your fellow executives normally ask here? Does the EMP you’re evaluating cover work areas like deliverables, investment categories, productivity or new hire integration, to name a few executive favorites.
Alternatively, perhaps there’s a disconnect between team level execution and management, which would require a more functional insight into operational decision making.
There are a lot of tools that focus on the latter; these tools align tightly with rendering the output of dev tools themselves because it’s much easier to derive insight in this way, rather than adding any of their own analysis. If you’re after a strictly operational tool, this isn’t an issue; but if you’re looking for something to augment strategic engineering goals, you’ll need to be more selective in the platform you choose.
Why Don’t We Just Build a Tool Ourselves?
In the not-so-distant past, when you might have been a hands-on-keyboard product builder, with a fresh pot of coffee and a fellow 10xer, you could probably have built Rome in close to a day, depending on scope creep. There’s an inherent muscle memory built-in to all former software devs when confronted with these situations.
But Despite such lofty aspirations, when faced with the decision of build vs. buy in the engineering management space, the question should be asked “Do we really want to build this ourselves?”
This approach, of course, has its benefits.
You have in-house experience regarding exactly how your organization works which doesn’t need to be translated for a 3rd party vendor. You also won’t be reliant on said vendor if your process/workflow suddenly changes. No subscription fee is just the cherry on the sundae.
But all roadmaps don’t lead to Rome and, inevitably, having to divert resources away from innovation towards maintenance of a bespoke analytics system is going to have a negative impact on your organization’s ability to stay competitive. Is it worth having to sacrifice precious engineering time to support a homebrewed solution that will only grow more difficult to maintain as your organization expands.
The exponentially rising complexity of Engineering means resource allocation is the true premium. It may be worth it to engage with an EMP vendor who’s engineering innovation is spent researching exactly this issue.
____
If you enjoyed this blog and would like a more in-depth guide to evaluating EMPs, check out the Jellyfish Buyer’s Guide to Engineering Management Platforms.
