SPS Commerce is a publicly traded technology company that provides cloud-based supply chain management and retail data collaboration solutions
As Director of Product Engineering Operations, Amanda Swan supports 50 engineering teams and 270+ engineers. Her organization spans shared-service teams for network access, developer productivity, and AI, plus agile program management, technical product management, solution architecture, and AI principal engineers. SPS rolled out Claude across the entire engineering organization in April of 2026, and its top strategic priority is now an AI development lifecycle, with agents embedded in software delivery.
SPS uses Jellyfish to build the visibility it needs to show what its AI investment is returning, to replace manual reporting with metrics leaders trust, and to reorganize teams around the work that matters most.
Measuring AI ROI When the Unit of Work Keeps Changing
SPS’s AI journey has run for three years: an internal AI chat, then Copilot, then a year comparing Cursor, Copilot, and TAD, a homegrown agent. After the org-wide Claude rollout in April of 2026, the question changed. “It’s not, are people using it? It’s, are we getting faster?” Swan says. With token spend rising month over month, proving ROI is “the hottest topic” heading into budget planning.
The challenge is that AI has redefined the work itself.
pull quote
"Seven months ago, they would have been portfolio epics that would have taken us three to six months to deliver," Swan says. "Now it may take just 60 days. It's hard to compare because the playing field has just changed so much."
Jellyfish helps leaders look beneath the org-wide average. Rolled up across 270+ engineers, the numbers move slowly; at the team and group level, the step changes from AI are visible. “We have pockets of success all over the place,” Swan says. “The fact that we have the metrics and are keeping an eye on it will allow us over the next few months to really show what that looks like.”
Replacing Reports Nobody Fully Trusted
Before Jellyfish, SPS ran on a SAFe framework, with Jira Align and Power BI behind quarterly planning and executive reviews. Swan has been at SPS for eight years and led its original move to SAFe, so she knew the reporting stack’s limits firsthand.
"I had an FTE that was literally just working on Power BI reports," Swan says. "It was really manual and really kind of shaky, which didn't always feel good when you're trying to present to the executives and something goes wrong."
The deeper problem was confidence. SPS had pockets of data on velocity and throughput, but team-level metrics were hard to pull, and directors didn’t share a common set. “It’s not like we never had metrics,” Swan says. “It was more, did we have full confidence that we could answer it as a director group, all consistently?”
A Plug-and-Play Rollout to 45 Teams
SPS was weighing a move from Jira Align to plain Jira when Jellyfish came in. Metrics out of the box, aligned to industry-standard definitions for allocation, cycle time, and throughput, removed the guesswork about what to measure. Benchmarks showed how SPS compared to the industry, and former SPS colleagues already using Jellyfish showed the team what was possible.
"When we did the POC, it was such an easy and quick transition to just operate," Swan says. "It wasn't like a four-month overhaul of all of our systems and process. It was plug and play. And now we have metrics."
With leadership bought in, SPS rolled Jellyfish out to managers across 45 teams, with every engineer’s data connected. Swan was deliberate about adoption: a manager community of practice led by early champions, one shared set of KPIs across directors, and monthly checkpoints with directors and the VP. “We didn’t just roll out a tool and say, go have at it,” she says. “It was literally one of the quickest tools to onboard and just get going.”
Using Allocation Data to Redesign the Organization
Allocation changed the conversations SPS leaders were having about business priority: were people working on the most impactful things, and did teams need to shift? Those conversations led SPS to reorganize into focused pods aligned to its biggest strategic initiatives, replacing teams that had split capacity across many efforts. 20 or more engineers now work on AI full-time within those pods.
Jellyfish keeps the pods honest. “Jellyfish shows that while we said we have three people in this pod, there’s actually one of them that’s splitting their time with something else,” Swan says. “It’s nice to be able to have those conversations: is this the right person, or do we need somebody else who can drive focus?”
The redesign came with new rhythms, including weekly demos and shorter sprints. The Jellyfish MCP and Data Hub let Swan’s team build the specific reports those conversations need, and the product organization uses the same allocation data to break down investment by feature and portfolio epic.
A Partner That Keeps Pace
Swan values that Jellyfish evolves as fast as SPS does. “You all don’t just deliver something and then hope that lasts forever,” she says. “You’re constantly thinking ahead about things the industry is facing.”
Her advice to leaders earlier in their own AI transformation is direct:
"Don't expect the metrics to catch up to the story right away. You've got to add in org design, weekly demos, governance work, things you can act on now. But also constantly evaluate your metrics. If you set it and forget it, you're going to have a lot of cleanup to do."
As SPS embeds agents across its development lifecycle, it is looking to Jellyfish for visibility into what those agents are doing, how they contribute to the business, and where AI investment is paying off.
Key Takeaways
Before Jellyfish
- Manual, frequently questioned reporting: A dedicated FTE maintained Power BI reports that regularly broke after updates, making every executive update feel shaky.
- No consistent team-level metrics: Velocity and throughput lived in Jira Align, but team-level data was hard to pull and directors lacked a shared set of KPIs.
- Rising AI spend, unclear ROI: After rolling out Claude org-wide, token spend grew month over month with no clear way to tie it to delivery outcomes.
With Jellyfish
- Trusted metrics across 45 teams: Industry-standard metrics and benchmarks came out of the box, rolled out quickly after a short POC with every engineer’s data connected.
- Strategic pods shaped by allocation: Allocation data drove a reorganization into focused pods and flags when pod members are split across other work.
- Team-level visibility into AI impact: Leaders can see where AI is driving step changes, even as work that once took 3–6 months now ships in about 60 days.
