A global financial services provider operating mission-critical infrastructure for high-volume, regulated transactions needed to move beyond guessing whether AI-assisted development was working. Facing board-level scrutiny on AI return on investment (ROI), the organization selected Jellyfish’s Software Engineering Intelligence platform procured through Amazon Web Services (AWS) Marketplace to measure AI adoption and impact across 2,500 engineers.
The result: Board-level AI ROI reporting shifted from estimated to measured, a failed homegrown analytics pipeline was retired, and side-by-side comparison of AI-assisted PRs versus non-AI-assisted PRs across cycle time and throughput allowed leadership to see whether the AI investment was translating into measurably faster delivery.
For the first time, the CIO and CFO can answer ‘Is AI working?’ with a defensible, data-backed view at the team, group, and org-level replacing the previous practice of estimating ROI for board reporting.
About the Customer
This customer is a global financial services provider operating mission-critical infrastructure for high-volume, regulated financial transactions across thousands of institutional clients worldwide. Engineering velocity, predictability, and auditability are not optional in this environment – they are regulated outcomes.
Following an enterprise-wide modernization mandate from the chief information officer (CIO), the company set out to scale AI across an engineering organization of approximately 2,500 developers. AI-assisted development was deployed through the company’s existing AWS infrastructure and procured through AWS Marketplace as part of a broader, multi-year AWS investment.
Customer Challenge
Access to AI tools was not the hard problem. Instead, the company needed to prove that AI investments were actually improving engineering productivity and delivering business outcomes.
Before working with Jellyfish, the organization relied on stale data, manual reporting, and homegrown dashboards to answer questions about how engineering was operating. Leadership could not clearly see where teams were spending time, where delivery was slowing down, or where targeted intervention was needed. Most critically, there was no credible way to connect AI usage to engineering productivity or business outcomes.
That gap was also felt at the top. The CIO had made AI-assisted development a strategic priority, and when the executive team was asked by the board and internal stakeholders what ROI those investments were actually returning, the answer was estimated rather than measured.
The organization had attempted to solve this problem with a custom internal analytics pipeline built on a general-purpose business intelligence stack. After more than a year of effort, the build proved too brittle and time-intensive to produce the trusted, defensible answers the executive team needed. The company concluded it required a purpose-built measurement layer, one capable of surviving the security, privacy, legal, and disaster-recovery scrutiny of a tier-1 regulated financial enterprise.
Goals and Objectives
Engineering and executive leadership aligned on four measurable objectives:
- Replace estimated AI ROI reporting with data-backed, board-ready measurement
- Understand AI tool adoption at the team, role, and individual contributor level, and direct enablement resources to where rollout was lagging
- Quantify whether AI-assisted development was translating into faster cycle times and higher PR throughput
- Determine whether AI generated capacity was being reinvested in roadmap work versus unplanned or maintenance work – a question the CFO and CIO had been asking for two quarters
Partner Solution: Jellyfish and AWS
After more than 18 months of regulated procurement, security, privacy, legal, and disaster-recovery review, the company selected Jellyfish’s Software Engineering Intelligence platform procured through AWS Marketplace as a multi-year, private offer. The Jellyfish products deployed include Engineering Management Platform (EMP) Professional, DevFinOps, and AI Impact.
Native Integration with AWS AI Services
Jellyfish ingests AI coding telemetry directly from the customer’s Amazon Simple Storage Service (Amazon S3) bucket and Amazon CloudWatch logs, including granular usage metrics, code suggestion acceptance rates, chat interactions, code fixes, and code reviews. This telemetry is mapped back to GitHub and Jira to surface actual delivery impact: PR cycle time, throughput, and where AI-generated code is shifting engineering capacity from technical debt to roadmap work.
The integration works seamlessly across the customer’s current Amazon Q Developer footprint and Kiro, AWS’s agentic development environment built for spec-driven development, with no re-instrumentation required as the organization migrates forward.
AWS Services Used
The solution leverages the following AWS services:
- Amazon Q Developer and Kiro: AI-assisted development in the IDE and CLI, with agentic coding, inline chat, terminal integration, and Model Context Protocol (MCP) support
- Amazon Bedrock: Foundation model routing for Jellyfish AI Assistant and coding assistant capabilities
- Amazon S3 and Amazon CloudWatch: AI coding telemetry ingestion
- Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Kubernetes Service (Amazon EKS), AWS Lambda, Amazon Relational Database Service (Amazon RDS), and Amazon DynamoDB: Jellyfish platform infrastructure
- Amazon SageMaker: Core machine learning workloads
- AWS CloudTrail and AWS X-Ray: Observability and auditability, supporting regulated compliance requirements
- AWS Marketplace: Procurement path via the customer’s existing Enterprise Discount Program (EDP)
Why the Customer Chose to Work with Jellyfish
Two factors made the fit decisive. First, Jellyfish’s native, deep integration with the AWS AI development stack meant the customer could measure AI impact without building or maintaining custom instrumentation. Second, Jellyfish has run 100% on AWS since 2017, giving the platform the multi-availability zone architecture and high-availability disaster recovery posture required to clear the bar for financial services procurement.
AWS played an active role in moving the deal forward. The AWS Partner Solutions Manager helped align Jellyfish with the customer’s 2026 modernization priorities, and the AWS AI program team confirmed that Jellyfish was the right AWS Partner to measure AI-assisted development impact at this scale.
AWS provides the AI services and the procurement path. Jellyfish provides the proof.
Results and Benefits
This deployment is one of the largest in-production Jellyfish AI Impact implementations in financial services. The measured outcomes are already shaping how the customer scales AI across the rest of the organization.
- 2,500 engineers under unified measurement. EMP Professional, DevFinOps, and AI Impact replaced a patchwork of stale dashboards and monthly manual reports with a single source of truth.
- Homegrown analytics pipeline retired. More than a year of internal investment in a general-purpose business intelligence stack was replaced by Jellyfish’s purpose-built measurement layer, freeing platform engineering capacity to focus on AI enablement rather than dashboard maintenance.
- Board-level AI ROI reporting moved from estimated to measured. The CIO and CFO now answer “Is AI working?” with a defensible, data-backed view at the team, group, and org level.
- Side-by-side comparison of AI-assisted PRs versus non-AI-assisted PRs across cycle time and throughput, so leadership can see whether the AI investment is translating into measurably faster delivery.
- Days of manual financial reporting are eliminated each month. DevFinOps software capitalization automation replaced engineering-team surveys with a fully auditable trail tied directly to GitHub and Jira data, material in a regulated environment where auditability is a compliance requirement.
- A scalable measurement framework spanning the full AWS AI stack. Amazon Q Developer today, Kiro tomorrow, and Amazon Bedrock-routed coding assistants as they come online, with Jellyfish serving as the consistent measurement layer across all of them.
Top AI adopters achieve up to 2x PR throughput and measurably faster cycle times versus non-AI workflows, a benchmark the executive team now uses to set adoption targets organization-wide.
What’s Next
The customer is already expanding measurement to additional engineering teams beyond the initial 2,500-engineer cohort, on a path toward broader organization-wide coverage. As Kiro becomes the standard AI-assisted development environment and additional Amazon Bedrock-backed coding assistants are introduced, Jellyfish is the layer the organization uses to govern adoption, measure impact, and report ROI to the board.
This is what enterprise AI looks like when it moves past experimentation: AWS provides the AI services and the procurement path. Jellyfish provides the proof.
About Jellyfish
Jellyfish is the leading intelligence platform for AI-Integrated engineering, helping more than 1,000 companies including DraftKings, Box and Blue Yonder, leverage AI to transform how they build software. By combining the industry’s deepest engineering dataset with context-rich intelligence, Jellyfish helps R&D organizations understand what’s driving impact, adopt proven industry best practices, and make smarter decisions across AI adoption, planning, delivery, and engineering performance. Learn more at jellyfish.co.
Customer Details
- Industry: Financial Services
- Size: ~2,500 engineers (expanding organization-wide)
- AWS Services: Kiro, Amazon Q Developer, Amazon Bedrock, AWS Marketplace, Amazon S3, Amazon CloudWatch, Amazon EC2, Amazon EKS, AWS Lambda, Amazon RDS, Amazon DynamoDB, Amazon SageMaker, AWS CloudTrail, AWS X-Ray
- Jellyfish Products: EMP Professional, DevFinOps, AI Impact
