August 09, 2023
What Github Research Tells Us About AI Adoption on Software Teams
There have been many articles written about AI since the public release of ChatGPT, Dalle-E and others. But it can be difficult to parse the hype from reality. To what extent are software developers already using generative AI? If your teams are experimenting with AI are you falling behind?
The age of software is about to be disrupted by generative AI, and the role of a software engineer will change with this technological disruption. Github released new research that highlighted that there are real behavioral changes happening among software engineering teams as a result of the generative AI buzz. Developers are experimenting with AI, but the results of those experiments has not yet resulted in quantifiable operational changes to engineering teams. But this does not mean that engineering operations will not be disrupted by AI in the not so distant future.
Now is the time to begin building your team’s generative AI proficiency.
Let’s discuss the standout findings of this new research and why these findings support an evolved approach to determining engineering “operational efficiency.”
Key Generative AI Findings:
- 92% of U.S.-based developers already use AI coding tools both in and outside of work.
- Developers still say the most time-consuming thing they’re doing at work besides writing code is waiting on builds and tests.
- 70% say AI coding tools will offer them an advantage at work and cite better code quality, completion time, and resolving incidents as some of the top anticipated benefits.
AI Is Being Used at Scale
Github found that 92% of Engineering Teams are already using AI tools at work. This spans developers leveraging AI coding tools both in and outside of their work, as you can see in the image below, but the number is still eye-popping.
The adoption and use of this technology is in its infancy, and you can be sure that teams will have varying degrees of success. In a recent blog post, Andrew Lau discussed how elite engineering leaders will use AI to eliminate time spent on repetitive tasks, leave more time for focusing on product innovation, and enable the team with the appropriate tooling among other initiatives that improve the developer experience. Essentially, teams leveraging AI to gain back valuable developer time will unlock new levels of operational excellence, while increasing the total output of groundbreaking features.
Waiting on builds and tests is still a problem.
Despite industry-wide investments in DevOps, developers still say the most time-consuming thing they’re doing at work besides writing code is waiting on builds and tests.
The good news is that developers already understand and welcome the value proposition of AI.
Per the most recent research:
Developers say that AI coding tools can help them meet existing performance standards with improved code quality, faster outputs, and fewer production-level incidents.
70% say AI coding tools will offer them an advantage at work and cite better code quality, completion time, and resolving incidents as some of the top anticipated benefits.
Moving AI beyond software quality and team efficiency
In the not-too-distant future, we would expect many software teams to be leveraging AI to automate the test and build process where it’s feasible. But we also believe that the most elite software developers will re-evaluate their “existing performance standards” as a result of the newfound efficiency from AI. Developers won’t simply look to code quality, faster outputs, and fewer incidents as the benchmarks for elite development. Instead, teams will consider other “performance standards” such as:
- How tightly is work aligned with business priorities?
- What is the total impact of engineering work on the business?
- Are teams able to regularly reach committed project milestone dates?
- How is the overall health of the team and what factors are driving positive developer experience outcomes?
The next question that we will have to answer as a community: how does generative AI unlock the potential for tighter business alignment and to dramatically increase the impact of engineering on the business? This question and others will be a direct focus of upcoming Jellyfish Research efforts such as the annual State of Engineering Management Report. Stay tuned as that research releases next year.
