NAB expands CodeWhisperer access to 450 engineers

NAB had to make unspecified changes to its environment to address a lag in the delivery of code suggestions to developers trialling Amazon’s ‘pair programming’ service CodeWhisperer.

NAB's Andrew Brydon.

Chief engineer Andrew Brydon said the bank has rapidly grown its use of CodeWhisperer, from an initial proof-of-concept involving 20-to-30 engineers, to “about 450 engineers in different teams and regions today”.

Other ‘big four’ banks, ANZ and CBA, have run similar exercises but with a different service, GitHub Copilot.

Brydon said NAB’s approach with CodeWhisperer was to “try it out with teams working on code in small groups in our development environment”.

“Then, we scaled out to a bigger group of people, to really enhance what they were working on, and also started working with our production codebases to get people to start using this day to day as part of their role,” he said.

“We were focused on how it helps with code generation in the IDE, [and] what we started to see as people used the tool [was] that they were accepting more and more of the suggestions that came from CodeWhisperer.

“Fifity (50) percent of code suggestions by CodeWhisperer were being accepted by the engineers, and when we surveyed them after they’d been using it, at least 40 percent of people felt they were more productive using the tool.”

One of the early use cases for so-called ‘pair programming’ tools is to surface information in the same screen, instead of developers toggling between windows in search of answers.

Brydon suggested CodeWhisperer stopped developers toggling between the IDE and either Confluence or Google in search of answers.

He noted that it had taken some time for developers to incorporate CodeWhisperer into their workflows.

“The key thing for me, and the thing I’ve noticed the most, is it’s really quite hard to change the way people work, and I think this is true of all the GenAI tools that are available,” Brydon said.

“When you’re actually asking someone to work differently, they have to consciously think about it, and that’s the thing - it takes a bit of time for teams to actually start using tools, make it part of their rhythm, but once they do, we definitely see them working more effectively and efficiently.”

Brydon also noted that the bank had encountered delays in getting responses back to engineers that asked questions - but that these had since been fixed through changes made on NAB’s end.

“We actually had to make some tech changes within our environment as we were getting some lag just in terms of getting prompts back,” he said.

“We had to work through that because at the end of the day, the experience someone wants to have is they want the suggestions to come back quickly, and if there’s any lag, they lose interest in using it, and usage drops away.”

NAB’s development resources are based in Australia, Vietnam and India. The idea of adopting CodeWhisperer was to help teams deliver customer-facing features and improvements faster.

Ry Crozier attended AWS Summit Sydney as a guest of AWS.