Research has shown that artificial intelligence (AI) coding aids do not increase the productivity of all developers. In particular, it is pointed out that there may be an adverse effect of increasing working time for skilled developers.

The AI safety non-profit research institute METR said a recent randomized controlled trial found that AI-based coding tools such as GitHub Copilot and Cursor Pro reduce the speed of career developers by 19% on average. The experiment involved 16 skilled developers who have contributed to open-source projects, and a total of 246 real-world tasks were targeted for the experiment.

Participants predicted that using AI would reduce working time by an average of 24%. However, the actual measured time was the opposite. The group using AI, on average, took nearly 19% more time. “It goes against the general belief that the use of AI tools speeds up,” the METR said of the results.

The METR pointed to the increase in prompt writing and response waiting times, the limitations of AI’s contextual understanding in complex large-scale codebases, and the lack of consistency in AI-based coding assistance systems as reasons for slowing the use of AI. In fact, it was found that developers spent more time writing prompts and reviewing results for writing AI than coding. In the case of experienced developers, it may have taken more time to modify the code proposed by AI without writing it as it is, as they are familiar with the code they enjoy using.

Other previous studies have shown that AI coding tools speed up by 21% on average and up to 56%.

However, the researchers explained, “It is only an observation at the present time,” adding, “It means that not all AI coding tools increase productivity.”