By Anna Tong

SAN FRANCISCO (Reuters) -Contrary to popular belief, using cutting-edge artificial intelligence tools slowed down experienced software developers when they were working in codebases familiar to them, rather than supercharging their work, a new study found.

AI research nonprofit METR conducted the in-depth study on a group of seasoned developers earlier this year while they used Cursor, a popular AI coding assistant, to help them complete tasks in open-source projects they were familiar with.

Before the study, the open-source developers believed using AI would speed them up, estimating it would decrease task completion time by 24%. Even after completing the tasks with AI, the developers believed that they had decreased task times by 20%. But the study found that using AI did the opposite: it increased task completion time by 19%.

The study’s lead authors, Joel Becker and Nate Rush, said they were shocked by the results: prior to the study, Rush had written down that he expected “a 2x speed up, somewhat obviously.”

The findings challenge the belief that AI always makes expensive human engineers much more productive, a factor that has attracted substantial investment into companies selling AI products to aid software development.

AI is also expected to replace entry-level coding positions. Dario Amodei, CEO of Anthropic, recently told Axios that AI could wipe out half of all entry-level white collar jobs in the next one to five years.

Prior literature on productivity improvements has found significant gains: one study found using AI sped up coders by 56%, another study found developers were able to complete 26% more tasks in a given time.

But the new METR study shows that those gains don’t apply to all software development scenarios. In particular, this study showed that experienced developers intimately familiar with the quirks and requirements of large, established open source codebases experienced a slowdown.

Other studies often rely on software development benchmarks for AI, which sometimes misrepresent real-world tasks, the study’s authors said.

The slowdown stemmed from developers needing to spend time going over and correcting what the AI models suggested.

“When we watched the videos, we found that the AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what’s needed,” Becker said.

The authors cautioned that they do not expect the slowdown to apply in other scenarios, such as for junior engineers or engineers working in codebases they aren’t familiar with.

Still, the majority of the study’s participants, as well as the study’s authors, continue to use Cursor today. The authors believe it is because AI makes the development experience easier, and in turn, more pleasant, akin to editing an essay instead of staring at a blank page.

“Developers have goals other than completing the task as soon as possible,” Becker said. “So they’re going with this less effortful route.”

(Reporting by Anna Tong in San Francisco; Editing by Sonali Paul)

“Leading cloud providers are actively exploring pathways for AI adoption, unlocking capabilities and building ecosystems through model open-sourcing, while accelerating task execution and scenario delivery via AI agent platforms,” Canalys senior analyst Yi Zhang said in a report on Thursday.

A test subject completes a task by pressing a dot when it appears on a computer screen. Photo by James Brosher, Indiana University It can take as long as 18 months for children with suspected autism spectrum or attention-deficit-hyperactivity disorders to get a diagnostic appointment with a psychiatrist in Indiana. But an interdisciplinary team led by an Indiana University researcher has developed a new diagnostic approach using artificial intelligence that could speed up and improve the detection of neurodivergent disorders.

Psychiatrists, who currently use a variety of tests and patient surveys to analyze symptoms such as communication impairments, hyperactivity or repeated behaviors, have no widely available quantitative or biological tests to diagnose autism, ADHD or related disorders.

“The symptoms of neurodivergent disorders are very heterogeneous; psychiatrists call them ‘spectrum disorders’ because there’s no one observable thing that tells them if a person is neurotypical or not,” said Jorge José, the James H. Rudy Distinguished Professor of Physics in the College of Arts and Sciences at IU Bloomington and member of the Stark Neuroscience Research Institute at the IU School of Medicine in Indianapolis.

That’s why José — in collaboration with an interdisciplinary team of scholars, including IU School of Medicine Distinguished Professor Emeritus John I. Nurnberger and associate professor of psychiatry Martin Plawecki — dedicated his recent research to improving diagnostic tools for children with these symptoms.

A new study on the use of artificial intelligence to quickly diagnose autism and ADHD, published July 8 in Nature’s Scientific Reports, details the latest step in his team’s development of a data-driven approach to rapidly and accurately assess neurodivergent disorders using quantitative biomarkers and biometrics.

Their method — which has the potential to diagnose autism or ADHD in as little as 15 minutes — could be used in schools to triage students who might need further care, said Khoshrav Doctor, a Ph.D. student at the University of Massachusetts Amherst and former visiting research scholar at IU who has been a member of José’s team since 2016.

Both he and José said their approach is not meant to replace the role of psychiatrists in the diagnosis and treatment of neurodivergent disorders.

“It could help as an additional tool in the clinician’s toolbelt,” Doctor said. “It also gives us the ability to see who might need the quickest intervention and direct them to providers earlier.”

Finding the biomarkers

Jorge José, Indiana University Bloomington James H. Rudy Distinguished Professor of Physics. Photo by James Brosher, Indiana University

In 2018, José published an autism study in collaboration with Rutgers, revealing that there are “movement biomarkers” that, while imperceptible to the naked eye, can be identified and measured in severity by using sensors.

José and his team instructed a group of participants to reach for a target when it appeared on a computer touch screen in front of them. Using sensors attached to participants’ hands, researchers recorded hundreds of images of micromovements per second.

The images showed that neurotypical patients moved in a measurably different way than participants with autism. The researchers were able to correlate increased randomness in movement with the participants who had previously been diagnosed with autism.

Improving treatment

In the years since their landmark 2018 study, José and his present team took advantage of new high-definition kinematic Bluetooth sensors to collect information not just on the velocity of study participants’ movements, but also to measure acceleration, rotation and many other variables.

“We’re taking a physicist’s approach to looking at the brain and analyzing movement specifically,” said IU physics graduate student Chaundy McKeever, who recently joined José’s group. “We’re looking at how sporadic the movement of a patient is. We’ve found that, typically, the more sporadic their movement, the more severe a disorder is.”

The team also introduced the use of a specialized area of artificial intelligence known as deep learning to analyze the new measurements. Using a supervised deep-learning technique, the team studied raw movement data from participants with autism spectrum disorder, ADHD, comorbid autism and ADHD, and neurotypical development.

This enhanced method, detailed in their July 8 Scientific Reports paper, introduced an ability to better analyze a patient’s neurodivergent disorder.

“By studying the statistics of the motion fluctuations, invisible to the naked eye, we can assess the severity of a disorder in terms of a new set of biometrics,” José said. “No psychiatrist can currently tell you how serious a condition is.”

With the added ability to assess a neurodivergent disorder’s severity, health care providers can better set up and monitor the impact of their treatments.

“Some patients will need a significant number of services and specialized treatments,” José said. “If, however, the severity of a patient’s disorder is in the middle of the spectrum, their treatments can be more minutely adjusted, will be less demanding and often can be carried out at home, making their care more affordable and easier to carry out.”