Connect with us

AI Insights

Artificial intelligence and the wellbeing of workers

Published

on


Previous work

Our paper builds on and extends recent research examining the impact of AI on labor market outcomes and worker well-being. Prior studies have largely found positive effects of AI exposure on employment and wages at the industry or occupational level. For example, Felten et al. (2019)22 document small wage gains in AI-exposed occupations in the US, while Gathmann and Grimm (2022)23 find a positive relationship between AI exposure and employment in Germany, particularly in the service sector. Acemoglu et al. (2022)20 analyze U.S. labor markets using establishment-level vacancy data from 2010 onward, finding rapid AI adoption, particularly in firms with tasks suited for automation. Their findings suggest that AI-exposed firms increase AI-related hiring while simultaneously reducing non-AI hiring and altering skill demands. Despite these firm-level shifts, they find no significant impact on overall employment or wage growth in AI-exposed industries and occupations, indicating that AI’s displacement effects may currently outweigh productivity-driven job creation. When examining the heterogeneity of results across occupations, Bonfiglioli et al. (2025)24 find evidence that AI exposure led to job losses across US commuting zones, particularly for low-skill and production workers, while benefiting high-wage and STEM occupations. Unlike other technologies, AI’s impact is driven by services rather than manufacturing, contributing to automation of jobs and rising inequality.

At the same time, researchers have raised concerns that AI could accelerate the erosion of middle-class job security by automating tasks without creating sufficient new roles for human workers25.Whether AI complements or displaces human labor depends not only on the extent of automation but also on which tasks are automated and which workers are affected26. In this regard, Brekelmans and Petropoulos (2020)27 highlight that mid-skilled occupations are among the most vulnerable to AI-driven disruption. However, some scholars suggest that AI can help reduce job performance inequalities by improving efficiency and decision-making support10,28.

While existing studies have extensively examined labor market effects of AI, relatively little attention has been paid to its broader impact on worker well-being. Our work addresses this gap, contributing to a growing body of research exploring the effects of automation technologies on workers’ health and psychological outcomes.

Previous research on the effects of automation on well-being has primarily examined industrial robots and mechanized systems13,29. However, AI represents a distinct form of automation, as it relies on computer-based learning and cognitive processing rather than physical manipulation9. Unlike traditional robotics, which primarily displaces routine manual tasks, AI has the potential to automate non-routine cognitive tasks that were once considered resistant to mechanization22,25,30. This shift suggests that AI may not only disrupt routine jobs but also reshape knowledge-based professions, placing even highly educated workers at risk of automation-driven changes. Moreover, as AI systems become more capable of complex reasoning, decision-making, and problem-solving, their impact on the labor market is likely to extend beyond simple task automation, influencing job design, skill requirements, and career trajectories across various industries.

The influence of AI on employee well-being may operate through multiple channels. On one hand, AI-driven automation can reduce physical strain in labor-intensive jobs, potentially improving physical health. On the other hand, AI adoption can increase cognitive and emotional demands in knowledge-intensive occupations, altering job content in ways that either enhance or undermine job satisfaction. Additionally, shifts in workplace dynamics—such as changes in perceived job security, workplace autonomy, and the sense of purpose derived from work—may further affect workers’ experiences with AI.

Worker attitudes toward AI remain mixed. Some global surveys indicate rising concerns about the consequences of AI on job opportunities31 yet a recent Pew study finds that US workers in AI-exposed industries do not perceive AI as an immediate threat32. AI has the potential to enhance productivity and complement human skills, but it can also displace workers in certain roles. As with past technological revolutions, the ultimate labor market impact of AI will depend on the evolving balance between complementarity and substitution between AI and human labor20,33,34. Moreover, AI alters the nature of work itself, influencing job satisfaction, professional identity, and the perceived dignity of labor35.

Whether the beneficial effects of AI on labor market outcomes offset or even outweigh its displacement effects remains an empirical question—particularly in the short term, as workers and labor markets undergo a period of transition and adaptation to this new general-purpose technology.

Conceptual framework

This study builds on task-based theories of technological change33,36, which conceptualize AI as a transformative force that reallocates tasks between humans and machines. Similarly to robots, AI automates routine and physically demanding tasks, enabling workers to transition into higher-skill, cognitively demanding roles. However, unlike robots, AI can also automatize non-routine tasks30. The dual role of AI—as both a complement and a substitute to human labor—is central to understanding its heterogeneous effects. In our context, complementarity arises when AI reduces physical strain and augments workers’ capabilities, enhancing productivity and job satisfaction. Conversely, substitutability occurs when AI displaces workers, increasing job insecurity and workplace anxiety.

AI adoption, as documented by Acemoglu et al. (2022)20, follows a pattern where firms with task structures conducive to AI integration experience substantial labor reconfigurations. While automation’s displacement effects have been widely studied, the role of AI in hazardous and physically strenuous occupations offers a distinct perspective on its labor market consequences. We hypothesize that AI’s integration into hazardous tasks—such as those involving exposure to toxic environments, heavy lifting, or repetitive strain—can reduce workplace injuries and long-term health risks. This hypothesis aligns with prior research on automation’s potential to enhance worker safety by shifting high-risk activities to machines, thereby improving physical well-being13.

Acemoglu et al. (2022)20 highlight that AI-exposed establishments tend to reduce overall hiring, indicating that productivity gains from AI do not necessarily lead to net job creation. This trend, coupled with AI’s ability to automate both routine and certain non-routine cognitive and abstract tasks, suggests that a broader range of workers—including those in knowledge-based professions—may experience job displacement pressures. Such uncertainty can have significant psychological consequences, as fears of automation-related job loss contribute to chronic stress, financial insecurity, and diminished workplace morale. The effects are likely to be unevenly distributed, disproportionately impacting low-wage workers while also altering career trajectories for higher-skilled professionals, ultimately reinforcing patterns of economic inequality across different skill levels.

Thus, while AI adoption holds promise for improving workplace safety and productivity, its net effect on worker well-being remains uncertain. Whether the benefits—such as reduced workplace hazards and improved job quality—outweigh the disruptions caused by job displacement and economic instability depends on how AI is integrated into labor markets. This duality underscores the need for policies that not only facilitate AI integration in ways that protect workers’ health but also address the economic and social ramifications of AI-driven labor shifts.

Furthermore, our study builds upon traditional job quality frameworks by examining AI-induced transformations in physical job intensity, cognitive demands, and health outcomes. While much of the public debate on digitalization has focused on job quantity (i.e., job creation vs. job loss), its effects on job quality are equally significant and warrant further attention37,38. Martin and Hauret (2022)37 identify six key dimensions of job quality commonly examined in the literature: labor income, workplace safety, working time and work-life balance, job security, skill development and training, and employment-related relationships and work motivation. Traditional models conceptualize job quality through physical, cognitive, and emotional dimensions. However, emerging technologies—especially AI—are reshaping these dimensions in profound ways. AI-driven workplace changes can lead to new forms of cognitive strain, shifts in workplace autonomy, and evolving skill demands, all of which have direct consequences for workers’ well-being. Further research is needed to fully understand how digitalization—particularly AI—modifies job demands, psychological stress, and long-term employment conditions.

AI technologies frequently automate repetitive, physically demanding, and hazardous tasks, thereby alleviating physical labor for workers. In manufacturing, AI-powered machinery has increasingly replaced tasks such as assembly-line work and heavy lifting, while in service industries, AI tools like virtual assistants reduce administrative workloads, complementing physical task reductions. As AI shifts job responsibilities from physical execution to supervisory or decision-making roles, the overall physical strain on workers declines. Empirical evidence supports this trend; for instance, Gihleb et al. (2022)13 document significant reductions in physically demanding work as automation becomes more prevalent. Using a physical burden metric from the SOEP, we examine the link between AI exposure and reduced physical job intensity.

Finally, our study engages with theories of institutional mediation13 to examine how labor market institutions shape AI’s effects on workers. Specifically, we hypothesize that Germany’s strong labor protections, high unionization rates, and employment legislation may moderate the adverse consequences of AI on worker well-being. Institutions that provide employment security, reskilling opportunities, and worker protections may buffer against the negative impacts of automation, reducing stress and job displacement fears. This contrasts with more flexible labor markets, where AI-induced disruptions may lead to greater economic precarity.

Our contribution

Our research highlights the importance of Germany’s unique institutional context, characterized by strong labor protections, extensive union representation, and comprehensive employment legislation. These factors, combined with Germany’s gradual adoption of AI technologies, create an environment where AI is more likely to complement rather than displace worker skills, mitigating some of the negative labor market effects observed in countries like the US. Germany’s institutional framework, marked by strong labor protections, widespread union representation, and comprehensive employment regulations, plays a crucial role in shaping the effects of AI adoption. These structural safeguards, along with the country’s more gradual integration of AI technologies, foster an environment where AI is more likely to enhance rather than replace worker skills, mitigating some of the negative labor market effects observed elsewhere. However, as Bonfiglioli et al. (2025)24 highlight, AI exposure has led to job losses in the US, particularly among low-skilled and production workers, while benefiting high-wage and STEM occupations. Given AI’s distinct impact on service industries rather than traditional manufacturing, workers in routine-intensive service sectors may still face heightened risks of displacement, even within Germany’s more protective labor market. These theoretically ambiguous effects make Germany a particularly interesting case for examining the interaction between AI adoption and labor market institutions, raising an important empirical question about the extent to which these protections can shield workers from displacement while enabling technological progress.

In addition, we explore heterogeneity in outcomes based on worker characteristics (e.g., gender, education, union membership) and regional differences (e.g., East vs. West Germany), offering a more granular perspective on the potential effects of AI on labor markets and worker well-being.

Unlike studies such as Nazareno and Schiff (2021) and Liu (2023)29,39, which rely on cross-sectional data and broad measures of automation exposure40, our analysis is uniquely AI-focused. We leverage the Webb measure of exposure to AI alongside a self-reported metric from the SOEP, ensuring a more precise assessment of the effects of AI on worker well-being and labor market dynamic. Using longitudinal data from the SOEP over the period 2000–2020, we employ an event study analysis and a DiD design to address selection bias and control for individual fixed effects. This methodological approach enables us to capture long-term trends and examine nuanced outcomes such as life satisfaction, mental health, and physical health—dimensions that have often been overlooked in prior research.

AI in Germany

The roll-out of AI in Germany accelerated only recently. As noted by Gathmann and Grimm (2022)23, patent applications for AI technologies started to grow strongly only after 2015, and more significantly in 2017 and 2018. The innovation survey conducted by ZEW- Leibniz Centre for European Economic Research provides a consistent longitudinal perspective on AI adoption in Germany18. Specifically, the most recent wave of the innovation survey contains information on AI adoption percentages for 2021. AI use was not widespread before 2010, and the rate of AI adoption was extremely low before 2016. To be conservative, we chose 2010 as the beginning of our treatment period. AI adoption rates have increased substantially over the last few years. While only 2% of firms adopted AI before 2016, this number rose to 6% in 2019 and 10% in 202118. Regarding the diffusion of AI across industries, the leading adopters of AI technology in 2019 were finance (24%) and IT (21%), followed by skilled services (18%) such as legal, architecture, consulting, and research. Conversely, the laggards in AI adoption include mining (1.6%), miscellaneous business services (2.3%), and transportation (5.3%). These cross-sectoral differences in AI adoption are qualitatively reflected in our individual-level data on AI exposure from the SOEP, with IT and finance being the most exposed (see Figure A.1 in the Appendix). As deatiled in Rammer et al. (2022), 75% of the firms in the finance sector that used AI technologies in 2019 began using AI in 2016. The share of the chemical and pharmaceutical sectors is 74%, whereas that of electronics and machines is 68%.

Increasing rates of AI adoption across German firms were accompanied by the German government’s investment in AI. In 2018, the German Federal Government launched its Artificial Intelligence Strategy and pledged to invest approximately 5 billion euros by 2025 in AI development. For these reasons, Germany is an interesting country for analyzing the effects of rising exposure to AI on the well-being and health of workers.



Source link

AI Insights

China’s artificial intelligence (AI) model is rapidly eroding the share of U.S. companies such as An..

Published

on


China’s artificial intelligence (AI) model is rapidly eroding the share of U.S. companies such as Anthropic’s Claude and Google’s Gemini in the global coding market. It is expanding its influence by introducing a series of open-source products that are comparable to U.S. Frontiers in terms of performance as well as price competitiveness, which was considered a strength of existing Chinese models. In particular, the pace of expanding its presence in emerging markets such as the Middle East and South America is remarkable. Although new models are steadily being born, it is compared to Korea, which has little presence.

According to the information technology (IT) industry on the 7th, the global share of Claude and Gemini in the programming sector has steadily declined, while China’s AI has risen significantly. According to OpenRouter, as of August 11 compared to July 21, the Anthropic Claude SONET 4 share fell 15.7 percentage points in the programming area, recording the biggest drop. The Gemini 2.5 Pro and Flash also decreased by 3.6 percentage points and 4.4 percentage points.

On the other hand, Alibaba’s Qwen 3 coder grew by 16.4 percentage points during the same period, accounting for 21.5 percent of the market share. In particular, Qwen’s growth was remarkable. While DeepSeek is slowing down, it has also ranked first among Chinese models in terms of performance. Alibaba’s frontier model “Qwen3-235B-A22B-Thinking-2507” scored 64 points, ahead of DeepSeek’s latest model V3.1 (60 points), according to Artificial Analytics indicators.

Chinese start-ups are also chasing after them. Z, which was released in July, according to the same survey by OpenRouter.AI’s GLM 4.5 and Moonshot AI’s Kimi-K2 had market share of 6.1% and 3.2%, respectively, as of August 11.

Among them, Kimi-K2 attracted so much attention that it was evaluated as bringing another “deep moment.” An industry official said, “Now, most of China’s open source models, as well as DeepSeek, have competitive edge to compete with U.S. big tech models in terms of functionality beyond cost-effectiveness.”

Although there are many performances, the biggest reason why Chinese models stand out in the global market is their price competitiveness. The Qwen 3 coder costs $1 per 1 million token of input and $5 per 1 million token of output, which is cheaper than the Claude Opus 4 (input $15, output $75). The startup model is more unconventional. Z.AI’s GLM 4.5 is $0.6 per 1 million tokens input and $2.2 output, the lowest among Chinese models. MoonshotAI’s Kimi-K2 is $0.6 input and $2.5 output.

This price competitiveness is particularly strong in emerging countries such as the Middle East and South America than in the United States or Northeast Asia. Qwen and Z.Analysts say that price competitiveness is effective against the background of Chinese models such as AI showing rapid growth in the coding market in emerging countries. Similar web data showed that Qwen models, excluding China, accounted for 27.5% of traffic in Iraq, 19.1% in Brazil and 12.1% in Turkiye. Z.AI also has offices in the Middle East and Africa to supply AI solutions to local governments and state-owned companies.

The rise of China’s open-source model is not just a corporate-level result. Since the “deep shock” earlier this year, China has established an open-source strategy nationwide and provided full support to related companies. This year, Chinese companies launched a series of frontier models and their global share rose due to the government’s support.

As such, the Chinese model has emerged rapidly this year and is competing in the U.S. and global coding markets, while the Korean model’s presence is still insignificant. Although LG AI Research Institute’s recently released ‘Exemployee 4.0’ was evaluated as being at the top of the global rankings in coding performance, it is far from the actual market share. The reality is that many Korean companies use overseas models. Industry experts point out that it is urgent to strengthen the coding sector’s capabilities, one of the key areas of AI competitiveness.

[Reporter Ahn Seonje]



Source link

Continue Reading

AI Insights

AI can be a great equalizer, but it remains out of reach for millions of Americans; the Universal Service Fund can expand access

Published

on


In an age defined by digital transformation, access to reliable, high-speed internet is not a luxury; it is the bedrock of opportunity. It impacts the school classroom, the doctor’s office, the town square and the job market.

As we stand on the cusp of a workforce revolution driven by the “arrival technology” of artificial intelligence, high-speed internet access has become the critical determinant of our nation’s economic future. Yet, for millions of Americans, this essential connection remains out of reach.

This digital divide is a persistent crisis that deepens societal inequities, and we must rally around one of the most effective tools we have to combat it: the Universal Service Fund. The USF is a long-standing national commitment built on a foundation of bipartisan support and born from the principle that every American, regardless of their location or income, deserves access to communications services.

Without this essential program, over 54 million students, 16,000 healthcare providers and 7.5 million high-need subscribers would lose internet service that connects classrooms, rural communities (including their hospitals) and libraries to the internet.

Related: A lot goes on in classrooms from kindergarten to high school. Keep up with our free weekly newsletter on K-12 education.

The discussion about the future of USF has reached a critical juncture: Which communities will have access to USF, how it will be funded and whether equitable access to connectivity will continue to be a priority will soon be decided.

Earlier this year, the Supreme Court found the USF’s infrastructure to be constitutional — and a backbone for access and opportunity in this country. Congress recently took a significant next step by relaunching a bicameral, bipartisan working group devoted to overhauling the fund. Now they are actively seeking input from stakeholders on how to best modernize this vital program for the future, and they need our input.

I’m urging everyone who cares about digital equity to make their voices heard. The window for our input in support of this vital connectivity infrastructure is open through September 15.

While Universal Service may appear as only a small fee on our monthly phone bills, its impact is monumental. The fund powers critical programs that form a lifeline for our nation’s most vital institutions and vulnerable populations. The USF helps thousands of schools and libraries obtain affordable internet — including the school I founded in downtown Brooklyn. For students in rural towns, the E-Rate program, funded by the USF, allows access to the same online educational resources as those available to students in major cities. In schools all over the country, the USF helps foster digital literacy, supports coding clubs and enables students to complete homework online.

By wiring our classrooms and libraries, we are investing in the next generation of innovators.

The coming waves of technological change — including the widespread adoption of AI — threaten to make the digital divide an unbridgeable economic chasm. Those on the wrong side of this divide experienced profound disadvantages during the pandemic. To get connected, students at my school ended up doing homework in fast-food parking lots. Entire communities lost vital connections to knowledge and opportunity when libraries closed.

But that was just a preview of the digital struggle. This time, we have to fight to protect the future of this investment in our nation’s vital infrastructure to ensure that the rising wave of AI jobs, opportunities and tools is accessible to all.

AI is rapidly becoming a fundamental tool for the American workforce and in the classroom. AI tools require robust bandwidth to process data, connect to cloud platforms and function effectively.

The student of tomorrow will rely on AI as a personalized tutor that enhances teacher-led classroom instruction, explains complex concepts and supports their homework. AI will also power the future of work for farmers, mechanics and engineers.

Related: Getting kids online by making internet affordable

Without access to AI, entire communities and segments of the workforce will be locked out. We will create a new class of “AI have-nots,” unable to leverage the technology designed to propel our economy forward.

The ability to participate in this new economy, to upskill and reskill for the jobs of tomorrow, is entirely dependent on the one thing the USF is designed to provide: reliable connectivity.

The USF is also critical for rural health care by supporting providers’ internet access and making telehealth available in many communities. It makes internet service affordable for low-income households through its Lifeline program and the Connect America Fund, which promotes the construction of broadband infrastructure in rural areas.

The USF is more than a funding mechanism; it is a statement of our values and a strategic economic necessity. It reflects our collective agreement that a child’s future shouldn’t be limited by their school’s internet connection, that a patient’s health outcome shouldn’t depend on their zip code and that every American worker deserves the ability to harness new technology for their career.

With Congress actively debating the future of the fund, now is the time to rally. We must engage in this process, call on our policymakers to champion a modernized and sustainably funded USF and recognize it not as a cost, but as an essential investment in a prosperous, competitive and flourishing America.

Erin Mote is the CEO and founder of InnovateEDU, a nonprofit that aims to catalyze education transformation by bridging gaps in data, policy, practice and research.

Contact the opinion editor at opinion@hechingerreport.org.

This story about the Universal Service Fund was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Hechinger’s weekly newsletter.

The Hechinger Report provides in-depth, fact-based, unbiased reporting on education that is free to all readers. But that doesn’t mean it’s free to produce. Our work keeps educators and the public informed about pressing issues at schools and on campuses throughout the country. We tell the whole story, even when the details are inconvenient. Help us keep doing that.

Join us today.



Source link

Continue Reading

AI Insights

Examining the Evolving Landscape of Medical AI

Published

on


I. Glenn Cohen discusses the risks and rewards of using artificial intelligence in health care.

In a discussion with The Regulatory Review, I. Glenn Cohen offers his thoughts on the regulatory landscape of medical artificial intelligence (AI), the evolving ways in which patients may encounter AI in the doctor’s office, and the risks and opportunities of a rapidly evolving technological landscape.

The use of AI in the medical field poses new challenges and tremendous potential for scientific and technological advancement. Cohen highlights how AI is increasingly integrated into health care through tools such as ambient scribing and speaks to some of the ethical concerns around data bias, patient privacy, and gaps in regulatory oversight, especially for underrepresented populations and institutions lacking resources. He surveys several of the emerging approaches to liability for the use of medical AI and weighs the benefits and risks of permitting states to create their own AI regulations in the absence of federal oversight. Despite the challenges facing regulators and clinicians looking for ways to leverage these new technologies, Professor Cohen is optimistic about AI’s potential to expand access to care and improve health care quality.

A leading expert on bioethics and the law, Cohen is the James A. Attwood and Leslie Williams Professor of Law at Harvard Law School. He is an elected member of the National Academy of Medicine. He has addressed the Organisation for Economic Co-operation and Development, members of the U.S. Congress, and the National Assembly of the Republic of Korea on medical AI policy, as well as the North Atlantic Treaty Organization on biotechnology and human advancement. He has provided bioethical advising and consulting to major health care companies.

The Regulatory Review is pleased to share the following interview with I. Glenn Cohen.

The Regulatory Review: In what ways is the average patient today most likely to encounter artificial intelligence (AI) in the health care setting?

Cohen: Part of it will depend on what we mean by “AI.” In a sense, using Google Maps to get to the hospital is the most common use, but that’s probably not what you have in mind. I think one very common use we are already seeing deployed in many hospitals is ambient listening or ambient scribing. I wrote an article on that a few months ago with some colleagues. Inbox management—drafting initial responses to patient queries that physicians are meant to look over—is another way that patients may encounter AI soon. Finally, in terms of more direct usage in clinical care, AI involvement in radiology is one of the more typical use cases. I do want to highlight your use of “encounter,” which is importantly ambiguous between “knowingly” or “unknowingly” encounter. As I noted several years ago, patients may never be told about AI’s involvement in their care. That is even more true today.

TRR: Are some patient populations more likely to encounter or benefit from AI than others?

Cohen: Yes. There are a couple of ethically salient ways to press this point. First, because of contextual bias, those who are closer demographically or in other ways to the training data sets are more likely to benefit from AI. I often note that, as a middle-aged Caucasian man living in Boston, I am well-represented in most training data sets in a way that, say, a Filipino-American woman living in rural Arkansas may not be. There are many other forms of bias, but this form of missing data bias is pretty straightforward as a barrier to receiving the benefits from AI.

Second, we have to follow the money. Absent charitable investment, what gets built depends on what gets paid for. That may mean, to use the locution of my friend and co-author W. Nicholson Price II, that that AI may be directed primarily toward “pushing frontiers”—making excellent clinicians in the United States even better, rather than “democratizing expertise”—taking pretty mediocre physician skills and scaling access to them up via AI to improve access across the world and in parts of the United States without good access to healthcare.

Third, ethically and safely implementing AI requires significant evaluation, which requires expertise and imposes costs. Unless there are good clearinghouses for expertise or other interventions, this evaluation is something that leading academic medical centers can do, but many other kinds of facilities cannot.

TRR: What risks does the use of AI in the medical context pose to patient privacy? How should regulators address such challenges?

Cohen: Privacy definitely can be put at risk by AI. There are a couple of ways that come to mind. One is just the propensity to share information that AI invites. Take, for example, large language models such as ChatGPT. If you are a hospital system getting access for your clinicians, you are going to want to get a sandboxed instance that does not share queries back to OpenAI. Otherwise, there is a concern you may have transmitted protected information in violation of the Health Insurance Portability and Accountability Act (HIPAA), as well as your ethical obligations of confidentiality. But if the hospital system makes it too cumbersome to access the LLM, your clinicians are going to start using their phones to access it, and there goes your HIPAA protections. I do not want to make it sound like this is a problem unique to medical AI. In one of my favorite studies—now a bit dated—someone rode in elevators at a hospital and recorded the number of privacy and other violations.

A different problem posed by AI in general is that it worsens a problem I sometimes call data triangulation: the ability to reidentify users by stitching together our presence in multiple data sets, even if we are not directly identified in some of the sensitive data sets. I have discussed this issue in an article, where I include a good illustrative real-life example involving Netflix.

As for solutions, although I think there is space for improving HIPAA—a topic I have discussed along with the sharing of data with hospitals—I have not written specifically about AI privacy legislation in any great depth.

TRR: What are some emerging best practices for mitigating the negative effects of bias in the development and use of medical AI?

Cohen: I think the key starting point is to be able to identify biases. Missing data bias is a pretty obvious one to spot, though it is often hard to fix if you do not have resources to try to diversify the population represented in your data set. Even if you can diversify, some communities might be understandably wary of sharing information. But there are also many harder-to-spot biases.

For example, measurement or classification bias is where practitioner bias is translated into what is in the data set. What this may look like in practice is that women are less likely to receive lipid-lowering medications and procedures in the hospital compared to men, despite being more likely to present with hypertension and heart failure. Label bias is particularly easy to overlook, and it occurs when the outcome variable is differentially ascertained or has a different meaning across groups. A paper published in Science by Ziad Obermeyer and several coauthors has justifiably become the locus classicus example.

A lot of the problem is in thinking very hard at the front end about design and what is feasible given the data and expertise you have. But that is no substitute for auditing on the back end because even very well-intentioned designs may prove to lead to biased results on the back end. I often recommend a paper by Lama H. Nazer and several coauthors, published in PLOS Digital Health, to folks as a summary of the different kinds of bias.

All that said, I often finish talks by saying, “If you have listened carefully, you have learned that medical AI often makes errors, is bad at explaining how it is reaching its conclusion and is a little bit racist. The same is true of your physician, though. The real question is what combination of the two might improve on those dimensions we care about and how to evaluate it.”

TRR: You have written about the limited scope of the U.S. Food and Drug Administration (FDA) in regulating AI in the medical context. What health-related uses of AI currently fall outside of the FDA’s regulatory authority?

Cohen: Most is the short answer. I would recommend a paper written by my former post-doc and frequent coauthor, Sara Gerke, which does a nice job of walking through it. But the punchline is: if you are expecting medical AI to have been FDA-reviewed, your expectations are almost always going to be disappointed.

TRR: What risks, if any, are associated with the current gaps in FDA oversight of AI?

The FDA framework for drugs is aimed at showing safety and efficacy. With devices, the way that review is graded by device classes means that some devices skirt by because they can show a predicate device—in an AI context, sometimes quite unrelated—or they are classified as devices rather than general wellness products. Then there is the stuff that FDA never sees—most of it. For all these products, there are open questions about safety and efficacy. All that said, some would argue that the FDA premarket approval process is a bad fit for medical AI. These critics may defend FDA’s lack of review by comparing it to areas such as innovation in surgical techniques or medical practices, where FDA largely does not regulate the practice of medicine. Instead, we rely on licensure of physicians and tort law to do a lot of the work, as well as on in-house review processes. My own instinct as to when to be worried—to give a lawyerly answer—is it depends. Among other things, it depends on what non-FDA indicia of quality we have, what is understood by the relevant adopters about how the AI works, what populations it does or does not work for, what is tracked or audited, what the risk level in the worst-case scenario looks like, and who, if anyone, is doing the reviewing.

TRR: You have written in the past about medical liability for harms caused to patients by faulty AI. In the current technological and legal landscape, who should be liable for these injuries?

Cohen: Another lawyerly answer: it’s complicated, and the answer will be different for different kinds of AI. Physicians ultimately are responsible for a medical decision at the end of the day, and there is a school of thought that treats AI as just another tool, such as an MRI machine, and suggests that physicians are responsible even if the AI is faulty.

The reality is that few reported cases have succeeded against physicians for a myriad of reasons detailed in a paper published last year by Michelle M. Mello and Neel Guha. W. Nicholson Price II and I have focused on two other legs of the stool in the paper you asked about: hospital systems and developers. In general, and this may be more understandable given that in tort liability for hospital systems is not all that common, it seems to me that most policy analyses place too little emphasis on the hospital system as a potential locus of responsibility. We suggest “the application of enterprise liability to hospitals—making them broadly liable for negligent injuries occurring within the hospital system—with an important caveat: hospitals must have access to the information needed for adaptation and monitoring. If that information is unavailable, we suggest that liability should shift from hospitals to the developers keeping information secret.”

Elsewhere, I have also mused as to whether this is a good space for traditional tort law at all and whether instead we ought to have something more like the compensation schemes we see for vaccine injuries or workers’ compensation. In those schemes, we would have makers of AI pay into a fund that could pay for injuries without showing fault. Given the cost and complexity of proving negligence and causation in cases involving medical AI, this might be desirable.

TRR: The U.S. Senate rejected adding a provision to the recently passed “megalaw” that would have set a 10-year moratorium on any state enforcing a law or regulation affecting “artificial intelligence models,” “artificial intelligence systems,” or “automated decision systems.” What are some of the pros and cons of permitting states to develop their own AI regulations?

Cohen: This is something I have not written about, so I am shooting from the hip here. Please take it with an even larger grain of salt than what I have said already. The biggest con to state regulation is that it is much harder for an AI maker to develop something subject to differential standards or rules in different states. One can imagine the equivalent of impossibility-preemption type effects: state X says do this, state Y says do the opposite. But even short of that, it will be difficult to design a product to be used nationally if there are substantial variations in the standards of liability.

On the flip side, this is a feature of tort law and choice of law rules for all products, so why should AI be so different? And unlike physical goods that ship in interstate commerce, it is much easier to geolocate and either alter or disable AI running in states with different rules if you want to avoid liability.

On the pro side for state legislation, if you are skeptical that the federal government is going to be able to do anything in this space—or anything you like, at least—due to the usual pathologies of Congress, plus lobbying from AI firms, action by individual states might be attractive. States have innovated in the privacy space. The California Consumer Privacy Act is a good example. For state-based AI regulation, maybe there is a world where states fulfill the Brandeisian ideal of laboratories of experimentation that can be used to develop federal law.

Of course, a lot of this will depend on your prior beliefs about federalism. People often speak about the “Brussels Effect,” relating to the effects of the General Data Protection Regulation on non-European privacy practices. If a state the size of California was to pass legislation with very clear rules that differ from what companies do now, we might see a similar California effect with companies conforming nationwide to these standards. This is particularly true given that much of U.S. AI development is centered in California. One’s views about whether that is good or bad depend not only on the content of those rules but also on the views of what American federalism should look like.

TRR: Overall, what worries you most about the use of AI in the medical context? And what excites you the most?

Cohen: There is a lot that worries me, but the incentives are number one. What gets built is a function of what gets paid for. We may be giving up on some of what has the highest ethical value, the democratization of expertise and improving access, for lack of a business model that supports it. Government may be able to step in to some extent as a funder or for reimbursement, but I am not that optimistic.

Although your questions have led me to the worry side of the house, I am actually pretty excited. Much of what is done in medicine is unanalyzed, or at least not rigorously so. Even the very best clinicians have limited experience, and even if they read the leading journals, go to conferences, and complete other standard means of continuing education for physicians, the amount of information they can synthesize is orders of magnitude smaller than that of AI. AI may also allow scaling of the delivery of some services in a way that can serve underrepresented people in places where providers are scarce.



Source link

Continue Reading

Trending