Timeframe

SR1 is focused on articles published between 2021 (n = 68), 2022 (n = 61), and 2023 (n = 34) (Fig. 3). The discrepancy between 2023 and the other years is due to data for 2021 and 2022 comprising the full year while that for 2023 contains only articles published until the searches were conducted (August 2023). For SR2, we found articles as early as 2018 (n = 2), while most scoping reviews were published in 2022 (n = 7) and 2023 (n = 5). The timeframes covered by the scoping and systematic literature reviews in SR2 also show a wide variety in duration and date ranges, with the earliest search commencing in 1946 and the most recent concluding in 2023 (Fig. 4). The number of articles involved also returned significantly different volumes of papers for each scoping review, ranging from 8 as the smallest data set to 263 papers in the largest. This is because each scoping review included in SR2 addressed a different research question and mobilised different keywords and search parameters. The full list of date range, research questions, and volume of results for scoping reviews returned in SR2 are presented in Appendix 5.Please note that the full list of included papers in SR1 is not included in this paper, considering the high number of included articles and standards of reporting for such extensive findings. However, the list is available upon request.

Included timeframe and number of articles identified in reviews found by SR2. Ref. – Reference of the review article, No. of articles included – Number of articles included on the reviews, Blue line – the review had a general scope in terms of field of medicine, Orange line – the review had a specific field of medicine as a scope, Arrow – the review had no minimum date set in their search sting

Methodologies of included papers

The majority of publications returned in SR1 were non-empirical studies (n = 142) while empirical studies were a minority (n = 23). This makes sense given that ethical consideration of novel, emerging technologies typically begins with philosophical analysis and only later progress to empirical studies as real world examples start to emerge which can then be studied. For SR2 all papers were necessarily systematic (scoping) reviews as this was a key inclusion criteria of our search. Of the empirical studies reported in SR1 a variety of common methodologies were employed, with the most frequently reported methodology being qualitative, semi-structured research interviews (see Fig. 5).

Proportion of methods reported by empirical studies in SR1

Of the empirical studies reported in SR1, the majority were conducted in a single country (n = 15), some were conducted in multiple countries (n = 6) and two reported no data on the location of the study (n = 2). The most commonly reported geographic region for empirical studies was Europe only (n = 9), then North America only (n = 3), with additional papers reporting on studies from both European and North American countries in the form of a comparative analysis (n = 3). Two studies were conducted in Asian and African countries only respectively (n = 2), and a single study reported from Australia (n = 1).(Fig. 6).

Map showing frequency of countries included in empirical studies

Audiences and subjects of study

The empirical studies in SR1 reported a variety of different subject groups enrolled in the study (see Fig. 7). Mixed groups of different types of experts (and or lay representatives) were most commonly used, followed by studies with healthcare professionals (n = 5) and patients (n = 4). This makes sense given the interdisciplinary nature of AI and the healthcare setting.

Groups or populations identified as subjects of empirical research in SR1

Where information was reported, a majority of empirical studies used hypothetical scenarios describing potential uses of AI in healthcare to elicit responses (n = 5) rather than compiling experience based recommendations, suggesting that a good deal of AI ethics discussion remains anticipatory in nature. Nonetheless, overall most studies did not provide this information (n = 12).

Audiences to which articles were directed also varied significantly. Articles were explicitly directed to patients in 23 cases; to healthcare professionals (including nurses and physicians) in 32 cases; to developers in 20 cases; to regulatory authorities or regulatory audiences in 14 cases; in 85 cases it was not specified (see Fig. 8).

Proportion of articles directed to particular audiences in SR1

Medical domain and type of application of AI

Among the papers returned in SR1, the majority either did not specify a specific area of medicine or were considered to cover medicine in general (n = 78). Where a field or type of medicine was specified, the most commonly addressed area was mental health, broadly defined to include applications in psychology and psychiatry, dementia research et cetera (n = 13), followed by a range of public health applications including covid-19 and HIV surveillance (n = 8). There was a long ‘tail’ of medical domains that only received one or two papers, as the use of AI in niche or specialist areas of medicine typically attracted limited attention, with articles usually published in journals dedicated to those particular medical specialisms. The full range of medical domains reported, and their respective rates of occurrence in SR1, are displayed in Table 1.

Areas of medicine considered in the scoping review papers returned in SR2 were medicine in general (n = 12), public health (n = 1), dentistry (n = 1), primary care (n = 1), pharmacovigilance (n = 1), mental health (n = 2), radiology (n = 1), clinical research informatics (n = 1). In both SR1 and SR2, the majority of studies refer to AI broadly conceived (as to include most common types of algorithmic or Big Data technologies) or without specifying the type of AI (SR1 n = 70; SR2 n = 18). In SR1, where an application type was specified, by far the most common type was diagnostic and treatment applications of AI (n = 60), with only very small numbers of studies falling into the categories of databases (n = 3), conversational tools and chatbots (n = 3), patient engagement and adherence tools (n = 2) and ‘other’ (n = 10). The latter category included types of application such as LLMs, ‘digital twin’s et cetera In SR 2, two studies referred explicitly to Natural Language Programming as the type of AI addressed in the review while the others did not specify a type of application.

Ethical and societal themes addressed in SR1 and SR2

In both SR1 and SR2 a wide range of different ethical and societal issues were reported. SR1 utilised our a priori list of issues to classify the contents of each paper (see Appendix 3). Articles were coded to allow for multiple themes to be captured for each paper, reflecting the fact that some papers focused explicitly on one dimension such as ‘trust in AI systems’ while many others covered multiple issues pertaining to the ethical application of AI in healthcare. As a result, there are more codes reported for each arm of the scoping review than the total number of papers in either of SR1 and SR2. In SR1 the largest single category was the ‘other’ category for novel, unexpected or miscellaneous themes, where these are ‘novel’ or ‘unexpected’ with respect to widespread general AI ethics principles mentioned in the literature [33], as we show in the Discussion. The named issue categories that attracted the most frequent mention among papers in SR1 were ‘fairness/bias’, ‘privacy and data protection’ and ‘trust’, as evidenced by Fig. 9.

A similar pattern was seen in SR2. As noted, SR2 did not use predefined categories, but reported the major ethical and societal issues reported by each scoping review using that article’s own terminology. However, since the use of terminology was not consistent across papers, we have aggregated similar terms such as ‘privacy’ and ‘confidentiality’ or ‘justice’, ‘social justice’ and ‘fairness’ into composite categories as displayed in Fig. 9. The three most frequently reported categories in SR2 were ‘privacy and confidentiality’, ‘bias’ and ‘trust/trustworthiness’ as shown in Fig. 10. In both SR1 and SR2 the next most frequently encountered category was ‘autonomy’, while ‘consent’ ‘justice’ and ‘transparency’ were also popular themes in both strands of the review. SR2 has a longer ‘tail’ of issues receiving only one or two mentions. Many of these issues would have been compiled into the ‘other’ category in SR1 which goes some way to explaining its high frequency. Notably, our a priori categories ‘research ethics’ ‘epistemic injustice’ and ‘responsible innovation’ did not occur in SR2, suggesting that these terms, and the concepts they represent, have not been used widely in the history of debates on the ethics of AI in healthcare.

Ethical themes identified in SR2

A total of 114 entries were recorded in the ‘other’ category in the broad review (SR1). These covered a wide variety of terminology and phrasing and it was decided that this data needed further clarification to be of use. The full list of ‘other’ values reported in SR1 was reviewed by all three authors and each entry was either assigned as being a subclass of, or adding a particular nuance to, one of the existing categories of values reported in SR1 (as described in Sect. “Data charting” above) or was recognised as belonging to a novel category not deployed in our list of a priori ethical and social issues used to chart data in SR1. The new categories derived from reanalysing the ‘other’ column were numerous. One example is the deskilling of medical professionals [45, 51, 55], which can potentially harm patients in the long run, but it is not captured adequately by a vague concept like ‘maleficence’. Another example of a novel category is ‘human-centredness’, which captures the impact of medical AI tools on human relations [45]. The need for this new category has emerged in discussions on ‘care’, which plays a central role in a few articles (see, e.g., [55]), while as a notion it is typically neglected in the field of AI ethics. Ultimately ‘care’ was coded as a specific dimension of ‘human centredness’ in our analysis, because concerns over the quality of care are also related to how AI tools are going to influence care practices, where the ‘human’ dimension is central. A growing concern which is not covered in classic discussion of AI ethics is animal suffering—in one article [53] AI is also seen as contributing to what we termed ‘animal justice’ due to its potential to reduce animal suffering by replacing experimental animals with digital models. Environmental and economic sustainability of AI is another important challenge raised by AI tools that has not traditionally fallen within the scope of ‘classical’ discussions of AI ethics, although it is now beginning to garner ethical attention [43, 57], and again warranted a de novo category as part of the coding of the ‘Other’ category of ethical and social issues. Another novel theme identified was a possible concern over the high cost of AI and how this could limit its use. This is an interesting issue, because typical discussions in AI ethics are about deliberately limiting the use of AI. This was also connected to issues about the digital divide which, quite strangely, are neglected in AI ethics (probably for the emphasis on limiting the use of AI, rather than promoting it). These elements were often captured in another de novo category Distributive Justice (see Table s8 in Appendix 6). Although properly recognized as a subset of justice concerns, we decided to create a specific separate code for Distributive Justice to capture a growing number of discussions about global access to AI and the global distribution of benefits and harms of AI, especially with respect to divides between the the global north and global south, as well as between wealthy and poor populations within countries and regions [60]. The definitions of these new ethical and social themes are presented in Table s9 (Appendix 6). The meanings of these emergent categories and the nuances that the entries in the ‘other’ category added to discussion of more common principles of AI ethics such as privacy and bias are shown in Table s9, Appendix 6.

While we have not provided a similar analysis for SR2, there are significant hints that in existing scoping reviews, the typical way to understand AI ethics principles might have important limits, and that hyperspecialization (as we further discuss later) is a growing trend. For instance, [13], which has a short span (2017–2021), and it is on a narrow topic (i.e., primary care), address specific questions about the doctor-patient relationship. The nuances (e.g., participation, dehumanisation, agency of self-care) of this topic may not be easily covered by the general principles. Another aspect suggesting the increasing hyper-contextualisation of ethical and societal issues is the lack of homogeneity in reporting gaps. Depending on the context and research questions, different gaps were reported, where these gaps can hardly be reducible to a general and high-level discussion of AI ethics. For instance, in the context of dentistry [16], there is a lack of discussion on issues raised by 3D imaging technologies, which are especially salient in dentistry, but they may not in other fields (e.g. mental health). Finally, where reviews analysed in SR2 reported trends that are in principle categorizable in terms of the general principles of AI ethics, they nonetheless add nuances that point to the limitations of using those principles for analysing ethical and societal issues in this variegated context. For instance, Li et al. [41] stress the importance of data privacy. However, privacy in that work is mostly seen from the angles of data ownership, stigmatisation, dignity, and well-being. These are rich concepts that are hardly reducible to blanket characterizations referring to, e.g., respect for autonomy or non-maleficence. This is to suggest that a discussion of data privacy that only mentions respect for autonomy or non-maleficence might miss important challenges and nuances that a discussion based on ownership, stigmatisation, dignity and well-being would not.

An important aspect of the literature of both SR1 and SR2 is the type of tradeoffs among societal and ethical values. By ‘tradeoff’ here we mean that articles have emphasised that the use of AI in medicine may promote certain values at the expense of others. A typical tradeoff often commented in the literature is that, if one wants to have more accurate AI tools (where ‘accuracy’ is a value, in the sense of a desideratum of AI tools), then this often comes at the expense of interpretability/explainability. In other words, you cannot increase accuracy and interpretability/explainability at the same time. In SR1, accuracy vs explainability is widespread as noticed in [42]. Wehrens et al. [67] emphasise the importance of finding a balance between values that are perceived as typically in tradeoff, such as economic vs public values, data protection vs patient protection. Viberg et al. [66] and Kotsensas et al. [38] identify a tension between openness and privacy. Other examples of tradeoffs include fairness vs accuracy [37], regulation vs speed of innovation [40]. Another interesting tradeoff is between individual benefits vs group benefits. For instance, Stake et al. [61] notice that collecting sensitive data on a healthy child in an app might pose some privacy risks to the individual child, but at the same time the information may benefit certain groups. For instance, they mention that “children often continue to receive medications ‘off-label’ and the dosage is often based on the dosage for adults, as reliable data for children is lacking” (p 6). By collecting more data, these problems may be addressed more effectively for children collectively (i.e. as a group) even if this increases the identifiability and thereby reduces the privacy of individual paediatric patients. A number of scoping reviews analysed in SR2 also emphasised the existence of value tradeoffs. For instance, Goirand et al. [24] notice that beneficence can be compromised by fostering autonomy, and that in virtual bots for elderly care, trust may be compromised to ensure safety. They also point out that different dimensions of fairness are typically in contrast, as also widely known since the formulation of impossibility theorems for fair-ML [36]. Another tradeoff mentioned here is the classic one of accuracy vs transparency [25]. Li et al. [41] comment on possible conflicts between different objectives of local healthcare facilities and the high-level policies dictated by existing hard and soft regulations, or between patients’ and medical professional and insurance companies. Bear Don’t Walk et al. [4] notice that lack of stakeholder engagements can lead to conflicts in the ethical and societal values to embed in AI tools.

Trends identified in SR2

A majority of papers in SR2 (n = 12, 80%) reported some trend or pattern in the literature they reviewed. The remaining 40% of papers (n = 8) did not present any identifiable discussion of a trend in findings. Of those papers where discussion of a trend was discernible, the most common type of trend was ‘issue based’. This means that what we consider a ‘trend’ in these cases was a reported prevalence of a particular ethical and/or societal issue in the literature covered by that specific scoping review. This reflects the fact that most, though not all, scoping reviews in SR2 sought to identify key ethical issues in particular areas, whether subdomains of medicine or associated with particular types of application. ‘Privacy’ was most often reported as the most frequently discussed issue or the issue which attracted the greatest amount of discussion, with ‘bias’ a close second. These trends are equivalents of the type of findings reported for this study in Sect. “Ethical and societal themes addressed in SR1 and SR2“. However they do not match exactly our ranking of the prevalence of issues found in either SR1 or SR2, partly because each scoping review asked a different research question and because, as reported in Table 4, there was significant variation in the time periods covered by each review and the volume of literature considered.

Other studies identified the issues the authors considered most important (which was not always based on prevalence), for example Cartolovni et al. [11] state “the most critical social issue identified by our literature review is the impact of AI on the patient-physician relationship”. Some scoping reviews in SR2 also noted which issues were least discussed such as social justice [1], explicability of AI outputs [6] and Ienca et al. [31] who observed that “issues of data ownership, group-level ethical harms, and the distinction between academic and commercial uses of big data, do not appear as ethical priorities”. Maurud, Henni and Moen [44] examined the discourse on equity specifically, finding that the scope of discussion of health equity in clinical informatics had expanded from an early focus on race and ethnicity to encompass “diagnosis bias in rural populations, age, and gender or sex-specific bias”.

The remaining papers in SR2 in which any form of trend was discernible focused on ‘outcome’ based trends. These occur where the research question addressed by the scoping review set out to identify recommendations for best practice (e.g. for use of social media health data as investigated by Ford et al. [22]) or most frequently discussed challenges and benefits to e.g. implementating research ethics review of AI in healthcare [24, 31]. These trends pointed to findings such as the need for researchers to determine whether, and under which circumstances, social media users should count as human subjects for research ethics oversight [22], the risk of AI in primary care exacerbating health inequalities [13] or the major types of challenge facing ethical frameworks for AI [24].

Gaps identified in SR2

An important aspect of scoping reviews is their perception of things that need to be discussed, but that they could not find in the articles they analyse. Thirteen articles discussed gaps in the literature.

The gaps identified are highly specific to the context of the different scoping reviews, and for this reason they are sometimes surprising. For instance, Al-Hwsali et al. [1] notice that in AI-based public health there is no accountability framework. Rather than a gap in the literature, they seem to refer to a lack of effort to create such a framework. This is surprising because there are plenty of discussions on accountability in AI, but apparently not in the specific context of public health at the time of that review. Another interesting result comes from Benzinger et al. [6]. They point out that in AI-based clinical decision-making there are not many discussions on justice, and they think that this depends on the context dependence of justice, which makes it impossible to have an overarching concept of justice in place. In the context of radiology, Goisauf et al. [25] are concerned with the lack of attention on bias, and especially on gender bias. While issues of informed consent are typically at the forefront of discussions in biomedical ethics and AI ethics, Goording et al. [26] notice that when AI is used in the context of mental health, this is not as prominent as it should be, potentially due to concerns over mental health patients’ capacity to consent (see e.g. [59]). In the context of dentistry, Favaretto et al. [16] notice a number of gaps, which are due to how recent ethical issues in AI dentistry are.

Other issues are less surprising, and seem to identify gaps in discussions on ethical and societal issues in medical AI more in general. For instance, Cartolovni et al. [11] are especially concerned with the lack of qualitative and quantitative studies of ethics of AI in the medical context. Ienca et al. [31] emphasise the importance of re-use of data, and data-subjects’ control of data. Istasy et al. [32] make a plea for more discussions on the importance of infrastructure and human resources in the analysis of ethical and societal issues raised by medical AI. Others notice that the lack of applicability of high-level principles to practices can impair the ethical dimension of AI tools, and so far this is not much discussed in the literature [62].