Ethics & Policy
Unmasking secret cyborgs, California SB 1047, LLM creativity, toxicity evaluation ++
Welcome to another edition of the Montreal AI Ethics Institute’s weekly AI Ethics Brief that will help you keep up with the fast-changing world of AI Ethics! Every week, we summarize the best of AI Ethics research and reporting, along with some commentary. More about us at montrealethics.ai/about.
💖 To keep our content free for everyone, we ask those who can, to support us: become a paying subscriber for the price of a couple of ☕.
If you’d prefer to make a one-time donation, visit our donation page. We use this Wikipedia-style tipping model to support our mission of Democratizing AI Ethics Literacy and to ensure we can continue to serve our community.
-
On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research
-
On the Creativity of Large Language Models
-
Supporting Human-LLM collaboration in Auditing LLMs with LLMs
-
AI Governance Appears on Corporate Radar
-
TikTok to automatically label AI-generated user content in global first
-
Creating sexually explicit deepfake images to be made offence in UK
Bill SB 1047 in California aims to establish safety standards for the development of advanced AI models while authorizing a regulatory body to enforce compliance. However, there is ongoing debate about whether the bill strikes the right balance between mitigating AI risks and enabling innovation.
In brief, the bill requires the following from AI ecosystem actors:
But, there have been some very vocal concerns that have been raised by (influential) people in the AI ecosystem on how this might stifle innovation, including emigration of companies to other more hospitable jurisdictions to develop AI systems. Prominent figures like Andrew Ng argue the bill stokes unnecessary fear and hinders AI innovation. Critics say the bill burdens smaller AI companies with compliance costs and targets hypothetical risks, impacting open-source models which have driven a tremendous amount of capability advances in recent months, such as those enabled by Llama 3.
It will be interesting to see how the bill evolves given its current state, the arguments raised by industry actors, the profiles of the co-sponsors who are supporting the bill, and ultimately the balance that we need to strike in crafting such rules so that there is an appropriate balance between the ability to innovate while safeguarding end-user interests. Swinging the pendulum too far on either end is dangerous!
Did we miss anything?
Every week, we’ll feature a question from the MAIEI community and share our thinking here. We invite you to ask yours, and we’ll answer it in the upcoming editions.
Here are the results from the previous edition for this segment:
A little bit sad to see that there is a bigger percentage of readers who haven’t had a chance to engage in futures thinking at their organization. Hopefully, the guide from last week, Think further into the future: An approach to better RAI programs, with the suggested actions of (1) establishing a foresight team, (2) developing long-term metrics, (3) conducting regular futures scenario workshops, and (4) building flexible policies provides you with a starting point to experiment with this approach.
This week, reader Kristian B., asks us about being appointed/assigned as the first person in their organization to implement (ambitiously) sweeping changes to operationalize Responsible AI. Yet, this comes with a warning that they need to be cautious as they make those changes – so, they ask us, how to achieve that balance? (And yes, it seems like balance is the topic-du-jour this week!)
We believe that the right approach is one that makes large changes in small, safe steps as we write in this week’s exploration of the subject:
The “large changes in small safe steps” approach leads to more successful program implementation by effectively mitigating risks, enhancing stakeholder engagement and trust, and ensuring sustainable and scalable adoption of new practices. This strategic method balances innovation with caution, fostering a resilient and adaptive framework for Responsible AI programs.
What were the lessons you learned from the deployment of Responsible AI at your organization? Please let us know! Share your thoughts with the MAIEI community:
Unmasking Secret Cyborgs and Illuminating Their AI Shadows
To address the challenges of “shadow AI” adoption and “secret cyborgs,” in organizations, policymakers and governance professionals should focus on creating frameworks that require transparency and accountability in AI usage.
To delve deeper, read the full article here.
Raging debates, like the ones around California SB 1047, and how they approach the balance between safety and speed of innovation pose crucial questions for the Responsible AI community on how we should support such legislative efforts in the most effective manner so that the outcomes are something that achieve that balancing act in the best possible manner. What mediation techniques have you found that work well for such a process?
We’d love to hear from you and share your thoughts with everyone in the next edition:
In some essence, continuing to build on the idea of having to evaluate difficult tradeoffs, such as the ones presented in California SB 1047 as we discuss this week, let’s take a look at how we can determine tradeoffs when it comes to safety, ethics, and inclusivity in AI systems.
Design decisions for AI systems involve value judgements and optimization choices. Some relate to technical considerations like latency and accuracy, others relate to business metrics. But each require careful consideration as they have consequences in the final outcome from the system.
To be clear, not everything has to translate into a tradeoff. There are often smart reformulations of a problem so that you can meet the needs of your users and customers while also satisfying internal business considerations.
Take for example an early LinkedIn feature that encouraged job postings by asking connections to recommend specific job postings to target users based on how appropriate they thought them to be for the target user. It provided the recommending user a sense of purpose and goodwill by only sharing relevant jobs to their connections at the same time helping LinkedIn provide more relevant recommendations to users. This was a win-win scenario compared to having to continuously probe a user deeper and deeper to get more data to provide them with more targeted job recommendations.
This article will build on The importance of goal setting in product development to achieve Responsible AI adding another dimension of consideration in building AI systems that are ethical, safe, and inclusive.
You can either click the “Leave a comment” button below or send us an email! We’ll feature the best response next week in this section.
On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research
We show how silent changes in a toxicity scoring API have impacted a fair comparison of toxicity metrics between language models over time. This affected research reproducibility and living benchmarks of model risk such as HELM. We suggest caution in applying apples-to-apples comparisons between toxicity studies and lay recommendations for a more structured approach to evaluating toxicity over time.
To delve deeper, read the full summary here.
On the Creativity of Large Language Models
Large Language Models (LLMs) like ChatGPT are revolutionizing several areas of AI, including those related to creative writing. This paper offers a critical discussion of LLMs in the light of human theories of creativity, including the impact these technologies might have on society.
To delve deeper, read the full summary here.
Supporting Human-LLM collaboration in Auditing LLMs with LLMs
While large language models (LLMs) are being increasingly deployed in sociotechnical systems, in practice, LLMs propagate social biases and behave irresponsibly, imploring the need for rigorous evaluations. Existing tools for finding failures of LLMs leverage either or both humans and LLMs, however, they fail to bring the human into the loop effectively, missing out on their expertise and skills complementary to those of LLMs. In this work, we build upon an auditing tool to support humans in steering the failure-finding process while leveraging the generative skill and efficiency of LLMs.
To delve deeper, read the full summary here.
AI Governance Appears on Corporate Radar
-
What happened: The rapid evolution of AI is reshaping business strategies, prompting companies to integrate AI for efficiency, competitive advantages, and stakeholder engagement. As AI usage surges, so do concerns about its risks, prompting the White House to issue an executive order on AI regulation. Reflecting this, companies are adapting by recruiting directors with AI expertise and establishing board-level oversight.
-
Why it matters: AI’s potential benefits come with significant risks, urging companies to adopt proactive measures for oversight. While only a fraction of S&P 500 companies explicitly disclose AI oversight, sectors like Information Technology lead in integrating AI expertise on boards, with ‘30% of S&P 500 IT companies having at least one director with AI-related expertise.’ This trend indicates a growing recognition of AI’s impact, especially in industries where it’s most influential. Investors are beginning to demand greater transparency regarding AI’s use and impact, signaling a shift towards increased accountability and governance in AI integration.
-
Between the lines: As AI becomes more central to business operations, investor expectations for transparent and responsible AI governance are mounting. The emergence of shareholder proposals focusing on AI underscores this shift, signaling a potential future where AI oversight becomes a standard requirement. While regulatory changes and investor policies may evolve in response to AI’s growing influence, companies are urged to establish robust oversight mechanisms to navigate AI-related risks and opportunities effectively.
TikTok to automatically label AI-generated user content in global first
-
What happened: TikTok is taking proactive steps to address concerns surrounding the proliferation of AI-generated content, particularly deepfakes, by automatically labeling such content on its platform. This move comes amid growing worries about the spread of disinformation facilitated by advances in generative AI. TikTok’s announcement follows existing requirements by online groups, including Meta, for users to disclose AI-generated media.
-
Why it matters: TikTok’s decision to label AI-generated content is a significant response to the rising prevalence of harmful content generated through AI. By providing transparency, TikTok aims to preserve the authenticity of its platform and empower users to distinguish between human-created and AI-generated content. Other major social media platforms are also grappling with integrating generative AI while combatting issues like spam and deepfakes, especially in the context of upcoming elections. TikTok’s move underscores the broader industry efforts to address these challenges and foster a more trustworthy online environment.
-
Between the lines: While tech companies are exploring ways to embed AI technology into their platforms, concerns persist about the potential misuse of open-source AI tools by bad actors to create undetectable deepfakes. Meta has also announced plans to label AI-generated content, joining TikTok in this initiative. Experts suggest that transparency and authentication tools like those developed by Adobe could be crucial in mitigating the risks associated with AI-generated content, marking an initial step in addressing this complex issue.
Creating sexually explicit deepfake images to be made offence in UK
-
What happened: The Ministry of Justice has announced plans to criminalize the creation of sexually explicit “deepfake” images, regardless of whether they are shared. This amendment to the criminal justice bill aims to address concerns regarding the use of deepfake technology to produce intimate images without consent. The legislation aligns with the Online Safety Act, which already prohibits the sharing of such content.
-
Why it matters: The proposed offence signifies a significant step in safeguarding individuals’ privacy and dignity in the digital age. Laura Farris, the minister for victims and safeguarding, emphasized the need to combat the dehumanizing and harmful nature of deepfake sexual images, particularly in their potential to cause catastrophic consequences when shared widely. Yvette Cooper, the shadow home secretary, underscored the importance of equipping law enforcement with the necessary tools to enforce these laws effectively, thereby preventing perpetrators from exploiting individuals with impunity.
-
Between the lines: Deborah Joseph, the editor-in-chief of Glamour UK, expressed support for the legislative amendment, citing a Glamour survey highlighting widespread concerns among readers about the safety implications of deepfake technology. Despite this progress, Joseph emphasized the ongoing challenges in ensuring women’s safety and called for continued efforts to combat this disturbing activity effectively.
What is hallucination in LLMs?
👇 Learn more about why it matters in AI Ethics via our Living Dictionary.
Bridging the civilian-military divide in responsible AI principles and practices
Advances in AI research have brought increasingly sophisticated capabilities to AI systems and heightened the societal consequences of their use. Researchers and industry professionals have responded by contemplating responsible principles and practices for AI system design. At the same time, defense institutions are contemplating ethical guidelines and requirements for the development and use of AI for warfare. However, varying ethical and procedural approaches to technological development, research emphasis on offensive uses of AI, and lack of appropriate venues for multistakeholder dialogue have led to differing operationalization of responsible AI principles and practices among civilian and defense entities. We argue that the disconnect between civilian and defense responsible development and use practices leads to underutilization of responsible AI research and hinders the implementation of responsible AI principles in both communities. We propose a research roadmap and recommendations for dialogue to increase exchange of responsible AI development and use practices for AI systems between civilian and defense communities. We argue that generating more opportunities for exchange will stimulate global progress in the implementation of responsible AI principles.
To delve deeper, read more details here.
A Systematic Review of Ethical Concerns with Voice Assistants
We’re increasingly becoming aware of ethical issues around the use of voice assistants, such as the privacy implications of having devices that are always listening and the ways that these devices are integrated into existing social structures in the home. This has created a burgeoning area of research across various fields, including computer science, social science, and psychology, which we mapped through a systematic literature review of 117 research articles. In addition to analysis of specific areas of concern, we also explored how different research methods are used and who gets to participate in research on voice assistants.
To delve deeper, read the full article here.
We’d love to hear from you, our readers, on what recent research papers caught your attention. We’re looking for ones that have been published in journals or as a part of conference proceedings.
Ethics & Policy
AI and ethics – what is originality? Maybe we’re just not that special when it comes to creativity?
I don’t trust AI, but I use it all the time.
Let’s face it, that’s a sentiment that many of us can buy into if we’re honest about it. It comes from Paul Mallaghan, Head of Creative Strategy at We Are Tilt, a creative transformation content and campaign agency whose clients include the likes of Diageo, KPMG and Barclays.
Taking part in a panel debate on AI ethics at the recent Evolve conference in Brighton, UK, he made another highly pertinent point when he said of people in general:
We know that we are quite susceptible to confident bullshitters. Basically, that is what Chat GPT [is] right now. There’s something reminds me of the illusory truth effect, where if you hear something a few times, or you say it here it said confidently, then you are much more likely to believe it, regardless of the source. I might refer to a certain President who uses that technique fairly regularly, but I think we’re so susceptible to that that we are quite vulnerable.
And, yes, it’s you he’s talking about:
I mean all of us, no matter how intelligent we think we are or how smart over the machines we think we are. When I think about trust, – and I’m coming at this very much from the perspective of someone who runs a creative agency – we’re not involved in building a Large Language Model (LLM); we’re involved in using it, understanding it, and thinking about what the implications if we get this wrong. What does it mean to be creative in the world of LLMs?
Genuine
Being genuine, is vital, he argues, and being human – where does Human Intelligence come into the picture, particularly in relation to creativity. His argument:
There’s a certain parasitic quality to what’s being created. We make films, we’re designers, we’re creators, we’re all those sort of things in the company that I run. We have had to just face the fact that we’re using tools that have hoovered up the work of others and then regenerate it and spit it out. There is an ethical dilemma that we face every day when we use those tools.
His firm has come to the conclusion that it has to be responsible for imposing its own guidelines here to some degree, because there’s not a lot happening elsewhere:
To some extent, we are always ahead of regulation, because the nature of being creative is that you’re always going to be experimenting and trying things, and you want to see what the next big thing is. It’s actually very exciting. So that’s all cool, but we’ve realized that if we want to try and do this ethically, we have to establish some of our own ground rules, even if they’re really basic. Like, let’s try and not prompt with the name of an illustrator that we know, because that’s stealing their intellectual property, or the labor of their creative brains.
I’m not a regulatory expert by any means, but I can say that a lot of the clients we work with, to be fair to them, are also trying to get ahead of where I think we are probably at government level, and they’re creating their own frameworks, their own trust frameworks, to try and address some of these things. Everyone is starting to ask questions, and you don’t want to be the person that’s accidentally created a system where everything is then suable because of what you’ve made or what you’ve generated.
Originality
That’s not necessarily an easy ask, of course. What, for example, do we mean by originality? Mallaghan suggests:
Anyone who’s ever tried to create anything knows you’re trying to break patterns. You’re trying to find or re-mix or mash up something that hasn’t happened before. To some extent, that is a good thing that really we’re talking about pattern matching tools. So generally speaking, it’s used in every part of the creative process now. Most agencies, certainly the big ones, certainly anyone that’s working on a lot of marketing stuff, they’re using it to try and drive efficiencies and get incredible margins. They’re going to be on the race to the bottom.
But originality is hard to quantify. I think that actually it doesn’t happen as much as people think anyway, that originality. When you look at ChatGPT or any of these tools, there’s a lot of interesting new tools that are out there that purport to help you in the quest to come up with ideas, and they can be useful. Quite often, we’ll use them to sift out the crappy ideas, because if ChatGPT or an AI tool can come up with it, it’s probably something that’s happened before, something you probably don’t want to use.
More Human Intelligence is needed, it seems:
What I think any creative needs to understand now is you’re going to have to be extremely interesting, and you’re going to have to push even more humanity into what you do, or you’re going to be easily replaced by these tools that probably shouldn’t be doing all the fun stuff that we want to do. [In terms of ethical questions] there’s a bunch, including the copyright thing, but there’s partly just [questions] around purpose and fun. Like, why do we even do this stuff? Why do we do it? There’s a whole industry that exists for people with wonderful brains, and there’s lots of different types of industries [where you] see different types of brains. But why are we trying to do away with something that allows people to get up in the morning and have a reason to live? That is a big question.
My second ethical thing is, what do we do with the next generation who don’t learn craft and quality, and they don’t go through the same hurdles? They may find ways to use {AI] in ways that we can’t imagine, because that’s what young people do, and I have faith in that. But I also think, how are you going to learn the language that helps you interface with, say, a video model, and know what a camera does, and how to ask for the right things, how to tell a story, and what’s right? All that is an ethical issue, like we might be taking that away from an entire generation.
And there’s one last ‘tough love’ question to be posed:
What if we’re not special? Basically, what if all the patterns that are part of us aren’t that special? The only reason I bring that up is that I think that in every career, you associate your identity with what you do. Maybe we shouldn’t, maybe that’s a bad thing, but I know that creatives really associate with what they do. Their identity is tied up in what it is that they actually do, whether they’re an illustrator or whatever. It is a proper existential crisis to look at it and go, ‘Oh, the thing that I thought was special can be regurgitated pretty easily’…It’s a terrifying thing to stare into the Gorgon and look back at it and think,’Where are we going with this?’. By the way, I do think we’re special, but maybe we’re not as special as we think we are. A lot of these patterns can be matched.
My take
This was a candid worldview that raised a number of tough questions – and questions are often so much more interesting than answers, aren’t they? The subject of creativity and copyright has been handled at length on diginomica by Chris Middleton and I think Mallaghan’s comments pretty much chime with most of that.
I was particularly taken by the point about the impact on the younger generation of having at their fingertips AI tools that can ‘do everything, until they can’t’. I recall being horrified a good few years ago when doing a shift in a newsroom of a major tech title and noticing that the flow of copy had suddenly dried up. ‘Where are the stories?’, I shouted. Back came the reply, ‘Oh, the Internet’s gone down’. ‘Then pick up the phone and call people, find some stories,’ I snapped. A sad, baffled young face looked back at me and asked, ‘Who should we call?’. Now apart from suddenly feeling about 103, I was shaken by the fact that as soon as the umbilical cord of the Internet was cut, everyone was rendered helpless.
Take that idea and multiply it a billion-fold when it comes to AI dependency and the future looks scary. Human Intelligence matters
UNESCO, in collaboration with the Ministry of Transport and Communications, Catalpa International and national lead consultant, jointly conducted consultative and validation workshops as part of the AI Readiness assessment implementation in Timor-Leste. Held on 8–9 April and 27 May respectively, the workshops convened representatives from government ministries, academia, international organisations and development partners, the Timor-Leste National Commission for UNESCO, civil society, and the private sector for a multi-stakeholder consultation to unpack the current stage of AI adoption and development in the country, guided by UNESCO’s AI Readiness Assessment Methodology (RAM).
In response to growing concerns about the rapid rise of AI, the UNESCO Recommendation on the Ethics of Artificial Intelligence was adopted by 194 Member States in 2021, including Timor-Leste, to ensure ethical governance of AI. To support Member States in implementing this Recommendation, the RAM was developed by UNESCO’s AI experts without borders. It includes a range of quantitative and qualitative questions designed to gather information across different dimensions of a country’s AI ecosystem, including legal and regulatory, social and cultural, economic, scientific and educational, technological and infrastructural aspects.
By compiling comprehensive insights into these areas, the final RAM report helps identify institutional and regulatory gaps, which can assist the government with the necessary AI governance and enable UNESCO to provide tailored support that promotes an ethical AI ecosystem aligned with the Recommendation.
The first day of the workshop was opened by Timor-Leste’s Minister of Transport and Communication, H.E. Miguel Marques Gonçalves Manetelu. In his opening remarks, Minister Manetelu highlighted the pivotal role of AI in shaping the future. He emphasised that the current global trajectory is not only driving the digitalisation of work but also enabling more effective and productive outcomes.
Publishing stands at a pivotal juncture, said Jeremy North, president of Global Book Business at Taylor & Francis Group, addressing delegates at the 3rd International Conference on Publishing Education in Beijing. Digital intelligence is fundamentally transforming the sector — and this revolution will inevitably create “AI winners and losers”.
True winners, he argued, will be those who embrace AI not as a replacement for human insight but as a tool that strengthens publishing’s core mission: connecting people through knowledge. The key is balance, North said, using AI to enhance creativity without diminishing human judgment or critical thinking.
This vision set the tone for the event where the Association for International Publishing Education was officially launched — the world’s first global alliance dedicated to advancing publishing education through international collaboration.
Unveiled at the conference cohosted by the Beijing Institute of Graphic Communication and the Publishers Association of China, the AIPE brings together nearly 50 member organizations with a mission to foster joint research, training, and innovation in publishing education.
Tian Zhongli, president of BIGC, stressed the need to anchor publishing education in ethics and humanistic values and reaffirmed BIGC’s commitment to building a global talent platform through AIPE.
BIGC will deepen academic-industry collaboration through AIPE to provide a premium platform for nurturing high-level, holistic, and internationally competent publishing talent, he added.
Zhang Xin, secretary of the CPC Committee at BIGC, emphasized that AIPE is expected to help globalize Chinese publishing scholarships, contribute new ideas to the industry, and cultivate a new generation of publishing professionals for the digital era.
Themed “Mutual Learning and Cooperation: New Ecology of International Publishing Education in the Digital Intelligence Era”, the conference also tackled a wide range of challenges and opportunities brought on by AI — from ethical concerns and content ownership to protecting human creativity and rethinking publishing values in higher education.
Wu Shulin, president of the Publishers Association of China, cautioned that while AI brings major opportunities, “we must not overlook the ethical and security problems it introduces”.
Catriona Stevenson, deputy CEO of the UK Publishers Association, echoed this sentiment. She highlighted how British publishers are adopting AI to amplify human creativity and productivity, while calling for global cooperation to protect intellectual property and combat AI tool infringement.
The conference aims to explore innovative pathways for the publishing industry and education reform, discuss emerging technological trends, advance higher education philosophies and talent development models, promote global academic exchange and collaboration, and empower knowledge production and dissemination through publishing education in the digital intelligence era.
yangyangs@chinadaily.com.cn
-
Funding & Business7 days agoKayak and Expedia race to build AI travel agents that turn social posts into itineraries
-
Jobs & Careers7 days ago
Mumbai-based Perplexity Alternative Has 60k+ Users Without Funding
-
Mergers & Acquisitions7 days ago
Donald Trump suggests US government review subsidies to Elon Musk’s companies
-
Funding & Business6 days ago
Rethinking Venture Capital’s Talent Pipeline
-
Jobs & Careers6 days ago
Why Agentic AI Isn’t Pure Hype (And What Skeptics Aren’t Seeing Yet)
-
Funding & Business4 days ago
Sakana AI’s TreeQuest: Deploy multi-model teams that outperform individual LLMs by 30%
-
Jobs & Careers6 days ago
Astrophel Aerospace Raises ₹6.84 Crore to Build Reusable Launch Vehicle
-
Funding & Business1 week agoFrom chatbots to collaborators: How AI agents are reshaping enterprise work
-
Tools & Platforms6 days ago
Winning with AI – A Playbook for Pest Control Business Leaders to Drive Growth
-
Jobs & Careers6 days ago
Telangana Launches TGDeX—India’s First State‑Led AI Public Infrastructure
