Welcome to this edition of the Montreal AI Ethics Institute’s newsletter. Published bi-weekly, The AI Ethics Brief is designed to keep you informed about the rapidly evolving world of AI ethics by providing summaries of key research, insightful reporting, and thoughtful commentary. Learn more about us at montrealethics.ai/about.

Dear MAIEI Community,

The past few months have been a time of profound loss and reflection for all of us at the Montreal AI Ethics Institute (MAIEI). On September 30, we lost our dear friend and founder, Abhishek Gupta. His vision, brilliance, and tireless dedication to fostering AI ethics and responsible AI shaped MAIEI into what it is today and has inspired countless individuals worldwide, including myself.

As co-founder of MAIEI, I’ve reflected deeply on the path forward, consulting with close friends, colleagues, advisors, and our small but dedicated team. In moments of grief, closing MAIEI felt like the most straightforward option, but I quickly realized that the work we’ve built and the global community we’ve fostered over the past seven years are far too important to let go.

Abhishek’s loss is immeasurable.

My goal is not to fill his shoes but to honour his legacy by stewarding MAIEI into its next chapter—building a financially sustainable organization, expanding our reach, and ensuring its long-term resilience.

Here’s how we plan to move forward:

• Growing Our Impact: Continuing The AI Ethics Brief bi-weekly, starting with this edition, to grow our subscriber base from 15,000 to 100,000 and beyond.

• Strengthening Our Community: Inviting past contributors and partners to reconnect with MAIEI, share updates on your work, and collaborate on new opportunities.

• Honouring Abhishek’s Legacy: Developing a memorial scholarship or seminar series at McGill University, his alma mater, to inspire future leaders in AI ethics.

• Ensuring Sustainability and Growth: Building a fully functioning institute with paid staff, researchers, and funded projects supported by direct donations, corporate sponsorships, and grant opportunities.

You can read my full reflections: Open Letter: Moving Forward Together – MAIEI’s Next Chapter.

Read the Open Letter

In addition, you can find my remarks from the recent Montreal Startup Community Awards 2024 here. At the event, we honoured Abhishek’s life and legacy. Many thanks to Ilias Benjelloun, Simran Kanda, and the Montréal NewTech and Startupfest teams for their kind invitation and support.

Abhishek’s parents, Mr. Ashok Gupta and Mrs. Asha Gupta, and his brother Abhijay were present to celebrate his remarkable contributions. A memorial for Abhishek is being planned in Montreal for January 2025, with details to follow.

Thank you for being part of this journey as we navigate this challenging transition and continue shaping the future of AI ethics together.

Renjie Butalid
Co-founder & Director
Montreal AI Ethics Institute

Watch the tribute video below, presented at the Montreal Startup Community Awards on November 28, 2024. (Read BetaKit’s coverage here). Featuring Abhishek’s own words, this video beautifully captures his vision and passion. Edited by George Popi, Khaos.

Video credits (YouTube): Green Software Foundation, Brookfield Institute for Innovation + Entrepreneurship, Peter Carr.

💖 Help us keep The AI Ethics Brief free and accessible for everyone. Consider becoming a paid subscriber on Substack for the price of a ☕ or make a one-time or recurring donation at montrealethics.ai/donate.

Your support sustains our mission of Democratizing AI Ethics Literacy, honours Abhishek Gupta’s legacy through initiatives like a planned memorial scholarship or seminar series at McGill University, his alma mater, and ensures we can continue serving our community.

For corporate partnerships or larger donations, please contact us at support@montrealethics.ai.

• A collection of principles for guiding and evaluating large language models

• Risk of AI in Healthcare: A Study Framework

• FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation (NeurIPS 2024)

• AI Missteps Could Unravel Global Peace and Security – IEEE Spectrum

• I’m a neurology ICU nurse. The creep of AI in our hospitals terrifies me – Coda

• AI as an Invasive Species – Digital Public

• Atlantic Council – AI Connect II webinar: Sustainable development and inclusive growth of AI featuring Abhishek Gupta.

• U.S. Department of Commerce & U.S. Department of State Launch the International Network of AI Safety Institutes at Inaugural Convening in San Francisco

The recent killing of Brian Thompson, CEO of UnitedHealthcare—the largest health insurance company in the US—has drawn fresh attention to the ethical challenges of AI deployment in healthcare, particularly in insurance claims processing.

STAT News, a 2024 Pulitzer Prize Finalist in Investigative Reporting, uncovered that UnitedHealth pressured employees to use nH Predict, an AI model by NaviHealth, to prematurely deny payments by predicting patient lengths of stay. The model often overrode doctors’ judgments, wrongfully denying critical health coverage to elderly patients. Unlike clinical AI tools, these algorithms lack proper oversight despite their growing influence in coverage decisions.

As STAT reports:

One of the biggest and most controversial companies behind these models, NaviHealth, is now owned by UnitedHealth Group… These tools are becoming increasingly influential in decisions about patient care and coverage. For all of AI’s power to crunch data, insurers with huge financial interests are leveraging it to make life-altering decisions with little independent oversight.

AI models used by physicians to detect diseases such as cancer, or suggest the most effective treatment, are evaluated by the Food and Drug Administration. But tools used by insurers in deciding whether those treatments should be paid for are not subjected to the same scrutiny, even though they also influence the care of the nation’s sickest patients.

The impact is stark:

This has resulted in patients being kicked out of rehabilitation programs and care facilities far too early, forcing them to drain their life savings to obtain needed care that should be covered under their government-funded Medicare Advantage Plan. (Source: Ars Technica)

Key ethical concerns:

• Transparency gaps: Insurers’ algorithms lack the oversight applied to clinical AI tools.

• Patient harm: Flawed predictions lead to life-altering decisions, undermining medically necessary care.

• Regulatory mismatch: While clinical AI faces FDA regulation, insurer algorithms influencing coverage remain unregulated.

This case raises urgent questions about accountability and governance in using AI to influence healthcare decisions.

Read the full series of investigations by STAT News here.

Did we miss anything?

Leave a comment

Here are the results from the previous edition for this segment:

The results from the poll in our previous edition revealed that 63% of respondents have observed Goodhart’s Law in action within their organizations. This principle, which states that “when a measure becomes a target, it ceases to be a good measure,” highlights the risks of over-relying on metrics. This is especially relevant in programs like Responsible AI (RAI), where metrics are introduced to track progress and align goals but can sometimes lead to system gaming.

A recent article in the MIT Technology Review titled “The way we measure progress in AI is terrible,” illustrates this challenge in the context of AI benchmarks:

Every time a new AI model is released, it’s typically touted as acing its performance against a series of benchmarks. OpenAI’s GPT-4o, for example, was launched in May with a compilation of results that showed its performance topping every other AI company’s latest model in several tests.

The problem is that these benchmarks are poorly designed, the results hard to replicate, and the metrics they use are frequently arbitrary, according to new research. That matters because AI models’ scores against these benchmarks will determine the level of scrutiny and regulation they receive.

This disconnect between benchmarks and real-world outcomes creates significant risks, as poorly designed metrics can drive organizations to optimize for superficial targets rather than meaningful progress.

Read the full article here

How is your organization approaching AI adoption amidst all the hype? Are you prioritizing impactful projects, experimenting broadly, or waiting for proven use cases? Or is aligning AI initiatives with your goals a challenge?

Share your thoughts with the MAIEI community:

Leave a comment

Should we communicate with the dead to assuage our grief? An Ubuntu perspective on using griefbots

MAIEI’s Director of Partnerships, Connor Wright, recently explored this question in a thought-provoking paper. Drawing on Ubuntu Ethics—a Southern African philosophy centered on community and relationships—Connor examines the emerging use of griefbots (digital representations of deceased loved ones). His work argues that griefbots can offer meaningful connections for the bereaved, providing moral and ethical frameworks for their use.

The paper also addresses important considerations, such as privacy, commercialization, and the potential risks of replacing human relationships. Ultimately, Connor concludes that an Ubuntu perspective offers a valuable toolkit for navigating this sensitive intersection of grief and technology.

To dive deeper, read the full paper here.

Oh Canada!

Canada’s upcoming G7 presidency in 2025 offers a unique opportunity for MAIEI to shape global policy discussions. Through the Think7 (T7) process, organized by the Centre for International Governance Innovation (CIGI), contributions are being invited for policy briefs on key areas such as transformative technologies, digitalization of the global economy, environmental sustainability, and global peace and security.

Drawing on insights from our community of over 15,000 subscribers, we aim to highlight the most critical issues in AI ethics today.

Which AI governance gaps should we bring to the G7’s attention?

We’d love to hear your thoughts! If your institution is interested in partnering with us on this initiative, please get in touch. Leave a comment using the button below or reach out directly here.

Leave a comment

A collection of principles for guiding and evaluating large language models

This paper addresses the challenges in assessing and guiding the behavior of large language models (LLMs). It proposes a set of core principles based on an extensive review of literature from a wide variety of disciplines, including explainability, AI system safety, human critical thinking, and ethics.

To dive deeper, read the full summary here.

Risk of AI in Healthcare: A Study Framework

AI and its applications have found their way into many industrial and day-to-day activities through advanced devices and consumer’s reliance on the technology. One such domain is the multibillion-dollar healthcare industry, which relies heavily on accurate diagnosis and precision-based treatment, ensuring that the patient is relieved from his illness in an efficient and timely manner. This paper explores the risks of AI in healthcare by meticulously exploring the current literature and developing a concise study framework to help industrial and academic researchers better understand the other side of AI.

To dive deeper, read the full summary here.

FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation (NeurIPS 2024)

This paper will be presented at NeurIPS 2024 which takes place in Vancouver, Canada from December 10-15, 2024. It explores advancements in fair text-to-image (T2I) diffusion models, contributing to the growing body of research on fairness in generative AI. The study introduces FairQueue, a novel framework designed to achieve high-quality and fair T2I generation. Current T2I methods, like Stable Diffusion, often suffer from bias and quality degradation when addressing fairness. FairQueue tackles these challenges using two key strategies—Prompt Queuing and Attention Amplification—enhancing image quality, semantic preservation, and competitive fairness.

To dive deeper, read the full summary here.

AI Missteps Could Unravel Global Peace and Security – IEEE Spectrum

1. What happened: Earlier this year, an article co-authored by Abhishek Gupta and a global team of experts was published in IEEE Spectrum. The piece highlights the potential risks AI technologies pose to international peace and security, emphasizing the role of AI practitioners in mitigating these risks throughout the lifecycle of AI development. It details the dual-use nature of AI, with applications ranging from benign innovations to tools for disinformation, cyberattacks, and even biological weapons production. The article also examines indirect implications, such as decisions about open-sourcing AI technologies, which could have significant geopolitical consequences.

2. Why it matters: As advancements in AI accelerate, their integration into civilian and military applications poses unprecedented challenges. The authors argue that AI companies, researchers, and developers must take responsibility for the societal and security impacts of their work. They emphasize that education and training in responsible AI practices are essential to equipping practitioners to identify and address risks. This message highlights a critical need within the AI ethics community, particularly as discussions about global regulation gain momentum. The article serves as a vital call to action for policymakers to address the gaps in governance frameworks for dual-use AI technologies.

3. Between the lines: This article stands as one of Abhishek Gupta’s final contributions to the field, reflecting his enduring commitment to responsible AI innovation. Its publication reinforces the importance of equipping AI practitioners with both the technical and ethical tools needed to navigate the complex societal impacts of their work. The authors suggest concrete steps, such as integrating courses on AI governance into academic curriculums and encouraging lifelong learning through professional development programs. As the global community increasingly confronts the challenges posed by AI, this work provides a foundational perspective for aligning innovation with ethical standards and international security goals.

To dive deeper, read the full article here.

I’m a neurology ICU nurse. The creep of AI in our hospitals terrifies me – Coda

1. What happened: A neurology ICU nurse shares her firsthand experience with the integration of AI systems in hospital settings, emphasizing the unsettling challenges they pose. From predictive algorithms to robotic companions, AI is increasingly shaping healthcare decisions, often at the expense of human intuition and patient-centered care. The article highlights instances where AI-driven tools have led to questionable outcomes, raising concerns about their over reliance and lack of accountability.

2. Why it matters: The article highlights the ethical and practical dilemmas of embedding AI in critical care. While AI promises to improve efficiency and outcomes, its deployment without adequate oversight risks compromising patient safety and eroding trust. The nurse’s perspective sheds light on the limitations of AI tools, particularly when they override medical judgment or fail to adapt to complex, real-world scenarios. These issues stress the importance of balancing technological advancement with human expertise in healthcare.

3. Between the lines: This narrative illustrates the broader implications of unchecked AI integration in mission-critical domains. It calls for a re-evaluation of how and where AI is applied, advocating for greater collaboration between technologists and medical professionals. The article serves as a reminder that while AI has transformative potential, its deployment must be cautious, ethical, and centred around the needs of patients and providers.

To dive deeper, read the full article here.

AI as an Invasive Species – Digital Public

1. What happened: This article explores the concept of artificial intelligence (AI) as an “invasive species” within the digital and societal ecosystems. Drawing parallels to invasive biological species, the piece critiques how AI technologies—originally developed to optimize human activities—often lead to unintended, widespread consequences that disrupt ecosystems. It highlights the fundamental flaws in the business models underpinning AI technologies, which incentivize scale and dependency over sustainable and ethical growth.

2. Why it matters: The article introduces a thought-provoking analogy that positions AI as a force reshaping human relationships and ecosystems, often to their detriment. By focusing on rapid adoption, commodification, and scalability, AI systems risk exacerbating inequality, environmental degradation, and societal harms. This framing is crucial for policymakers and technologists to reconsider the unchecked proliferation of AI and ensure it aligns with broader societal and ecological goals.

3. Between the lines: The author highlights the extractive and exploitative nature of AI deployment, likening its growth to historical trends in industrial and technological revolutions. It challenges the prevailing narratives of AI as a purely beneficial force and urges governments, technology providers, and investors to prioritize not just regulation but also restorative measures. By embracing this perspective, stakeholders can address the root causes of AI’s disruptions while working to preserve the ecosystems—both human and natural—it impacts.

To dive deeper, read the full article here.

How is “Ethical Debt” relevant to AI Ethics?

👇 Learn more about why it matters in AI Ethics via our Living Dictionary.

Explore the Living Dictionary!

Atlantic Council – AI Connect II webinar: Sustainable development and inclusive growth of AI featuring Abhishek Gupta

In the fourth webinar of AI Connect II, hosted by the US Department of State and the Atlantic Council’s GeoTech Center on June 5, 2024, a discussion featuring Abhishek Gupta focused on sustainable development and inclusive growth through artificial intelligence (AI).

To dive deeper, read more details here.

U.S. Department of Commerce & U.S. Department of State Launch the International Network of AI Safety Institutes at Inaugural Convening in San Francisco

Last month, the U.S. Departments of Commerce and State launched the International Network of AI Safety Institutes during a convening in San Francisco. The United States will serve as the inaugural chair of this initiative, whose initial members include Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States. This network aims to address AI safety and governance challenges through global collaboration, best practices, and multidisciplinary research, with over $11 million committed to advancing its goals. This launch coincides with Canada establishing its own Canadian Artificial Intelligence Safety Institute (CAISI), further emphasizing the growing global focus on ethical and secure AI development

For further details, read the full announcement here.

Contextualizing Artificially Intelligent Morality: A Meta-Ethnography of Top-Down, Bottom-Up, and Hybrid Models for Theoretical and Applied Ethics in Artificial Intelligence

Ethics in Artificial Intelligence (AI) can emerge in many ways. This paper addresses developmental methodologies, including top-down, bottom-up, and hybrid approaches to ethics in AI, from theoretical, technical, and political perspectives. Examples through case studies of the complexity of AI ethics are discussed to provide a global perspective when approaching this challenging and often overlooked area of research.

To dive deeper, read the full article here.

We’d love to hear from you, our readers, on what recent research papers caught your attention. We’re looking for ones that have been published in journals or as a part of conference proceedings.

Leave a comment