“We need to ask ourselves: is AI actually worth the costs?” – SWZ
Immer informiert bleiben
Den SWZ-Newsletter abonnieren
Bozen – Brent Mittelstadt is a professor of data ethics and policy at the Oxford Internet Institute (OII) at the University of Oxford. As a philosopher of technology, he specializes in artificial intelligence ethics. His work critically examines algorithmic decision-making, fairness, transparency, and accountability, influencing AI governance and policy. By integrating multidisciplinary perspecti…
Jetzt Abo abschließen oder einloggen, um den ganzen Artikel zu lesen.
Flexi-Abo
Online | Unbegrenzter Zugang zu allen Inhalten auf www.swz.it
App | Unbegrenzter Zugang in der Mobile und Tablet App (iOS und Android)
ePaper | Die digitale 1:1-Ausgabe der Printversion (schon am Donnerstag ab 20 Uhr)
Diese Website verwendet Cookies während Sie durch die Website navigieren, um Ihre Erfahrungen zu verbessern. Für die Funktionalität der Website verwenden wir technische Cookies und für anonyme Statistiken Marketing Cookies. Durch das Klicken auf Akzeptieren, stimmen sie automatisch den Marketing Cookies zu, außer sie haben diese deaktiviert.
The world today is war-torn, starting with Russia’s attacks on Ukraine to Israel’s devastation in Palestine and now in Iran, putting the entire West Asia in jeopardy.
The geometrics of war has completely changed, from Blitzkrieg (lightning war) in World War II to the use of sophisticated and technologically driven missiles in these latest armed conflicts. The most recent wars are being driven by use of artificial intelligence (AI) to narrow down potential targets.
There have been multiple evidences which indicate that Israeli forces have deployed novel AI-driven targeting tools in Gaza. One system, nicknamed “Lavender” is an AI-enabled database that assigns risk scores to Gazans based on patterns in their personal data (communication, social connections) to identify “suspected Hamas or Islamic Jihad operatives”. Lavender has flagged up to 37,000 Palestinians as potential targets early in the war.
A second system, “Where is Daddy?”, uses mobile phone location tracking to notify operators when a marked individual is at home. The initial strikes using these automated generated systems targeted individuals in their private homes on the pretext of targeting the terrorists. But innocent women and young children also lost their lives in these attacks. This technology was developed as a replacement of human acumen and strategy to identify and target the suspects.
According to the Humans Rights Watch report (2024), around 70 per cent of people who have lost lives were women and children. The United Nations agency has also verified the details of 8,119 victims killed in Gaza from November 2023 to April 2024. The report showed that 44 per cent of the victims were children and 26 per cent were women. The humans are merely at the mercy of this sophisticated technology that identified the suspected militants and targeted them.
The use of AI-based tools like “Lavender” and “Where’s Daddy?” by Israel in its war against Palestine raises serious questions about the commitment of countries to the international legal framework and the ethics of war. Use of such sophisticated AI targeted tools puts the weaker nations at the dictate of the powerful nations who can use these technologies to inflict suffering for the non-combatants.
The international humanitarian law (IHL) and international human rights law (IHRL) play a critical yet complex role in the context of AI during conflict situations such as the Israel-Palestine Conflict. Such AI-based warfare violates the international legal framework principles of distinction, proportionality and precaution.
The AI systems do not inherently know who is a combatant. Investigations report that Lavender had an error rate on the order of 10 per cent and routinely flagged non-combatants (police, aid workers, people who merely shared a name with militants). The reported practice of pre-authorising dozens of civilian deaths per strike grossly violates the proportionality rule.
An attack is illegal if incidental civilian loss is “excessive” in relation to military gain. For example, one source noted that each kill-list target came with an allowed “collateral damage degree” (often 15–20) regardless of the specific context. Allowing such broad civilian loss per target contradicts IHL’s core balancing test (ICRC Rule 14).
The AI-driven process has eliminated normal safeguards (verification, warnings, retargeting). IHRL continues to apply alongside IHL in armed conflict contexts. In particular, the right to life (ICCPR Article 6) obliges states to prevent arbitrary killing.
The International Court of Justice has held that while the right to life remains in force during war, an “arbitrary deprivation of life” must be assessed by reference to the laws of war. In practice, this means that IHL’s rules become the benchmark for whether killings are lawful.
However, even accepting lex specialis (law overriding general law), the reported AI strikes raise grave human rights concerns especially the Right to Life (ICCPR Art. 6) and Right to Privacy (ICCPR Art. 17).
Ethics of war, called ‘jus in bello’ in the legal parlance, based on the principles of proportionality (anticipated moral cost of war) and differentiation (between combatants and non-combatants) has also been violated. Article 51(5) of Additional Protocol I of the 1977 Geneva Convention said that “an attack is disproportionate, and thus indiscriminate, if it may be expected to cause incidental loss of civilian life, injury to civilians, damage to civilian objects, or a combination thereof, which would be excessive in relation to the concrete and military advantage”.
The Israel Defense Forces have been indiscriminately using AI to target potential targets. These targets though aimed at targeting militants have been extended to the non-military targets also, thus causing casualties to the civilians and non-combatants. Methods used in a war is like a trigger which once warded off is extremely difficult to retract and reconcile. Such unethical action creates more fault lines and any alternate attempt at peace resolution and mediation becomes extremely difficult.
The documented features of systems like Lavender and Where’s Daddy, based on automated kill lists, minimal human oversight, fixed civilian casualty “quotas” and use of imprecise munitions against suspects in homes — appear to contravene the legal and ethical principles.
Unless rigorously constrained, such tools risk turning warfare into arbitrary slaughter of civilians, undermining the core humanitarian goals of IHL and ethics of war. Therefore, it is extremely important to streamline the unregulated use of AI in perpetuating war crimes as it undermines the legal and ethical considerations of humanity at large.
The AI landscape is evolving rapidly, and with the rise of agentic AI, trust has never been more critical. As businesses continue to integrate AI into their operations and customer experiences, leaders must ensure that these technologies are developed and deployed in a responsible manner. Leading with trust and responsibility is not optional. Enterprise customers require this as part of their AI adoption journey, and trust is essential to a future in which AI creates opportunities for everyone.
Salesforce is proud to be one of the first companies to contribute to the reporting framework developed by the OECD under the G7 Hiroshima AI Process (HAIP). Voluntary frameworks like this empower organisations to prioritise ethical practices, transparency, and governance at every stage of AI development and deployment, fostering more trustworthy AI ecosystems and enhancing global alignment on best practices.
Risk identification: Laying the foundation for trustworthy AI
An effective, responsible AI approach begins with a comprehensive strategy for risk identification and evaluation. Organisations should define and classify different types of AI-related risks, particularly those that could cause serious harm. This is especially important in enterprise settings, where AI systems are often tailored and used in various contexts. At Salesforce, the Responsible AI and Tech (RAIT) product managers within our Office of Ethical and Humane Use (OEHU) are central to this effort. During these reviews, RAIT product managers work closely with product teams to understand use cases, technology stacks, and intended audiences. The process involves identifying and categorising potential risks into subtypes of sociotechnical harm, as well as assessing both inherent and residual risks to provide a holistic view of potential impacts, enabling informed decision-making and effective mitigation strategies.
Our AI Acceptable Use Policy provides clear guidance on the uses for which our customers are prohibited from using AI tools. This includes automated decision-making with legal consequences, predictions of an individual’s protected characteristics, or high-risk scenarios that could result in serious harm or injury.
Ongoing risk management: Protecting AI systems in real-time
Responsible AI experts must collaborate closely with product teams at all stages of the innovation process to devise effective mitigation strategies. Standardised guardrails, such as Salesforce’s “trust patterns”, can include features like mindful friction, which introduces checkpoints for thoughtful decision-making, or transparency notifications that inform users when they are interacting with AI systems.
Organisations should also establish comprehensive frameworks that protect data privacy and security throughout every stage of the product development process. Salesforce’s Trust Layer includes functionalities such as secure data handling, zero data retention, ethics by design, an audit trail, and real-time toxicity detection.
Finally, Salesforce has clear evidence from enterprise customers that testing products against trust and safety metrics, such as bias, privacy, and truthfulness, is an important business strategy and benefit. At Salesforce, we regularly introduce red teaming exercises, which simulate potential risks in controlled environments, to identify vulnerabilities and risks within products. Tactics like this are particularly important as autonomous agents become increasingly widespread.
Transparency reporting: Building trust through honest communication and knowledge-sharing
Transparency and honesty are core tenets of our trusted AI principles, which we augmented with our guidelines for trusted generative AI, and remain applicable to the agentic AI era. Organisations should ensure that users and stakeholders are informed about how and when AI is used. At Salesforce, we regularly share information about our product capabilities through our newsroom, blogs, and Trailhead, our free online learning platform.
Salesforce also regularly reports on our progress in responsible AI efforts. Most recently, our Trusted AI and Agents Report explained our approach to designing and deploying AI agents.
Furthermore, we aim to be transparent about the use of personal data. Salesforce enables customers to control how their data is used for AI. Whether using our own Salesforce-hosted models or external models within our shared trust boundary, no context is stored. The large language model forgets both the prompt and the output immediately after processing.
Organisational governance: Embedding responsible AI practices across the company
Gaining the buy-in from all parts of the organisation to deliver a truly effective responsible AI approach is critical. Salesforce embeds AI risk management within its organisational governance framework through various structures and practices. The company’s trusted AI principles, first developed in 2018 and augmented for generative AI in 2023, guide responsible development and deployment, focusing on intentional design and system-level controls.
Our governance infrastructure includes:
The Office of Ethical and Humane Use (OEHU), which regularly interacts with the executive leadership team for policy and product review and approval. The OEHU also leads the Trusted AI Review process to identify, mitigate, and track potential risks early in development.
The AI Trust Council, comprising executives across various departments, aligns and speeds up decision-making for AI products.
The Ethical Use Advisory Council, established in 2018 with external experts and internal executives, provides strategic guidance on product and policy recommendations.
The Cybersecurity and Privacy Committee of the Board of Directors, which meets quarterly with the Chief Ethical and Humane Use Officer to review AI priorities.
The Human Rights Steering Committee, meeting quarterly, oversees the human rights program, including identifying and mitigating salient risks.
A shared commitment to responsible AI: Aligning with global standards
The future of responsible AI depends on a collective commitment to developing systems that are innovative, trustworthy, ethical, and secure. Emphasising transparency and robust governance will unlock AI’s full potential while ensuring the safety of customers and stakeholders.
The G7 HAIP reporting framework provides an effective global benchmark for responsible AI initiatives, providing a structured approach for organisations to manage the risks and benefits of AI technologies. As these frameworks gain widespread adoption, they will promote consistency in responsible AI practices, building greater trust among users and society. Salesforce is committed to working with all stakeholders and navigating this transformative AI era with trust, responsibility, and ethics guiding the way.
As enterprises rush to harness the value of artificial intelligence, questions of governance, ethics, and accountability are at the forefront. Responsible AI — systems developed in alignment with human values, legal safeguards and social norms — has become essential not just for risk mitigation, but for building lasting trust.
NetApp Inc. supports responsible AI both internally and externally by embedding ethical principles, governance structures and secure infrastructure into its AI governance frameworks, according to Beth O’Callahan (pictured), executive vice president, chief administrative officer and corporate secretary at NetApp.
NetApp’s Beth O’Callahan talks with theCUBE about the essence of responsible AI.
“What we’re putting out into the world is a reflection of NetApp’s culture, that we’re doing responsible AI, both internally and externally,” O’Callahan said. “That means that we are using and deploying AI in a way that’s responsible, unbiased, transparent [and] compliant. We can do that internally by having robust governance from the top of the company through cross-functionality. From an external perspective, we are actually helping customers do AI or use AI responsibly.”
Inside NetApp’s responsible AI and future-ready approach
NetApp bridges its legacy of trusted data management with the transformative power of generative AI to drive reinvention and growth. By combining innovation with reliability and a strong commitment to responsible AI, the company fosters a future-ready approach that keeps customer needs at the center, O’Callahan pointed out.
“I think that if you look at NetApp’s history, at any inflection point in technology, whether it’s the internet or virtualization, looking back on our storied history, that is really when we shine,” she said. “I think gen AI is yet another opportunity for us to embrace … it’s taking us to the next level. We are embracing it with our optimism that we have in our culture, and people know that we can get there. We have people internally super excited to use AI to make sure that we’re improving productivity.”
NetApp fosters a culture centered on customers and driven by innovation. The company prioritizes understanding and addressing customer needs, ensuring that solutions are tailored and impactful. Internally, NetApp supports innovation through dedicated hubs and encourages cross-functional collaboration to rapidly bring new ideas to market, according to O’Callahan.
“I think the thing that is a hallmark of the NetApp culture is that it’s a lot of great people, smart people, trying to do the right thing and trying to deliver great outcomes for customers,” she said. “The way that we build that in the culture is that we really operate as a team, and we put innovation first. The other thing is because we are a collaborative culture, we actually see a lot of cross-functional work together to drive those business outcomes, which in the end, is the best thing for the company as well.”
(* Disclosure: TheCUBE is a paid media partner for the “Architecting Outcomes in the Era of Intelligence” event. Neither NetApp Inc., the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
Photo: SiliconANGLE
Support our open free content by sharing and engaging with our content and community.
Join theCUBE Alumni Trust Network
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
11.4k+
CUBE Alumni Network
C-level and Technical
Domain Experts
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
