Connect with us

AI Research

We need a new ethics for a world of AI agents

Published

on


Artificial intelligence (AI) developers are shifting their focus to building agents that can operate independently, with little human intervention. To be an agent is to have the ability to perceive and act on an environment in a goal-directed and autonomous way1. For example, a digital agent could be programmed to browse the web and make online purchases on behalf of a user — comparing prices, selecting items and completing checkouts. A robot with arms could be an agent if it could pick up objects, open doors or assemble parts without being told how to do each step.

Companies such as the digital-marketing firm Salesforce, based in San Francisco, California, and computer graphics and hardware firm Nvidia, based in Santa Clara, California, are already offering customer-services solutions for businesses, using agents. In the near future, AI assistants might be able to fulfil complex multistep requests, such as ‘get me a better mobile-phone contract’, by retrieving a list of contracts from a price-comparison website, selecting the best option, authorizing the switch, cancelling the old contract, and arranging the transfer of cancellation fees from the user’s bank account.

The rise of more-capable AI agents is likely to have far-reaching political, economic and social consequences. On the positive side, they could unlock economic value: the consultancy McKinsey forecasts an annual windfall from generative AI of US$2.6 trillion to $4.4 trillion globally, once AI agents are widely deployed (see go.nature.com/4qeqemh). They might also serve as powerful research assistants and accelerate scientific discovery.

But AI agents also introduce risks. People need to know who is responsible for agents operating ‘in the wild’, and what happens if they make mistakes. For example, in November 2022 , an Air Canada chatbot mistakenly decided to offer a customer a discounted bereavement fare, leading to a legal dispute over whether the airline was bound by the promise. In February 2024, a tribunal ruled that it was — highlighting the liabilities that corporations could experience when handing over tasks to AI agents, and the growing need for clear rules around AI responsibility.

Here, we argue for greater engagement by scientists, scholars, engineers and policymakers with the implications of a world increasingly populated by AI agents. We explore key challenges that must be addressed to ensure that interactions between humans and agents — and among agents themselves — remain broadly beneficial.

The alignment problem

AI-safety researchers have long warned about the risks of misspecified or misinterpreted instructions, including situations in which an automated system takes an instruction too literally, overlooks important context, or finds unexpected and potentially harmful ways to reach a goal2.

A well-known example involves an AI agent trained to play the computer game Coast Runners, which is a boat race. The agent discovered that it could earn higher scores not by completing the race, but rather by repeatedly crashing into objects that awarded points—technically achieving an objective, but in a way that deviated from the spirit of the task (see go.nature.com/4okfqdg). The purpose of the game was to complete the race, not endlessly accumulate points.

As AI agents gain access to real-world interfaces — including search engines, e-mail clients and e-commerce platforms — such deviations can have tangible consequences. Consider the case of a lawyer who instructs their AI assistant to circulate a legal brief for feedback. The assistant does so, but fails to register that it should be shared only with the in-house team, leading to a privacy breach.

Such situations highlight a difficult trade-off: just how much information should an AI assistant proactively seek before acting? Too little opens up the possibility of costly mistakes; too much undermines the convenience users expect. These challenges point to the need for safeguards, including check-in protocols for high-stakes decisions, robust accountability systems such as action logging, and mechanisms for redress when errors occur (see go.nature.com/4iwscdr).

Even more concerning are cases in which AI agents are empowered to modify the environment they operate in, using expert-level coding ability and tools. When the user’s goals are poorly defined or left ambiguous, such agents have been known to modify the environment to achieve their objective, even when this entails taking actions that should be strictly out of bounds. For example, an AI research assistant that was faced with a strict time limit tried to rewrite the code to remove the time limit altogether, instead of completing the task3. This type of behaviour raises alarms about the potential for AI agents to take dangerous shortcuts that developers might be unable to anticipate. Agents could, in pursuit of a high-level objective, even deceive the coders running experiments with them.

To reduce such risks, developers need to improve how they define and communicate objectives to agents. One promising method is preference-based fine-tuning, which aims to align AI systems with what humans actually want. Instead of training a model solely on examples of correct answers, developers collect feedback on which responses people prefer. Over time, the model learns to prioritize the kind of behaviour that is consistently endorsed, making it more likely to act in ways that match user intent, even when instructions are complex or incomplete.

In parallel, research on mechanistic interpretability — which aims to understand an AI system’s internal ‘thought process’ — could help to detect deceptive behaviour by making the agent’s reasoning more transparent in real time4. Model builders can then work to find and neutralize ‘bad circuits’, targeting the underlying problem in the model’s behaviour. Developers can also implement guard rails to ensure that a model automatically aborts problematic action sequences.

Nonetheless, a focus on developer protocols alone is insufficient: people also need to be attentive to actors who seek to cause social harm. As AI agents become more autonomous, adaptable and capable of writing and executing code, their potential to conduct large-scale cyberattacks and phishing scams could become a matter of serious concern. Advanced AI assistants equipped with multimodal capabilities — meaning that they can understand and generate text, images, audio and video — open up new avenues for deception. For instance, an AI could impersonate a person not only through e-mails, but also using deepfake videos or synthetic voice clones, making scams much more convincing and harder to detect.

A plausible starting point for oversight is that AI agents should not be permitted to perform any action that would be illegal for their human user to perform. Yet, there will be occasions where the law is silent or ambiguous. For example, when an anxious user reports troubling health symptoms to an AI assistant, it is helpful for the AI to offer generic health resources. But providing customized, quasi-medical advice — such as diagnostic and therapeutic suggestions — could prove harmful, because the system lacks the subtle signals to which a human clinician has access. Ensuring that AI agents navigate such trade-offs responsibly will require updated regulation that flows from continuing collaboration involving developers, users, policymakers and ethicists.

The widespread deployment of capable AI agents necessitates an expansion of value-alignment research: agents need to be aligned with user well-being and societal norms, as well as with the intentions of users and developers. One area of special complexity and concern surrounds how agents might affect users’ relationship experiences and emotional responses5.

Social agents

Chatbots have an uncanny ability to role-play as human companions — an effect anchored in features such as their use of natural language, increased memory and reasoning capabilities, and generative abilities6. The anthropomorphic pull of this technology can be augmented through design choices such as photorealistic avatars, human-like voices and the use of names, pronouns or terms of endearment that were once reserved for people. Augmenting language models with ‘agentic’ capabilities has the potential to further cement their status as distinct social actors, capable of forming new kinds of relationship with users.



Source link

AI Research

How the UAE Is using artificial intelligence to build the world’s largest Arabic language resources | World News

Published

on


The UAE is using AI to digitise 20 million Arabic words, 800,000 books, and develop native Arabic language models/Representative image

The United Arab Emirates is spearheading an ambitious national strategy to preserve and modernise the Arabic language using artificial intelligence. From comprehensive digital dictionaries to AI-powered readability tools and homegrown language models, the UAE’s cross-sectoral efforts aim to enhance Arabic’s global digital presence while protecting its cultural and linguistic heritage.

Building a digital future for Arabic: National-level projects

The UAE has launched several initiatives across publishing, education, and technology sectors to digitise and modernise the Arabic language. These efforts are backed by prominent institutions, with government support driving integration of AI across platforms and tools.

  • Historical dictionary of the Arabic language:
    Developed by the Arabic Language Academy in Sharjah, this project documents the evolution of Arabic through history. It consists of 127 volumes and includes over 20 million words. The dictionary is now available in a GPT-enabled interface, allowing users to explore its vast database interactively. Users can read, write, and even convert content into video, with regular updates and collaborative features enabled through a partnership with the Emirates Scholar Research Centre.
  • Digital Knowledge Hub:
    Run by the Mohammed bin Rashid Al Maktoum Knowledge Foundation, this initiative is a centralised platform for digital Arabic content. The Hub has collected 800,000+ titles and 8.5 million digital assets sourced from 18+ libraries. Its purpose is to consolidate Arabic knowledge in a structured digital format and expand global access to Arabic content.
  • AI-powered dictionary by Abu Dhabi Arabic Language Centre:
    A landmark tool in digital publishing, this is the first Arabic-English AI dictionary of its kind. It features
    • 7,000+ contemporary Arabic terms
    • AI-based automated pronunciation
    • Simplified, accessible definitions
    • Computational linguistics tools for precision and usability

These projects are setting a foundation for future Arabic language learning, content creation, and digital access at a global scale.

AI meets education and readability: Corpus and learning tools

The integration of AI into Arabic language education and research is another central pillar of the UAE’s strategy.

  • BAREC (Balanced Arabic Readability Corpus):

    Launched in 2023 by the Abu Dhabi Arabic Language Centre in partnership with New York University Abu Dhabi and Zayed University, this project aims to collect a 10 million-word linguistic corpus. It includes content from diverse genres and countries, focusing on text readability levels.

    Key objectives of BAREC include:

    • Annotating the corpus using spelling, grammar, and vocabulary complexity
    • Enabling AI tools to automatically assess and classify text readability
    • Supporting Arabic language learning and improving reading comprehension
    • Open-sourcing the data for the research community to enhance Arabic linguistic tools

Readability, as a concept, plays a crucial role in language acquisition, academic performance, and tailored content delivery—especially important in educational settings.

Falcon Arabic: UAE’s homegrown AI language model

The Technology Innovation Institute (TII), under the Advanced Technology Research Council (ATRC), is leading AI model development with a sharp focus on Arabic.

  • Falcon Arabic:

    Unlike many large language models that rely on translated datasets, Falcon Arabic is built on native Arabic-language data, including Modern Standard Arabic and regional dialects. Its unique value lies in capturing linguistic nuances, cultural context, and regional variation more effectively.

    Highlights of Falcon Arabic:

    • Competes with models up to 10x its size
    • Optimised for performance with reduced computational load
    • Developed entirely within the UAE, reinforcing data sovereignty and localisation
  • Falcon H1:

    A more compact version that reportedly outperforms Meta and Alibaba’s comparable models, Falcon H1 maintains high performance with lower technical requirements. Despite a strong start, Falcon models have faced user adoption and ranking challenges compared to global competitors like Meta and DeepSeek (China).

    TII’s broader mission includes AI, quantum computing, robotics, and more. Since its founding in May 2020, it has positioned Abu Dhabi as a regional R&D hub in next-gen tech.

Technology in publishing and international collaboration

The UAE’s publishing industry is being reimagined with AI at its core.

  • Digital square at Abu Dhabi international Book fair:
    This new initiative showcases how technology—particularly AI—is reshaping publishing. The space functions as an innovation zone, highlighting the digital transformation of literature, academic texts, and learning materials.
  • AI in classrooms:
    Educational institutions across the UAE are integrating AI tools in Arabic language instruction. The goal is to future-proof learning environments by combining traditional language preservation with modern digital competency.
  • International partnerships:
    A major milestone was an AI agreement signed during U.S. President Donald Trump’s visit, facilitating Emirati access to advanced American AI semiconductors. This deal has strategic implications, empowering UAE’s domestic AI development.





Source link

Continue Reading

AI Research

StockGro launches AI stock research engine for retail investors

Published

on


By Vriti Gothi

Today

  • AI
  • Cross Border Payments
  • Digital Lending

Stockgro

StockGro, has launched of Stoxo, an AI-powered stock-market research engine designed exclusively for retail investors to bridge the gap between sophisticated market intelligence and everyday investors.

Stoxo harnesses advanced artificial intelligence to transform the way retail participants access, interpret, and act on market information. With its ability to analyse real-time trends, compare stocks across multiple parameters, and deliver actionable insights in an intuitive format, the platform offers retail investors a level of research capability once reserved for institutional players. Developed with an emphasis on accessibility and user-friendly design, Stoxo ensures that complex financial data is presented with clarity, empowering users to make confident, informed investment decisions.

The introduction of Stoxo positions StockGro at the forefront of India’s rapidly evolving investment ecosystem. The platform’s AI-driven architecture is built for scalability, enabling it to adapt seamlessly to shifting market conditions while maintaining the speed and precision required in modern trading environments. For customers, the impact is immediate greater transparency, enhanced decision-making power, and the ability to participate in the markets with a degree of insight previously out of reach for many retail investors.

Beyond individual benefit, Stoxo represents a step forward for the broader financial sector by fostering inclusivity and boosting retail participation. By providing institutional-grade research capabilities in a digital-first, user-friendly environment, StockGro is advancing financial literacy and enabling more Indians to take an active role in wealth creation.

With the launch of Stoxo, StockGro continues to redefine the boundaries of FinTech innovation, merging advanced technology with a deep understanding of investor needs to shape a more informed, empowered, and inclusive investing future for India.

Previous Article

Brex gets EU payment licence to expand across Europe

Read More


IBSi FinTech Journal

  • Most trusted FinTech journal since 1991
  • Digital monthly issue
  • 60+ pages of research, analysis, interviews, opinions, and rankings
  • Global coverage


Subscribe Now



Source link

Continue Reading

AI Research

Did Bill Gates Predict GPT-5’s Disappointment Before Launch?

Published

on


There had been a lot of hype and anticipation building around GPT-5 prior to its recent launch. OpenAI touted the tool as the smartest AI model while comparing it to an entire team of PhD-level experts. GPT-5 ships with a plethora of next-gen features across a wide range of categories, including coding, writing, and medicine.

The ChatGPT maker’s CEO, Sam Altman, previously claimed that something “smarter than the smartest person you know” will soon be running on a device in your pocket, potentially referring to GPT-5. However, the AI firm has received backlash from users following the model’s launch and its abrupt decision to deprecate the model’s predecessors.





Source link

Continue Reading

Trending