Connect with us

AI Research

A novel benchmark for evaluating cross-lingual knowledge transfer in LLMs

Published

on


Data creation and verification

To construct ECLeKTic, we started by selecting articles that only exist in a single language on Wikipedia for 12 languages — English, French, German, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Mandarin Chinese, Portuguese, and Spanish. These pages are often based on topics most salient to speakers of that language, but they may very well include information that is of interest to others around the world. Of course, models may learn about these topics from other sources, but since it is not possible to analyze the training data of every LLM, we use presence in Wikipedia as a proxy for whether the model has seen information in a particular language. With this assumption, focusing on this kind of content suggests that models would need to internally transfer the knowledge from the source language to the other 11 target languages in order to solve ECLeKTic’s QA task.

Specifically, we analyzed the July 2023 download of Wikipedia. For each language, we selected 100 random articles that contained at least 200 characters, had at least 100 views during 2023, and most importantly, did not have equivalent articles in any of the other 11 languages. From each selected article we extracted the first ten sentences. Based on one fact mentioned in these sentences, human annotators filtered and corrected question and answer pairs that were generated by Gemini. The annotators, each native in the relevant language, first made sure that the question is answerable in a closed book setting, i.e., it does not refer explicitly to the surrounding context in the Wikipedia article, nor does it mention the answer. Second, they validated that the question is related to information that is particularly salient for the speakers of the language in question, and less related to general knowledge, like science or current events. Questions and answers that did not meet these criteria were discarded. Third, in a process called decontextualization, the annotators confirmed that the question contains all the information needed to be answerable when translated. For example, a question in Hebrew relating to the “supreme court” was disambiguated by the annotators to explicitly mention “the Israeli supreme court”. Named entities were also clarified similarly, so a question referring to “Ambev” was modified to refer to “the Brazilian brewing company, Ambev”.

Finally, each retained question and answer were automatically translated into the other 11 languages. The translations were verified by another set of human annotators and modified when needed. At this stage, some examples were also discarded if they proved to be untranslatable — for example, when a question explicitly refers to the meaning of a word in the source language.

Based on this approach, the final ECLeKTic dataset consists of 384 unique questions and 4224 translated examples.



Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

AI Research

Which countries are producing more AI Researchers? Where does India stand? – WION

Published

on



Which countries are producing more AI Researchers? Where does India stand?  WION



Source link

Continue Reading

AI Research

3 Artificial Intelligence ETFs to Buy With $100 and Hold Forever

Published

on


If you want exposure to the AI boom without the hassle of picking individual stocks, these three AI-focused ETFs offer diversified, long-term opportunities.

Artificial intelligence (AI) has been a huge catalyst for the portfolios of many investors over the past several years. Large tech companies are spending hundreds of billions of dollars to build out their AI hardware infrastructure, creating massive winners like semiconductor designer Nvidia.

But not everyone wants to go hunting for the next big AI winner, nor is it easy to know which company will stay in the lead even if you do your own research and find a great artificial intelligence stock to buy. That’s where exchange-traded funds (ETFs) can help.

If you’re afraid of missing out on the AI boom, and have around $100 to invest right now, here are three great AI exchange-traded funds that will allow you to track some of the biggest names in artificial intelligence, no matter who’s leading the pack.

Image source: Getty Images.

1. Global X Artificial Intelligence and Technology ETF

The Global X Artificial Intelligence and Technology ETF (AIQ 0.87%) is one of the top AI ETF options for investors because it holds a diverse group of around 90 stocks, spanning semiconductors, data infrastructure, and software. Its portfolio includes household names like Nvidia, Microsoft, and Alphabet, alongside lesser-known players that give investors exposure to AI companies they might not otherwise consider.

Another strength of AIQ is its global reach: the fund invests in both U.S. and international companies, providing broader diversification across the AI landscape. Of course, this targeted approach comes at a cost. AIQ’s expense ratio of 0.68% is slightly higher than the average ETF (around 0.56%), but it’s in line with other AI-focused funds.

Performance-wise, the Global X Artificial Intelligence and Technology ETF has rewarded investors. Over the past three years, it gained 117%, trouncing the S&P 500‘s 63% return over the same period. While past performance doesn’t guarantee future results, this track record shows how powerful exposure to AI-focused companies can be.

2. Global X Robotics and Artificial Intelligence ETF

As its name suggests, the Global X Robotics and Artificial Intelligence ETF (BOTZ -0.21%) focuses on both robotics and artificial intelligence companies, as well as automation investments. Two key holdings in the fund are Pegasystems, which is an automation software company, as well as Intuitive Surgical, which creates robotic-assisted surgical systems. And yes, you’ll still have exposure to top AI stocks, including Nvidia as well.

Having some exposure to robotics and automation could be a wise long-term investment strategy. For example, UBS estimates that there will be 2 million humanoid robots in the workforce within the next decade and could reach 300 million by 2050 — reaching an estimated market size of $1.7 trillion.

If you’re inclined to believe that robotics is the future, the Global X Robotics and Artificial Intelligence ETF is a good way to spread out your investments across 49 individual companies that are betting on this future. You’ll pay an annual expense ratio of 0.68% for the fund, which is comparable to the Global X Artificial Intelligence and Technology ETF’s fees.

The fund has performed slightly better than the broader market over the past three years — gaining about 68%. Still, as robotics grows in the coming years, this ETF could be a good place to have some money invested.

3. iShares Future AI and Tech ETF

And finally, the iShares Future AI and Tech ETF (ARTY 1.72%) offers investors exposure to 48 global companies betting on AI infrastructure, cloud computing, and machine learning.

Some of the fund’s key holdings include the semiconductor company Advanced Micro Devices, Arista Networks, and the AI chip leader Broadcom, which just inked a $10 billion semiconductor deal with a large new client (widely believed to be OpenAI). In addition to its diversification across AI and tech companies, the iShares Future AI and Tech ETF also has a lower expense ratio than some of its peers, charging just 0.47% annually.

The fund has slightly underperformed the S&P 500 lately, gaining about 61% compared to the broader market’s 63% gains over the past three years. But with its strong diversification among tech and AI leaders, as well as its lower expense ratio, investors looking for a solid play on the future of artificial intelligence will find what they’re looking for in this ETF.

Chris Neiger has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Advanced Micro Devices, Alphabet, Arista Networks, Intuitive Surgical, Microsoft, and Nvidia. The Motley Fool recommends Broadcom and recommends the following options: long January 2026 $395 calls on Microsoft and short January 2026 $405 calls on Microsoft. The Motley Fool has a disclosure policy.



Source link

Continue Reading

AI Research

Companies Bet Customer Service AI Pays

Published

on

By


Klarna’s $15 billion IPO was more than a financial milestone. It spotlighted how the Swedish buy-now-pay-later (BNPL) firm is grappling with artificial intelligence (AI) at the heart of its operations.

Back in 2023, Chief Executive Sebastian Siemiatkowski suggested AI could replace large parts of the company’s customer-service workforce. The remarks sparked pushback from employees and skepticism from customers, many of whom doubted whether the technology was advanced enough to provide empathy and reliability at scale.

Pivoting and Learning

Klarna’s first wave of AI adoption proved too rigid, with customers finding the experience inconsistent. The company now pivoted toward a blended approach: AI for speed and scale, humans for empathy and trust. That adjustment echoes a lesson resonating across industries. AI works best when it augments, rather than replaces, human agents.

The company’s focus on human-powered customer support shows how the firm is hiring again to ensure customers always have the option of speaking to a person. “From a brand perspective, a company perspective, I just think it’s so critical that you are clear to your customer that there will be always a human if you want,” Siemiatkowski told Bloomberg News, as reported by PYMNTS.

As Vinod Muthukrishnan, vice president and chief operating officer of Webex Customer Experience Solutions at Cisco, explained, many financial institutions are moving past pilots and into deployment.

“These firms are increasingly leveraging their AI focus on hyper-personalized CX [customer experience] such as personal financial advice or dynamic credit limit adjustments and offers, all enabled via real-time analytics,” he told PYMNTS. Retailers and service providers face similar opportunities, provided they align strategy with measurable ROI.

Five Areas for AI, Customer Care

1. Proactive Issue Resolution

AI can anticipate problems before customers complain. Declined payments, unexpected fees or delivery delays can be flagged and addressed in real time, turning frustration into loyalty. Most firms still operate reactively, in part because data remains siloed across payments, logistics and support and closing these gaps could sharply reduce call volumes.

2. Hyper-Personalized Support

Consumers now expect service that reflects their history and preferences. AI can tailor repayment options, loyalty incentives, or offers based on real-time data. Walmart, for example, has deployed AI-powered personalization tools to refine its app and eCommerce experience. Predictive analytics can also flag anomalies that suggest fraud or disputes, thereby reducing chargebacks. Yet many retailers still rely on generic scripts.

3. Multilingual, 24/7 Coverage

Global commerce does not keep office hours. AI chatbots and voice systems provide round-the-clock, multilingual support. New multimodal systems can handle voice, text, and even images, creating richer customer interactions. PYMNTS has reported that customers value this always-on flexibility, but many firms still lean on nine-to-five call centers or outsourced night shifts.

4. Sentiment Detection and Emotional Intelligence

Speed matters, but empathy builds loyalty. AI can read tone and phrasing in real time, alerting human agents when a customer is upset. This hybrid model ensures efficiency without sacrificing trust. Rezolve’s Brain Suite applies empathy-driven AI to reduce cart abandonment, which accounts for nearly 70% of lost online sales. Yet sentiment detection remains rare in many call centers.

5. Insights Beyond the Call Center

Complaints can expose flaws in checkout flows, packaging or design. AI can analyze these patterns, turning customer service into a source of business intelligence. Google’s Vision Match tools, for example, feed insights from shopping behavior back into product strategy. Few enterprises close this loop.

ROI as the Deciding Factor

For executives, ROI is the real test. Projects that fail to deliver lower handle times, better satisfaction scores, or reduced churn rarely scale. “AI as with any new technology risks adoption and integration without a clear strategic alignment,” Muthukrishnan warned. “Too many pilots or implementations can lead to a fragmented focus.”

 “We’re already in market with our AI agent for autonomous and scripted self-service,” Todd Fisher, CEO and co-founder of CallTrackingMetrics, told PYMNTS.  

In a recent survey, 72% of respondents rated Webex AI Agent as equal, if not better, than a human agent. And our customers have reported an 85% reduction in agent call escalations, a 22% reduction in average handle time, and a 39% increase in CSAT [customer satisfaction] scores.” 



Source link

Continue Reading

Trending