News

Artificial intelligence (AI) is becoming increasingly relevant in healthcare, but at least 80% of GPs have reported that they are not at all, or not very, familiar with specific AI tools.

 

To help GPs broaden their understanding of the technology, and weigh up the potential advantages and disadvantages of its use in their practice, the RACGP has unveiled a comprehensive new resource focused on conversational AI.  

 

Unlike AI scribes, which convert a conversation with a patient into a clinical note that can be incorporated into a patient’s health record, conversational AI is technology that enables machines to interpret, process, and respond to human language in a natural way.

 

Examples include AI-powered chatbots and virtual assistants that can support patient interactions, streamline appointment scheduling, and automate routine administrative tasks.

 

The college resource offers further practical guidance on how conversational AI can be applied effectively in general practice and highlights key applications. These include:

• answering patient questions regarding their diagnosis, potential side effects of prescribed medicines or by simplifying jargon in medical reports
• providing treatment/medication reminders and dosage instructions
• providing language translation services
• guiding patients to appropriate resources
• supporting patients to track and monitor blood pressure, blood sugar, or other health markers
• triaging patients prior to a consultation
• preparing medical documentation such as clinical letters, clinical notes and discharge summaries
• providing clinical decision support by preparing lists of differential diagnoses, supporting diagnosis, and optimising clinical decision support tools (for investigation and treatment options)
• suggesting treatment options and lifestyle recommendations.

Dr Rob Hosking, Chair of the RACGP’s Practice and Technology Management Expert Committee, told newsGP there are several potential advantages to these tools in general practice.
 
‘Some of the potential benefits include task automation, reduced administrative burden, improved access to care and personalised health education for patients,’ he said.
 
Beyond the clinical setting, conversational AI tools can also have a range of business, educational and research applications, such as automating billing and analysing billing data, summarising the medical literature and answering clinicians’ medical questions.
 
However, while there are a number of benefits, Dr Hosking says it is important to consider some of the potential disadvantages to its use as well.
 
‘Conversational AI tools can provide responses that appear authoritative but on review are vague, misleading, or even incorrect,’ he explained.
 
‘Biases are inherent to the data on which AI tools are trained, and as such, particular patient groups are likely to be underrepresented in the data.
 
‘There is a risk that conversational AI will make unsuitable and even discriminatory recommendations, rely on harmful and inaccurate stereotypes, and/or exclude or stigmatise already marginalised and vulnerable individuals.’
 
While some conversational AI tools are designed for medical use, such as Google’s MedPaLM and Microsoft’s BioGPT, Dr Hosking pointed out that most are designed for general applications and not trained to produce a result within a clinical context.
 
‘The data these general tools are trained on are not necessarily up-to-date or from high-quality sources, such as medical research,’ he said.
 
The college addresses these potential problems, as well as other ethical and privacy considerations, that come with using AI in healthcare.
 
For GPs deciding whether to use conversational AI, Dr Hosking notes that there are a number of considerations to ensure the delivery of safe and quality care, and that says that patients should play a key role in the decision-making process as to whether to use it in their specific consultation.
 
‘GPs should involve patients in the decision to use AI tools and obtain informed patient consent when using patient-facing AI tools,’ he said.
 
‘Also, do not input sensitive or identifying data.’
 
However, before conversational AI is brought into practice workflows, the RACGP recommends GPs are trained on how to use it safely, including knowledge around the risks and limitations of the tool, and how and where data is stored.
 
‘GPs must ensure that the use of the conversational AI tool complies with relevant legislation and regulations, as well as any practice policies and professional indemnity insurance requirements that might impact, prohibit or govern its use,’ the college resource states.
 
‘It is also worth considering that conversational AI tools designed specifically by, and for use by, medical practitioners are likely to provide more accurate and reliable information than that of general, open-use tools.
 
‘These tools should be TGA-registered as medical devices if they make diagnostic or treatment recommendations.’
 
While the college recognises that conversational AI could revolutionise parts of healthcare delivery, in the interim, it recommends that GPs be ‘extremely careful’ in using the technology at this time.
 
‘Many questions remain about patient safety, patient privacy, data security, and impacts for clinical outcomes,’ the college said.
 
Dr Hosking, who has yet to implement conversational AI tools in his own clinical practice, shared the sentiment.
 
‘AI will continue to evolve and really could make a huge difference in patient outcomes and time savings for GPs,’ he said.
 
‘But it will never replace the important role of the doctor-patient relationship. We need to ensure AI does not create health inequities through inbuilt biases.
 
‘This will help GPs weigh up the potential advantages and disadvantages of using conversational AI in their practice and inform of the risks associated with these tools.’
 
Log in below to join the conversation.

AI AI scribes artificial intelligence conversational AI

AI doesn’t care about your beautiful website.

Visit any fashion brand’s homepage and you’ll see all sorts of dynamic or interactive elements from image carousels to dropdown menus that are designed to catch shoppers’ eyes and ease navigation.

To the large language models that underlie ChatGPT and other generative AI, many of these features might as well not exist. They’re often written in the programming language JavaScript, which for the moment at least most AI struggles to read.

This giant blindspot didn’t matter when generative AI was mostly used to write emails and cheat on homework. But a growing number of startups and tech giants are deploying this technology to help users shop — or even make the purchase themselves.

“A lot of your site might actually be invisible to an LLM from the jump,” said A.J. Ghergich, global vice president of Botify, an AI optimisation company that helps brands from Christian Louboutin to Levi’s make sure their products are visible to and shoppable by AI.

The vast majority of visitors to brands’ websites are still human, but that’s changing fast. US retailers saw a 1,200 percent jump in visits from generative AI sources between July 2024 and February 2025, according to Adobe Analytics. Salesforce predicts AI platforms and AI agents will drive $260 billion in global online sales this holiday season.

Those agents, launched by AI players such as OpenAI and Perplexity, are capable of performing tasks on their own, including navigating to a retailer’s site, adding an item to cart and completing the checkout process on behalf of a shopper. Google’s recently introduced agent will automatically buy a product when it drops to a price the user sets.

This form of shopping is very much in its infancy; the AI shopping agents available still tend to be clumsy. Long term, however, many technologists envision a future where much of the activity online is driven by AI, whether that’s consumers discovering products or agents completing transactions.

To prepare, businesses from retail behemoth Walmart to luxury fashion labels are reconsidering everything from how they design their websites to how they handle payments and advertise online as they try to catch the eye of AI and not just humans.

“It’s in every single conversation I’m having right now,” said Caila Schwartz, director of consumer insights and strategy at Salesforce, which powers the e-commerce of a number of retailers, during a roundtable for press in June. “It is what everyone wants to talk about, and everyone’s trying to figure out and ask [about] and understand and build for.”

From SEO to GEO and AEO

As AI joins humans in shopping online, businesses are pivoting from SEO — search engine optimisation, or ensuring products show up at the top of a Google query — to generative engine optimisation (GEO) or answer engine optimisation (AEO), where catching the attention of an AI responding to a user’s request is the goal.

That’s easier said than done, particularly since it’s not always clear even to the AI companies themselves how their tools rank products, as Perplexity’s chief executive, Aravind Srinivas, admitted to Fortune last year. AI platforms ingest vast amounts of data from across the internet to produce their results.

Though there are indications of what attracts their notice. Products with rich, well-structured content attached tend to have an advantage, as do those that are the frequent subject of conversation and reviews online.

“Brands might want to invest more in developing robust customer-review programmes and using influencer marketing — even at the micro-influencer level — to generate more content and discussion that will then be picked up by the LLMs,” said Sky Canaves, a principal analyst at Emarketer focusing on fashion, beauty and luxury.

Ghergich pointed out that brands should be diligent with their product feeds into programmes such as Google’s Merchant Center, where retailers upload product data to ensure their items appear in Google’s search and shopping results. These types of feeds are full of structured data including product names and descriptions meant to be picked up by machines so they can direct shoppers to the right items. One example from Google reads: Stride & Conquer: Original Google Men’s Blue & Orange Power Shoes (Size 8).

Ghergich said AI will often read this data before other sources such as the HTML on a brand’s website. These feeds can also be vital for making sure the AI is pulling pricing data that’s up to date, or as close as possible.

As more consumers turn to AI and agents, however, it could change the very nature of online marketing, a scenario that would shake even Google’s advertising empire. Tactics that work on humans, like promoted posts with flashy visuals, could be ineffective for catching AI’s notice. It would force a redistribution of how retailers spend their ad budgets.

Emarketer forecasts that spending on traditional search ads in the US will see slower growth in the years ahead, while a larger share of ad budgets will go towards AI search. OpenAI, whose CEO, Sam Altman, has voiced his distaste for ads in the past, has also acknowledged exploring ads on its platform as it looks for new revenue streams.

“The big challenge for brands with advertising is then how to show up in front of consumers when traditional ad formats are being circumvented by AI agents, when consumers are not looking at advertisements because agents are playing a bigger role,” said Canaves.

Bots Are Good Now

Retailers face another set of issues if consumers start turning to agents to handle purchases. On the one hand, agents could be great for reducing the friction that often causes consumers to abandon their carts. Rather than going through the checkout process themselves and stumbling over any annoyances, they just tell the agent to do it and off it goes.

But most websites aren’t designed for bots to make purchases — exactly the opposite, in fact. Bad actors have historically used bots to snatch up products from sneakers to concert tickets before other shoppers can buy them, frequently to flip them for a profit. For many retailers, they’re a nuisance.

“A lot of time and effort has been spent to keep machines out,” said Rubail Birwadker, senior vice president and global head of growth at Visa.

If a site has reason to believe a bot is behind a transaction — say it completes forms too fast — it could block it. The retailer doesn’t make the sale, and the customer is left with a frustrating experience.

Payment players are working to create methods that will allow verified agents to check out on behalf of a consumer without compromising security. In April, Visa launched a programme focused on enabling AI-driven shopping called Intelligent Commerce. It uses a mix of credential verification (similar to setting up Apple Pay) and biometrics to ensure shoppers are able to checkout while preventing opportunities for fraud.

“We are going out and working with these providers to say, ‘Hey, we would like to … make it easy for you to know what’s a good, white-list bot versus a non-whitelist bot,’” Birwadker said.

Of course the bot has to make it to checkout. AI agents can stumble over other common elements in webpages, like login fields. It may be some time before all those issues are resolved and they can seamlessly complete any purchase.

Consumers have to get on board as well. So far, few appear to be rushing to use agents for their shopping, though that could change. In March, Salesforce published the results of a global survey that polled different age groups on their interest in various use cases for AI agents. Interest in using agents to buy products rose with each subsequent generation, with 63 percent of Gen-Z respondents saying they were interested.

Canaves of Emarketer pointed out that younger generations are already using AI regularly for school and work. Shopping with AI may not be their first impulse, but because the behaviour is already ingrained in their daily lives in other ways, it’s spilling over into how they find and buy products.

More consumers are starting their shopping journeys on AI platforms, too, and Schwartz of Salesforce noted that over time this could shape their expectations of the internet more broadly, the way Google and Amazon did.

“It just feels inevitable that we are going to see a much more consistent amount of commerce transactions originate and, ultimately, natively happen on these AI agentic platforms,” said Birwadker.