AI Research
Benchmarking LLMs for global health
Large language models (LLMs) have shown potential for medical and health question-answering across various health-related tests and spanning different formats and sources. Indeed we have been on the forefront of efforts to expand the utility of LLMs for health and medical applications, as demonstrated in our recent work on Med-Gemini, MedPaLM, AMIE, Multimodal Medical AI, and our release of novel evaluation tools and methods to assess model performance across various contexts. Especially in low-resource settings, LLMs can potentially serve as valuable decision-support tools, enhancing clinical diagnostic accuracy, accessibility, and multilingual clinical decision support, and health training, especially at the community level. Yet despite their success on existing medical benchmarks, there is still some uncertainty about how well these models generalize to tasks involving distribution shifts in disease types, region-specific medical knowledge, and contextual variations across symptoms, language, location, linguistic diversity, and localized cultural contexts.
Tropical and infectious diseases (TRINDs) are an example of such an out-of-distribution disease subgroup. TRINDs are highly prevalent in the poorest regions of the world, affecting 1.7 billion people globally with disproportionate impacts on women and children. Challenges in preventing and treating these diseases include limitations in surveillance, early detection, accurate initial diagnosis, management, and vaccines. LLMs for health-related question answering could potentially enable early screening and surveillance based on a person’s symptoms, location, and risk factors. However, only limited studies have been conducted to understand LLM performance on TRINDs with few datasets existing for rigorous LLM evaluation.
To address this gap, we have developed synthetic personas — i.e., datasets that represent profiles, scenarios, etc., that can be used to evaluate and optimize models — and benchmark methodologies for out-of-distribution disease subgroups. We have created a TRINDs dataset that consists of 11,000+ manually and LLM-generated personas representing a broad array of tropical and infectious diseases across demographic, contextual, location, language, clinical, and consumer augmentations. Part of this work was recently presented at the NeurIPS 2024 workshops on Generative AI for Health and Advances in Medical Foundation Models.
AI Research
Radiomics-Based Artificial Intelligence and Machine Learning Approach for the Diagnosis and Prognosis of Idiopathic Pulmonary Fibrosis: A Systematic Review – Cureus
AI Research
A Real-Time Look at How AI Is Reshaping Work : Information Sciences Institute
Artificial intelligence may take over some tasks and transform others, but one thing is certain: it’s reshaping the job market. Researchers at USC’s Information Sciences Institute (ISI) analyzed LinkedIn job postings and AI-related patent filings to measure which jobs are most exposed, and where those changes are happening first.
The project was led by ISI research assistant Eun Cheol Choi, working with students in a graduate-level USC Annenberg data science course taught by USC Viterbi Research Assistant Professor Luca Luceri. The team developed an “AI exposure” score to measure how closely each role is tied to current AI technologies. A high score suggests the job may be affected by automation, new tools, or shifts in how the work is done.
Which Industries Are Most Exposed to AI?
To understand how exposure shifted with new waves of innovation, the researchers compared patent data from before and after a major turning point. “We split the patent dataset into two parts, pre- and post-ChatGPT release, to see how job exposure scores changed in relation to fresh innovations,” Choi said. Released in late 2022, ChatGPT triggered a surge in generative AI development, investment, and patent filings.
Jobs in wholesale trade, transportation and warehousing, information, and manufacturing topped the list in both periods. Retail also showed high exposure early on, while healthcare and social assistance rose sharply after ChatGPT, likely due to new AI tools aimed at diagnostics, medical records, and clinical decision-making.
In contrast, education and real estate consistently showed low exposure, suggesting they are, at least for now, less likely to be reshaped by current AI technologies.
AI’s Reach Depends on the Role
AI exposure doesn’t just vary by industry, it also depends on the specific type of work. Jobs like software engineer and data scientist scored highest, since they involve building or deploying AI systems. Roles in manufacturing and repair, such as maintenance technician, also showed elevated exposure due to increased use of AI in automation and diagnostics.
At the other end of the spectrum, jobs like tax accountant, HR coordinator, and paralegal showed low exposure. They center on work that’s harder for AI to automate: nuanced reasoning, domain expertise, or dealing with people.
AI Exposure and Salary Don’t Always Move Together
The study also examined how AI exposure relates to pay. In general, jobs with higher exposure to current AI technologies were associated with higher salaries, likely reflecting the demand for new AI skills. That trend was strongest in the information sector, where software and data-related roles were both highly exposed and well compensated.
But in sectors like wholesale trade and transportation and warehousing, the opposite was true. Jobs with higher exposure in these industries tended to offer lower salaries, especially at the highest exposure levels. The researchers suggest this may signal the early effects of automation, where AI is starting to replace workers instead of augmenting them.
“In some industries, there may be synergy between workers and AI,” said Choi. “In others, it may point to competition or replacement.”
From Class Project to Ongoing Research
The contrast between industries where AI complements workers and those where it may replace them is something the team plans to investigate further. They hope to build on their framework by distinguishing between different types of impact — automation versus augmentation — and by tracking the emergence of new job categories driven by AI. “This kind of framework is exciting,” said Choi, “because it lets us capture those signals in real time.”
Luceri emphasized the value of hands-on research in the classroom: “It’s important to give students the chance to work on relevant and impactful problems where they can apply the theoretical tools they’ve learned to real-world data and questions,” he said. The paper, Mapping Labor Market Vulnerability in the Age of AI: Evidence from Job Postings and Patent Data, was co-authored by students Qingyu Cao, Qi Guan, Shengzhu Peng, and Po-Yuan Chen, and was presented at the 2025 International AAAI Conference on Web and Social Media (ICWSM), held June 23-26 in Copenhagen, Denmark.
Published on July 7th, 2025
Last updated on July 7th, 2025
AI Research
SERAM collaborates on AI-driven clinical decision project
The Spanish Society of Medical Radiology (SERAM) has collaborated with six other scientific societies to develop an AI-supported urology clinical decision-making project called Uro-Oncogu(IA)s.
The initiative produced an algorithm that will “reduce time and clinical variability” in the management of urological patients, the society said. SERAM’s collaborators include the Spanish Urology Association (AEU), the Foundation for Research in Urology (FIU), the Spanish Society of Pathological Anatomy (SEAP), the Spanish Society of Hospital Pharmacy (SEFH), the Spanish Society of Nuclear Medicine and Molecular Imaging (SEMNIM), and the Spanish Society of Radiation Oncology (SEOR).
SERAM Secretary General Dr. MaríLuz Parra launched the project in Madrid on 3 July with AEU President Dr. Carmen González.
On behalf of SERAM, the following doctors participated in this initiative:
- Prostate cancer guide: Dr. Joan Carles Vilanova, PhD, of the University of Girona,
- Upper urinary tract guide: Dr. Richard Mast of University Hospital Vall d’Hebron in Barcelona,
- Muscle-invasive bladder cancer guide: Dr. Eloy Vivas of the University of Malaga,
- Non-muscle invasive bladder cancer guide: Dr. Paula Pelechano of the Valencian Institute of Oncology in Valencia,
- Kidney cancer guide: Dr. Nicolau Molina of the University of Barcelona.
-
Funding & Business7 days ago
Kayak and Expedia race to build AI travel agents that turn social posts into itineraries
-
Jobs & Careers7 days ago
Mumbai-based Perplexity Alternative Has 60k+ Users Without Funding
-
Mergers & Acquisitions7 days ago
Donald Trump suggests US government review subsidies to Elon Musk’s companies
-
Funding & Business6 days ago
Rethinking Venture Capital’s Talent Pipeline
-
Jobs & Careers6 days ago
Why Agentic AI Isn’t Pure Hype (And What Skeptics Aren’t Seeing Yet)
-
Funding & Business4 days ago
Sakana AI’s TreeQuest: Deploy multi-model teams that outperform individual LLMs by 30%
-
Jobs & Careers6 days ago
Astrophel Aerospace Raises ₹6.84 Crore to Build Reusable Launch Vehicle
-
Funding & Business1 week ago
From chatbots to collaborators: How AI agents are reshaping enterprise work
-
Tools & Platforms6 days ago
Winning with AI – A Playbook for Pest Control Business Leaders to Drive Growth
-
Jobs & Careers4 days ago
Ilya Sutskever Takes Over as CEO of Safe Superintelligence After Daniel Gross’s Exit