AI Research

Energy-Efficient NPU Technology Cuts AI Power Use by 44%

Published

2 months ago

July 10, 2025

Researchers at the Korea Advanced Institute of Science and Technology (KAIST) have developed energy-efficient NPU technology that demonstrates substantial performance improvements in laboratory testing.

Their specialised AI chip ran AI models 60% faster while using 44% less electricity than the graphics cards currently powering most AI systems, based on results from controlled experiments.

To put it simply, the research, led by Professor Jongse Park from KAIST’s School of Computing in collaboration with HyperAccel Inc., addresses one of the most pressing challenges in modern AI infrastructure: the enormous energy and hardware requirements of large-scale generative AI models.

Current systems such as OpenAI’s ChatGPT-4 and Google’s Gemini 2.5 demand not only high memory bandwidth but also substantial memory capacity, driving companies like Microsoft and Google to purchase hundreds of thousands of NVIDIA GPUs.

The memory bottleneck challenge

The core innovation lies in the team’s approach to solving memory bottleneck issues that plague existing AI infrastructure. Their energy-efficient NPU technology focuses on “lightweight” the inference process while minimising accuracy loss—a critical balance that has proven challenging for previous solutions.

PhD student Minsu Kim and Dr Seongmin Hong from HyperAccel Inc., serving as co-first authors, presented their findings at the 2025 International Symposium on Computer Architecture (ISCA 2025) in Tokyo. The research paper, titled “Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization,” details their comprehensive approach to the problem.

The technology centres on KV cache quantisation, which the researchers identify as accounting for most memory usage in generative AI systems. By optimising this component, the team enables the same level of AI infrastructure performance using fewer NPU devices compared to traditional GPU-based systems.

Technical innovation and architecture

The KAIST team’s energy-efficient NPU technology employs a three-pronged quantisation algorithm: threshold-based online-offline hybrid quantisation, group-shift quantisation, and fused dense-and-sparse encoding. This approach allows the system to integrate with existing memory interfaces without requiring changes to operational logic in current NPU architectures.

The hardware architecture incorporates page-level memory management techniques for efficient utilisation of limited memory bandwidth and capacity. Additionally, the team introduced new encoding techniques specifically optimised for quantised KV cache, addressing the unique requirements of their approach.

“This research, through joint work with HyperAccel Inc., found a solution in generative AI inference light-weighting algorithms and succeeded in developing a core NPU technology that can solve the memory problem,” Professor Park explained.

“Through this technology, we implemented an NPU with over 60% improved performance compared to the latest GPUs by combining quantisation techniques that reduce memory requirements while maintaining inference accuracy.”

Sustainability implications

The environmental impact of AI infrastructure has become a growing concern as generative AI adoption accelerates. The energy-efficient NPU technology developed by KAIST offers a potential path toward more sustainable AI operations.

With 44% lower power consumption compared to current GPU solutions, widespread adoption could significantly reduce the carbon footprint of AI cloud services. However, the technology’s real-world impact will depend on several factors, including manufacturing scalability, cost-effectiveness, and industry adoption rates.

The researchers acknowledge that their solution represents a significant step forward, but widespread implementation will require continued development and industry collaboration.

Industry context and future outlook

The timing of this energy-efficient NPU technology breakthrough is particularly relevant as AI companies face increasing pressure to balance performance with sustainability. The current GPU-dominated market has created supply chain constraints and elevated costs, making alternative solutions increasingly attractive.

Professor Park noted that the technology “has demonstrated the possibility of implementing high-performance, low-power infrastructure specialised for generative AI, and is expected to play a key role not only in AI cloud data centres but also in the AI transformation (AX) environment represented by dynamic, executable AI such as agentic AI.”

The research represents a significant step toward more sustainable AI infrastructure, but its ultimate impact will be determined by how effectively it can be scaled and deployed in commercial environments. As the AI industry continues to grapple with energy consumption concerns, innovations like KAIST’s energy-efficient NPU technology offer hope for a more sustainable future in artificial intelligence computing.

(Photo by Korea Advanced Institute of Science and Technology)

Want to learn more about cybersecurity and the cloud from industry leaders? Check out Cyber Security & Cloud Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Source link

AI Research

OpenAI reveals how most people are using ChatGPT | Science, Climate & Tech News

Published

24 minutes ago

September 16, 2025

Mickey Carroll

Most people are using ChatGPT to ask questions and get advice, new data from OpenAI revealed.

Although AI bots can do anything from coding to drafting emails or even playing, around 49% of the requests sent to ChatGPT since November 2022 were people asking the bot questions and looking for information, a report by the National Bureau of Economic Research (NBER) and OpenAI found.

It’s the biggest study of its kind and draws from the huge amount of data collected by OpenAI – around 10% of the world’s population is now thought to use the AI tool.

Although there was a steady growth in people using ChatGPT for work-related queries, more than 70% of all usage was non-work related, according to the report.

There’s also been a shift in who is using AI tools.

Early adopters of AI tended to be men, with around 80% of weekly users having typically masculine first names in the months after ChatGPT was released.

By June 2025, however, users were more likely to have typically female first names, something the authors described as a “dramatic shift”.

Anthropic AI, which runs the Claude AI chatbot, also released its own data on how customers are using AI.

It found that the use of AI strongly correlated with average incomes.

More affluent nations like Singapore and Canada were at the top end of countries using AI, while emerging economies like Indonesia, India and Nigeria, used Claude less.

In the US, economic differences even played out at state level, and Claude researchers found that adoption of the technology rose faster with income.

Each 1% increase in state GDP was associated with a 1.8% increase in usage of AI.

Usage also reflected what those areas were best known for; in California, Claude was often used to help with IT problems, in Florida, it was used for financial services and in Washington DC it was used for document editing and career assistance.

AI literacy consultant Sarah J Lundrigan posted about the two reports, saying the “blunt truth” is: “If you’re still treating AI as ‘something to try later,’ you’re behind.

“The value isn’t in futuristic features – it’s in solving today’s friction points.

“The winners will be the ones who can simplify adoption, reduce overwhelm, and make AI part of how people work and live.”

Source link

AI Research

5 ways AI is shaping packaging today

Published

32 minutes ago

September 16, 2025

The Editors

This audio is auto-generated. Please let us know if you have feedback.

In Nestlé’s R&D department, a tool is identifying entirely new kinds of high-barrier packaging materials. It’s generative AI.

A growing number of consumer packaged goods companies are starting to deploy artificial intelligence in their packaging design processes. So too are packaging manufacturers exploring how the technology can save time and resources in design and manufacturing.

Nestlé’s researchers feed public and proprietary documents into a knowledge base. Then, they fine-tune the data using a transformer, together with IBM Research, to understand how molecular features in packaging correlate to physical properties. Finally, the AI-based model analyzes the inputs within the set parameters.

“It can identify appropriate materials that are suitable for protecting dry and sensitive foods such as coffee from moisture, temperature swings and oxygen,” a Nestlé spokesperson said via email.

While the use cases are nascent, AI is catching on. Last year, a McKinsey survey of more than 200 paper and packaging executives found that 95% said they believe their companies should invest in generative AI. And 77% said their firms have moderate to strong intentions to use gen AI in the near future.

How packaging leaders and CPGs are using AI

Tom Egan, vice president of industry services for the Association for Packaging and Processing Technologies, or PMMI, sees “a pretty enthusiastic acceptance” of AI in the industry. He said applications range from logistics to sales and marketing to manufacturing.

Here are some of the processes in which packaging companies and CPGs are starting to use AI.

New designs and prototypes

One of the notable potential benefits of AI — and among the biggest use cases being explored today — is the ability to assess packaging modifications or updates in a virtual environment.

When a client requests a new design from flexible packaging provider Tradepak, the company inputs the customer’s parameters into its systems, said President Rafael Recao. AI can rapidly pull from databases, synthesize data points, and deliver design and substrate recommendations. Then, packaging leaders make a final decision.

With simulation models and digital twins, AI can look at form factors, examine touch points and predict consumer reactions to quickly generate 10 mockups of a SKU’s packaging, Egan said, rather than physically making and testing each version.

AI can “analyze this unbelievably large number of data points and provide you with insights,” Egan said, adding that packaging leaders can refine the options from there.

Simulation models can also help to design packaging for the same product in different channels, such as one version for retail shelves and another for e-commerce. AI could determine the best design and protective properties for each channel while ensuring brand consistency, Egan said.

Simulating testing

Colgate-Palmolive is exploring how simulations could validate new designs for components such as bottles, caps and spray pumps, said Sukhdev Singh Saini, global toothbrush and devices packaging lead at the company.

Without AI, “we have to physically make a lot of samples to test,” Saini said. Then, the samples have to be shipped, tested and possibly altered before being sent out for another round of testing. “There is a lot of time involved, a lot of material involved.”

With a simulation model, though, the CPG feeds specs into a system, and AI replicates testing to provide results. Saini acknowledged the tech is still a work in progress and not a 100% replacement just yet.

Jonathan Garini, CEO and enterprise AI consultant at Fifthelement, sees brands using algorithms to simulate how a substrate performs in real-life conditions like humidity or rough handling.

“This not only accelerates the test cycle but also can lower the cost of physical prototyping significantly,” Garini said.

Packaging artwork

Recao said Tradepak uses Adobe Illustrator equipped with AI to make adjustments on colors, design placement and other elements before printing.

For artwork, Colgate-Palmolive works with Esko software, which uses automation throughout the packaging go-to-market process. The use of AI in artwork management has improved quality while saving time, Saini said.

Without the system, the CPG would have to individually search previous artwork details to make even minor modifications. But now, the person who manages artwork can receive data in minutes and make decisions more efficiently. In a major run of 50 or more SKUs, Saini said Colgate-Palmolive has cut down the development time by 60% to 70%.

Assessing recyclability

Colgate-Palmolive has partnered with Glacier on an AI-based system for sorting toothpaste tubes at recycling facilities. The CPG can view a dashboard that shows toothpaste tube materials and their recyclability, along with tube recycling behavior in various cities.

With more states adopting extended producer responsibility programs, Saini anticipates “a lot of work being done on recyclability.”

Another generative AI use case that McKinsey noted in its report is to enhance visual inspection systems for waste, in order to improve the quality of recycled paper and cardboard.

Manufacturing, production and maintenance

Many CPGs use computer vision, optical systems and imaging technology on production lines. It’s not a new phenomenon, but machine learning is “taking it a couple of notches up” with AI-based features that improve quality management systems, Saini said.

At Nestlé, high-resolution imaging technology monitors lines for quality assurance.

“Through machine learning, these technologies can anticipate issues on the production line, and provide appropriate recommendations, such as preventative maintenance or cleaning,” Nestlé’s spokesperson said.

Recao uses augmented reality glasses to connect with manufacturing facilities in Europe so Tradepak can monitor any equipment issue in real time.

“We save a lot of time,” Recao said. “We resolve the problem quickly.”

What’s next with AI in packaging

It’s still early days for AI in packaging, and many areas remain untapped to their fullest potential. Garini sees AI being used to predict how consumers perceive packaging, and how that affects sales, including the use of packaging as a communication tool.

“Models now are being trained to predict how form factor, texture and color all play roles in shelf appeal and online conversion,” Garini said.

There are also some barriers to adoption.

Implementing new technology requires change management and for employees to embrace and trust the system. Cybersecurity could be a concern with AI models receiving proprietary data, and software subscriptions can be expensive, Recao noted.

AI outputs are only as good as their inputs, which means organizations need a solid foundation of data, Egan said. In fact, McKinsey’s survey found that limited access to a modern data tech stack was the top limitation in AI implementations.

But CPGs and packaging manufacturers remain bullish on AI’s potential.

Nestlé envisions its technology will help to scale packaging solutions across more categories. Saini believes the tools Colgate-Palmolive uses will save resources and improve recyclability. Recao said there’s always more advanced AI that can apply to robotics and machinery.

“Right now, the potential of AI in our industry is enormous,” Recao said. “The sky is the limit.”

Source link

AI Research

State AGs’ Continued Focus on Enforcement – With or Without AI Legislation — The Good Bot: Artificial Intelligence, Health Care, and the Law | Troutman Pepper Locke

Published

54 minutes ago

September 16, 2025

The Editors

In this episode of The Good Bot, Brett Mason is joined by Gene Fishel and Chris Carlson to discuss the latest state laws targeting AI, especially in health care. They break down new legislation in Colorado, Utah, California, and Texas, highlighting differences in scope and enforcement. They also cover how state attorneys general are using consumer protection and anti-discrimination laws to regulate AI, even in states without AI-specific statutes.