AI Insights
New Attack Can Make AI ‘See’ Whatever You Want – Eurasia Review
Researchers have demonstrated a new way of attacking artificial intelligence computer vision systems, allowing them to control what the AI “sees.” The research shows that the new technique, called RisingAttacK, is effective at manipulating all of the most widely used AI computer vision systems.
At issue are so-called “adversarial attacks,” in which someone manipulates the data being fed into an AI system to control what the system sees, or does not see, in an image. For example, someone might manipulate an AI’s ability to detect traffic signals, pedestrians or other cars – which would cause problems for autonomous vehicles. Or a hacker could install code on an X-ray machine that causes an AI system to make inaccurate diagnoses.
“We wanted to find an effective way of hacking AI vision systems because these vision systems are often used in contexts that can affect human health and safety – from autonomous vehicles to health technologies to security applications,” says Tianfu Wu, co-corresponding author of a paper on the work and an associate professor of electrical and computer engineering at North Carolina State University. “That means it is very important for these AI systems to be secure. Identifying vulnerabilities is an important step in making these systems secure, since you must identify a vulnerability in order to defend against it.”
RisingAttacK consists of a series of operations, with the goal of making the fewest changes to an image that will allow users to manipulate what the vision AI “sees.”
First, RisingAttacK identifies all of the visual features in the image. The program also runs an operation to determine which of those features is most important to achieve the attack’s goal.
“For example,” says Wu, “if the goal of the attack is to stop the AI from identifying a car, what features in the image are most important for the AI to be able to identify a car in the image?”
RisingAttacK then calculates how sensitive the AI system is to changes in data and, more specifically, how sensitive the AI is to changes in data of the key features.
“This requires some computational power, but allows us to make very small, targeted changes to the key features that makes the attack successful,” Wu says. “The end result is that two images may look identical to human eyes, and we might clearly see a car in both images. But due to RisingAttacK, the AI would see a car in the first image but would not see a car in the second image.
“And the nature of RisingAttacK means we can influence the AI’s ability to see any of the top 20 or 30 targets it was trained to identify. So, that might be a car, a pedestrian, a bicycle, a stop sign, and so on.”
The researchers tested RisingAttacK against the four most commonly used vision AI programs: ResNet-50, DenseNet-121, ViTB and DEiT-B. The technique was effective at manipulating all four programs.
“While we demonstrated RisingAttacK’s ability to manipulate vision models, we are now in the process of determining how effective the technique is at attacking other AI systems, such as large language models,” Wu says.
“Moving forward, the goal is to develop techniques that can successfully defend against such attacks.”
Chief financial officers are recognizing the need to upskill their workforce to ensure their teams can effectively harness artificial intelligence (AI).
The Velvet Sundown, a four-person band, or so it seems, has garnered a lot of attention on Spotify. It started posting music on the platform in early June and has since released two full albums with a few more singles and another album coming soon. Naturally, listeners started to accuse the band of being an AI-generated project, which as it now turns out, is true.
The band or music project called The Velvet Sundown has over a million monthly listeners on Spotify. That’s an impressive debut considering their first album called “Floating on Echoes” hit the music streaming platform on June 4. Then, on June 19, their second album called “Dust and Silence” was added to the library. Next week, July 14, will mark the release of the third album called “Paper Sun Rebellion.” Since their debut, listeners have accused the band of being an AI-generated project and now, the owners of the project have updated the Spotify bio and called it a “synthetic music project guided by human creative direction, and composed, voiced, and visualized with the support of artificial intelligence.”
It goes on to state that this project challenges the boundaries of “authorship, identity, and the future of music itself in the age of AI.” The owners claim that the characters, stories, music, voices, and lyrics are “original creations generated with the assistance of artificial intelligence tools,” but it is unclear to what extent AI was involved in the development process.
The band art shows four individuals suggesting they are owners of the project, but the images are likely AI-generated as well. Interestingly, Andrew Frelon (pseudonym) claimed to be the owner of the AI band initially, but then confirmed that was untrue and that he pretended to run their Twitter because he wanted to insert an “extra layer of weird into this story,” of this AI band.
As it stands now, The Velvet Sundown’s music is available on Spotify with the new album releasing next week. Now, whether this unveiling causes a spike or a decline in monthly listeners, remains to be seen.
I have always been passionate about gaming and technology, which drove me towards pursuing a career in the tech writing industry. I have spent over 7 years in the tech space and about a decade in content writing. I hope to continue to use this passion and generate informative, entertaining, and accurate content for readers.
In artificial intelligence, the trend du jour is AI agents, or algorithmic bots that can autonomously retrieve data and act on it.
-
Funding & Business7 days agoKayak and Expedia race to build AI travel agents that turn social posts into itineraries
-
Jobs & Careers7 days ago
Mumbai-based Perplexity Alternative Has 60k+ Users Without Funding
-
Mergers & Acquisitions7 days ago
Donald Trump suggests US government review subsidies to Elon Musk’s companies
-
Funding & Business6 days ago
Rethinking Venture Capital’s Talent Pipeline
-
Jobs & Careers6 days ago
Why Agentic AI Isn’t Pure Hype (And What Skeptics Aren’t Seeing Yet)
-
Funding & Business4 days ago
Sakana AI’s TreeQuest: Deploy multi-model teams that outperform individual LLMs by 30%
-
Jobs & Careers6 days ago
Astrophel Aerospace Raises ₹6.84 Crore to Build Reusable Launch Vehicle
-
Funding & Business7 days agoFrom chatbots to collaborators: How AI agents are reshaping enterprise work
-
Jobs & Careers4 days ago
Ilya Sutskever Takes Over as CEO of Safe Superintelligence After Daniel Gross’s Exit
-
Funding & Business4 days ago
Dust hits $6M ARR helping enterprises build AI agents that actually do stuff instead of just talking
