Generative AI attacks are accelerating at an alarming rate
Two new reports from Gartner highlight the new AI-related pressures companies face, and the tools they are using to counter them
Generative AI attacks are accelerating at an alarming rate, according to Gartner, with 29% of organizations experiencing an attack on their AI application infrastructure in the last 12 months.
In a survey of 302 cybersecurity leaders in North America, EMEA, and Asia-Pacific, the consultancy found that 62% of organizations experienced a deepfake attack involving social engineering or exploiting automated processes.
Audio incidents were more common than video, with 44% reporting social engineering during a call with a supposed employee, compared with 36% in the case of video calls.
Similarly, 32% experienced deepfake audio used against automated voice biometrics, compared with 30% in the case of face biometrics or identity verification.
Analysis from the consultancy found AI assistants are now a top target for threat actors, and they’re vulnerable to a variety of adversarial prompting techniques.
Attack methods highlighted in the study included prompts aimed at manipulating large language models (LLMs) or duping multimodal models into generating malicious outputs.
All told, 32% of respondents to the Gartner survey said they’d experienced an attack of this kind over the last year, representing a significant uptick.
“As adoption accelerates, attacks leveraging GenAI for phishing, deepfakes and social engineering have become mainstream, while other threats — such as attacks on GenAI application infrastructure and prompt-based manipulations — are emerging and gaining traction," said Akif Khan, VP analyst at Gartner.
Generative AI attacks are changing the game
While 67% of cybersecurity leaders said emerging generative AI risks require significant changes to existing cybersecurity approaches, Gartner recommends a more cautious strategy.
“Rather than making sweeping changes or isolated investments, organizations should strengthen core controls and implement targeted measures for each new risk category,” said Khan.
Meanwhile, in a separate report, Gartner noted that organizations are increasingly turning to pre-emptive cybersecurity practices rather than standalone detection and response (DR).
By 2030, pre-emptive cybersecurity solutions are expected to account for 50% of IT security spending - up from less than 5% just a year ago.
Pre-emptive cybersecurity technologies use advanced AI and machine learning to anticipate and neutralize threats before they materialize. They include capabilities such as predictive threat intelligence, advanced detection, and automated moving target defense.
“Pre-emptive cybersecurity will soon be the new gold standard for every entity operating on, in, or through the various interconnected layers of the global attack surface grid (GASG),” said Carl Manion, managing vice president at Gartner.
“DR-based cybersecurity will no longer be enough to keep assets safe from AI-enabled attackers. Organizations will need to deploy additional countermeasures that act pre-emptively and independently of humans to neutralize potential attackers before they strike.”
Gartner predicts a shift from broad, one-size-fits-all DR security platforms toward more targeted pre-emptive cybersecurity tactics, many of which will be based on agentic AI and domain-specific language models (DSLMs).
Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.
MORE FROM ITPRO
- Security researchers have just identified what could be the first ‘AI-powered’ ransomware strain
- The top ransomware trends for businesses in 2025
- Agentic AI could be a blessing and a curse for cybersecurity
-
OpenAI's 'Skills in Codex' service aims to supercharge agent efficiency for developersNews The Skills in Codex service will provide users with a package of handy instructions and scripts to tweak and fine-tune agents for specific tasks.
-
Cloud infrastructure spending hit $102.6 billion in Q3 2025News Hyperscalers are increasingly offering platform-level capabilities that support multi-model deployment and the reliable operation of AI agents
-
NHS supplier DXS International confirms cyber attack – here’s what we know so farNews The NHS supplier says front-line clinical services are unaffected
-
LastPass hit with ICO fine after 2022 data breach exposed 1.6 million users – here’s how the incident unfoldedNews The impact of the LastPass breach was felt by customers as late as December 2024
-
Researchers claim Salt Typhoon masterminds learned their trade at Cisco Network AcademyNews The Salt Typhoon hacker group has targeted telecoms operators and US National Guard networks in recent years
-
Trend Micro issues warning over rise of 'vibe crime' as cyber criminals turn to agentic AI to automate attacksNews Trend Micro is warning of a boom in 'vibe crime' - the use of agentic AI to support fully-automated cyber criminal operations and accelerate attacks.
-
Cyber budget cuts are slowing down, but that doesn't mean there's light on the horizon for security teamsNews A new ISC2 survey indicates that both layoffs and budget cuts are on the decline
-
NCSC issues urgent warning over growing AI prompt injection risks – here’s what you need to knowNews Many organizations see prompt injection as just another version of SQL injection - but this is a mistake
-
Chinese hackers are using ‘stealthy and resilient’ Brickstorm malware to target VMware servers and hide in networks for months at a timeNews Organizations, particularly in the critical infrastructure, government services, and facilities and IT sectors, need to be wary of Brickstorm
-
AWS CISO Amy Herzog thinks AI agents will be a ‘boon’ for cyber professionals — and teams at Amazon are already seeing huge gainsNews AWS CISO Amy Herzog thinks AI agents will be a ‘boon’ for cyber professionals, and the company has already unlocked significant benefits from the technology internally.
