Hackers are capitalizing on AI hype to ramp up social engineering attacks – and they're using big brands like Anthropic, OpenAI, and DeepSeek as ‘bait’ to lure victims

Cyber criminals are exploiting AI hype to impersonate the branding of AI platforms such as ChatGPT, Microsoft Copilot, DeepSeek, and Anthropic’s Claude, according to new research.

Microsoft Threat Intelligence said it's observed an uptick in phishing, malvertising, and search engine optimization (SEO)-driven attacks that ultimately lead to credential theft, financial fraud, or malware infection.

Campaigns focus on highly anticipated launches or emerging trends, using tried-and-tested tactics such as urgency-driven messaging, abuse of trusted services, and multi-stage redirection chains that require user interaction to evade detection.

"While traditional lures like invoices, payment notifications, or delivery alerts remain effective and continue to be widely used, AI-themed lures reflect a shift in social engineering that is likely to persist as a long-term tactic used by threat actors, from cyber criminal groups to nation states," the company warned.

ChatGPT users in the crosshairs

In one example, Microsoft said it had observed a ChatGPT-themed phishing attack delivering malicious URLs which led to phishing pages that collected credit card and personal information such as names and addresses.

The emails used the sender display name ChatGPT and the subject line: “To ensure your ChatGPT Plus continues to work – please update your payment method”.

This phishing activity, which consisted of 4,500 emails sent to targets in South Africa, was part of a broader campaign using similar themes and infrastructure that delivered as many as 100,000 emails on a single day to targets in Switzerland, Austria, and South Africa.

Microsoft noted the campaign affected a broad range of industries, including higher education and professional services.

Thousands targeted in a Claude-themed phishing attack

In another example, security experts spotted a phishing campaign impersonating Anthropic-branded services to target users with account-related lures tied to the Claude AI platform.

The campaign sent phishing emails to targets across more than 2,000 organizations, mainly in the US, UK, and India.

"The campaign used enforcement-themed messaging claiming that the recipient’s account was in violation of acceptable use policies and required immediate action," the company noted.

"The emails impersonated Anthropic’s popular AI service Claude using the display names Anthropic Teams and Anthropic PBC, masquerading as legitimate account-related communications. Subject lines followed a consistent structure of 'Claude Appeal Request' combined with date elements."

DeepSeek malvertising is a growing threat

Other examples included malvertising campaigns that use AI-themed terms such as 'Awesome AI Windows Plugin' and 'Flux Pro AI' in social engineering lures, and fake DeepSeek V4 installers on GitHub that delivered Vidar Stealer.

"Within hours of DeepSeek previewing their latest version, V4, attackers created a fake GitHub organization and repository. They copied real branding and benchmark data, added AI and SEO-search-friendly content, and pushed malicious archives that looked like installers," explained John Bruggeman, vCISO at CBTS.

"What the attacker did was not particularly exotic, but it was well timed and convincingly packaged. A user searching for the newest model could very easily end up in the wrong place, especially because the malicious repository showed up in GitHub, Google, Bing, or AI-assisted search results. The search results added legitimacy to the malware."

Remain vigilant

To counter these rising threats, Microsoft advised customers to configure automatic attack disruption in Microsoft Defender XDR, enforce multi-factor authentication (MFA) on all accounts, use the Microsoft Authenticator app for passkeys and MFA, and scope conditional access policies to strengthen privileged accounts with phishing-resistant MFA.

Other tips included:

• Enabling Zero-hour auto purge (ZAP) in Office 365
• Configuring Microsoft Defender for Office 365 Safe Links
• Invest in ‘advanced’ anti-phishing solutions

"The companies that have a handle on AI governance (policies and procedures) well will be the ones that make safe AI use easy, risky AI use visible, and malicious activity hard to ignore. That means publishing a clear list of approved tools, blocking obvious lookalike domains and very recently registered domains can help stop this kind of threat," said Bruggeman.

"Monitoring suspicious downloads and sign-ins, and training employees on the AI-themed lures should also be done right now - don't think that generic phishing examples from five years ago are going to cut it today."

FOLLOW US ON SOCIAL MEDIA

Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.

You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.