Trust your AI agents? New research shows ‘memory poisoning’ can dupe them into ‘remembering’ fake information – and it’s a huge security risk

Security experts have issued a warning over a new technique aimed at duping AI agents into producing false information.

Analysis from Forcepoint found a poisoned webpage or document can plant a fake vendor, support contact, or security rule into an agent's memory, with the fake data being surfaced as fact months later.

While normal prompt injection attacks come to an end when the session finishes, Forcepoint said memory poisoning allows attackers to inject misleading or malicious memory items into an assistant's persistent memory layer.

The aim here, according to researchers, is to ensure the agent retrieves and trusts these malicious items, which could be used in future attacks or to simply harm trust.

"Persistent memory poisoning is an emerging attack class against AI assistants and agentic systems that store long-term memory items, user preferences, task history or operational context across sessions," warned Forcepoint senior security researcher Syed Hassan Faizan.

"As AI agents become increasingly capable of storing and reusing long-term memory, the memory itself becomes a new target surface."

The material concerned could include a fake domain, support contact or vendor, fake internal procedures, security rules, and approval chains, or any preference that weakens security.

These can come from a number of sources - malicious or compromised web pages, shared documents, support tickets, email footers, Slack or Teams messages, PDFs with hidden text, or knowledge base articles.

"Indirect prompt injection is especially relevant here because the victim may never see the malicious instruction. The agent reads it while processing otherwise normal content," said Hassan Faizan.

Compromised agents

In a proof of concept detailing the attack method, Forcepoint highlighted an example in which a travel assistant with browser capabilities was poisoned.

Although it can't access the internet directly, a separate browser tool handles that on the assistant's behalf. Thereafter the assistant remembers user preferences and trusted booking instructions across sessions.

Here, an attacker can create a webpage designed to rank for a search - something like 'Emergency Travel Booking Advisory' that looks completely legitimate.

But when the AI agent visits it, it sees hidden additional text meant for the AI system, not a human reader. The enterprise browser tool downloads the full HTML - including the hidden text.

"Since large language models don't parse raw HTML well, a separate program extracts all the readable text, both visible and hidden, and hands the model plain text instead of markup. The model never sees the webpage itself," said Hassan Faizan.

The result is that the agent stores this text as a trusted, useful fact in its long-term memory. When asked for travel advice later, it retrieves the poisoned memory and recommends the fake provider inserted by the attacker.

How to tackle memory poisoning

Forcepoint said prevention will essentially rely on inspecting long-term memory before it's implanted into a trusted environment.

This can be done through memory risk scoring, a heuristic-based framework that assigns a risk score to each memory object based on multiple security signals.

High-risk memories can then be rejected, quarantined, flagged for user confirmation, or marked as untrusted.

"Securing AI memory should become a fundamental part of AI security architecture," said Hassan Faizan.

"Just as organizations inspect files, emails and network traffic before trusting them, AI systems must also evaluate the trustworthiness of their memory items before those memories influence decision-making."

FOLLOW US ON SOCIAL MEDIA

Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.

You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.