Bringing data to the heart of AI

It’s no secret that AI is changing and shaping the way we think about data. The rules for how we capture, store, and use information are being rewritten in real time, with greater emphasis on the speed with which we can move data closer to where data is being processed.

“For a long time, storage has been seen as just a commodity,” Richard Trembecki, HPE’s sales architect for file and object storage, explains.

“When customers set budgets for AI projects, they start with the big, heavy, expensive GPUs, the servers that contain them, the network to connect them, and then eventually they'll start thinking about how data is accessed and stored. Now we need to think of data as the heart of the project. Connecting data sources that are separated through geography, platform, protocol or cloud provider plays an important part in making data available for AI pipelines – cleaning and preparing data for efficient reuse and iterative training and inference.”

IDC believes that organizations must approach AI projects from a data-centric perspective. In its Future Enterprise Spending and Resilience research, the figures suggest that less than half (44%) of AI pilot projects reach production.

AI-ready data storage infrastructure should involve “embedded” AI, according to IDC. This refers to AI technologies used internally to enhance system use, performance, reliability, and operational efficiency. Data readiness is also a high priority – as the saying goes, your AI outcomes will only be as good as the data you feed it. These are also specific to the hardware or software in which it is embedded.

A real-world example of this is HPE Alletra Storage MP X10000, which features embedded data intelligence to power AI workloads.

The way to think of storage in the age of AI

When it comes to AI, businesses must establish a single source of truth. The long-standing principle of "garbage in, garbage out" has never been more relevant.

AI systems are only as effective as the data that powers them; poor-quality, inconsistent, or incomplete data will inevitably produce unreliable outcomes. High-quality, trusted data is the foundation of successful AI deployments, yet many organizations remain constrained by fragmented legacy storage systems that limit access to accurate, consistent information. Without a robust data foundation, even the most advanced AI models cannot deliver meaningful business value.

Many IT organizations deal with an average of 6.4 data silos per organization, according to IDC’s AI-Ready Data Storage Infrastructure white paper. Further analysis from IDC suggests that these IT teams must manage 13 copies of data, which may be spread across primary, secondary, cloud, and edge storage.

The reasons for these multiple copies of data may well be valid and necessary – created for various reasons, including backups, tests, analytics, and so on. But it presents specific challenges for AI workloads that depend on data accuracy and data timeliness. This is the only way for AI workloads to properly learn and respond to changing requirements quickly – sometimes in real-time.

To maximize the success of AI initiatives, businesses need storage infrastructure that is truly AI-ready, not simply capable of storing data, but of making it easier to find, understand, and use in real-time.

According to IDC's white paper, AI-ready infrastructure continuously organises and enriches information as it is created, giving AI applications faster access to trusted, relevant data without the need for extensive manual preparation. As legacy systems reach end of life, technology refreshes should prioritise AI readiness, ensuring every new piece of data contributes to a richer, more intelligent data foundation that accelerates AI adoption, improves the quality of AI insights, and unlocks greater business value.

The always-on approach to AI performance

Typically, AI interactions have been relatively short-lived and bursty, according to Trembecki. This may have been satisfactory for our compute needs before the AI era, but today we’re finding that even ‘fast’ can be too slow when it comes to AI.

“During the working day, users come on stream, they start interacting with models, and then when they go home at night, the model interactions stop, and so the workload reduces,” Trembecki explains. “You have peaks and troughs during the day.”

He adds: “We move on to agentic, and things become much longer-lived. The queries can take significantly longer. They can be recursive and dynamically expand in scope. This impacts costs which can be unpredictable when hosted in the public cloud. As the run time for these agentic queries extend, the concept of peaks disappears, almost towards a steady state.”

The first thing to ask here, according to Trembecki, is how we can make sure that data is available. Agentic workloads will place greater demands on existing infrastructure which may no longer be fit for purpose. This means that data storage needs to be considered earlier in the design process to ensure that this foundation is able to support this new generation of demanding workloads. Without those strong foundations the models will be starved of the vital data they need to process.

“It’s no longer enough for storage platforms to just store and make data available,” Trembecki says. “As well as accelerating access to data, we need to be part of the data pipeline, enriching the data and simplifying access for LLMs via new protocols like MCP. This is where we can add meaningful value.”

HPE Alletra Storage MP X10000

When it comes to turning data into intelligence, organizations need trusted products. This is where HPE Alletra Storage MP X10000 stands out as a leading enterprise-class storage platform.

Built with a disaggregated, shared-everything architecture that’s designed to use standard hardware components and enable the independent scaling of compute nodes and flash storage expansion shelves. It gives organizations the flexibility to increase storage capacity, independently from controllers, in contrast to traditional paired-node storage systems.

HPE’s internally developed all-flash HPE Alletra MP X10000, which came to market at the end of 2024, offers high-performance, scale-out object and file storage based on the same modular HPE Alletra Storage MP hardware architecture as the company’s block and file-based array, the HPE Alletra Storage MP B10000.

“The guidance I give my customers is to size what you can immediately forecast for the next 12 months, 18 months, whatever you have visibility of coming down the pipe of new applications,” Trembecki suggests.

“We size for that workload, and then as more demands come down the line, you can grow. You can add controllers to add performance, or you can add capacity as you need it, dynamically, immediately and transparently.

“We take away that complexity, that guesswork, which means that you don't need to over specify as an insurance policy. You don't need to power it, cool it, or find the racking for it. The ROI is good on this because you only need to invest in what you need rather than what you think you might need.”

Alongside its high performance, linear scale, and disaggregated architecture, the X10000 redefines the role of enterprise storage with its integrated Data Intelligence capabilities. Rather than simply storing data, the X10000 continuously enriches information with meaningful context as it is written, making it immediately easier for AI applications to discover, understand, and use. This dramatically reduces the time and complexity traditionally required to prepare data for AI, enabling organizations to connect trusted enterprise data to large language models with minimal development effort. By transforming raw data into AI-ready data from day one, the X10000 accelerates RAG deployments, simplifies AI pipelines, and gives organisations a foundation that is purpose-built for the next generation of AI-driven applications. Instead of managing data for AI, businesses can start generating value from it immediately.

To get the best business outcomes from AI, organizations need AI-RDSI – read more about HPE Alletra Storage MP X10000 here.