Amazon Web Services (AWS) has announced significant performance improvements for its Amazon S3 Vectors service in a bid to compensate for surging enterprise AI workload demands.
New updates for the cloud object storage service, first unveiled in July this year, mean S3 Vectors now scale up to two billion vectors per index, which the hyperscaler noted is a 40-times increase compared to preview capacity.
The service also supports up to 20 trillion vectors per bucket, AWS revealed, and will offer users 2-3x faster query performance.
The update aims to accommodate surging data volumes amidst continued enterprise AI adoption, according to AWS. The S3 service already stores more than 500 trillion objects and hundreds of exabytes of data, AWS noted.
To meet this growing demand, the hyperscaler plans to increase the maximum S3 object size to 50TB, marking a ten-times increase. This, the hyperscaler said, will enable customers to "store massive data files like high-resolution videos, seismic data, and AI training datasets as single objects in their original form".
Amazon S3 Vectors
When AWS announced the launch of Amazon S3 Vectors in July, it hailed the service as the first "purpose-built durable vector storage solution" aimed at improving affordability for enterprises with large-scale data storage requirements.
The hyperscaler claims the service can reduce the cost of uploading, storing, and querying vectors by around 90%. These cost savings are crucial given the intense enterprise focus on generative AI over the last three years.
Vector search is a technique used in generative AI applications to identify similarities between specific data points. According to AWS, vectors are a "numerical representation of unstructured data created from embedding models".
"You use embedding models to generate vector embeddings of your data and store them in S3 Vectors to perform semantic searches," the company explained in a July announcement statement.
In launching S3 Vectors, AWS aims to bring these capabilities to customer data, integrating closely with Amazon Bedrock Knowledge Bases and the Amazon OpenSearch service.
The goal here is to make it easier to build AI agents, RAG systems, and semantic search applications that understand both context and intent; crucial factors for enterprises developing customer-facing applications.
According to the hyperscaler, a host of customers, including BMW Group, Twilio, and Qlik are using S3 Vectors to "accelerate AI search and power recommendation systems at scale".
MORE FROM ITPRO
-
HPE and AMD reveal first Helios offeringNews The rack-scale AI architecture also features hardware from HPE Juniper Networking hardware and Broadcom
-
AWS says new ‘frontier agents’ will transform software developmentNews A new class of AI agents promises days of autonomous work and added safety checks
