Amazon OpenSearch update targets performance boosts and lower costs – and at no extra charge for users
Surging data volumes have prompted an overhaul of Amazon’s OpenSearch service
Amazon Web Services (AWS) has announced a series of changes to Amazon OpenSearch aimed at streamlining data log analysis.
A new purpose-built log analytics engine for the service will provide users with marked performance gains and up to 70% lower storage costs, according to the firm.
OpenSearch allows users to search, analyze, and visualize data in real-time, and is typically used for application monitoring, observability, and AI/ML search applications.
The hyperscaler said the update comes in direct response to surging volumes of log data handled by enterprises in 2026, largely due to AI.
Research from AWS shows that log volumes have grown between 30-40% year over year, which is stretching budgets and creating bottlenecks.
Amazon OpenSearch overhaul
As part of the update, Amazon revealed OpenSearch will now store data in the Apache Parquet format. This is a column-oriented data file format designed specifically for more efficient data storage and retrieval purposes.
The service previously relied on inverted indexing. Amazon noted the shift will deliver significant efficiency gains and faster search and retrieval capabilities.
“Apache Calcite parses and optimizes each query, then routes operations to the engine best suited to execute them: Apache DataFusion for analytical operations on columnar data, or Lucene for search predicates,” the company explained in a blog post.
“The two hand off mid-query, so a single query can search log content and aggregate the results without additional roundtrips.”
Amazon noted that OpenSearch now also supports the Piped Processing Language (PPL) and SQL query languages, both of which “execute natively” through the platform.
Amazon eyes optimized log analytics
From a performance perspective, Amazon noted users can expect significant improvements alongside lower costs.
OpenSearch now boasts up to four-times price improvements compared to the existing engine, based on internal benchmarks.
Users can also capitalize on two-times faster analytics queries due to the fact the engine processes data in columnar batches, even across larger data sets. A two-fold improvement on ingestion rates also aims to reduce bottlenecks.
“The optimized engine sustained 1.78 million documents per second at matched concurrency, approximately 2x the throughput of the Lucene baseline, while consuming less CPU,” the hyperscaler explained.
“Both domains ran with zero write rejections. For teams ingesting terabytes per day, the throughput advantage translates to fewer nodes for the same volume, or longer retention on the same infrastructure.”
How to access Amazon OpenSearch
The new log analytics engine is available now for enterprises in AWS regions where OpenSearch Optimized instances are supported, the company noted. Similarly, there will be no additional premium for the optimized engine.
It’s worth noting that the new engine is a setting selected at creation time, meaning users can’t add this to an existing domain or enable it within individual indices or fields within a general-purpose domain.
“To adopt the optimized engine, create a new domain and migrate your ingestion pipelines into it,” the company explained.
“Create a new domain in the Amazon OpenSearch Service console and select Observability as your use case. The optimized engine is enabled by default.”
The in-built console will provide users with a side-by-side comparison of capabilities based on individual needs.
“After your domain is ready, ingest JSON documents through the same Bulk API and client libraries you use today. No changes to your ingestion pipelines or application code are required.”
FOLLOW US ON SOCIAL MEDIA
Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.
You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.
-
Cyber experts say they've identified the first case of ‘agentic ransomware’News While the JadePuffer ransomware has alarm bells ringing, it still needed a human in the loop
-
Yorkshire Building Society touts customer service gains with AI agentsNews The firm said that agents Penelope, Sam, and Alf are helping improve customer service, while adhering to high levels of regulation
-
AWS says cloud market gatekeeper designation risks ‘deterring European investment and innovation’ as EU regulators plot competition crackdownNews Gatekeeper designation under the legislation would force AWS and Microsoft to make concessions
-
Big tech earnings: Hyperscaler growth rates impress, but concerns remain over surging infrastructure capexNews Concerns remain over big tech infrastructure spending despite positive earnings reports
-
The first hurdle is the hardest in generative AI adoption – and businesses keep fallingAnalysis AWS’ UK chief said AI advances “feel like magic” at its recent London summit, but many firms are facing the reality of sluggish gains
-
Microsoft, Amazon cloud practice changes spark mixed industry reactionNews Concerns have been raised over the voluntary actions agreed to by Microsoft and Amazon
-
January rundown: Amazon layoffs and the return of XPSITPro Podcast This year's tech layoffs have just begun, as Amazon sheds 16,000 workers in one go
-
Nationwide forges closer ties with AWS in cloud transformation pushNews The building society is “consolidating and modernizing” cloud infrastructure and focusing heavily on internal skills development
-
Are hyperscalers backing out of Net Zero?ITPro Podcast Expanding data center construction and demand for high-energy workloads are pushing hyperscalers off course on carbon
-
Startups get seal of approval from CrowdStrike, AWS, and NvidiaNews 35 startups are promised mentorship, technical expertise, go-to-market support, and ecosystem visibility
