Amazon OpenSearch Service’s New Log Analytics Engine: How to Cut Log Storage Costs Up to 70% Without Dropping Data

Every scale-up eventually meets the same wall. The application is growing, the traffic is growing, and quietly the log bill is growing faster than either. Observability, the thing that is supposed to help you find problems, becomes a problem of its own on the invoice. For most engineering leaders this shows up as an uncomfortable conversation about which logs to keep and which to throw away to keep the storage line under control. On the first of July 2026, AWS shipped something that changes the maths behind that conversation. Amazon OpenSearch Service now has a new engine purpose built for log analytics, and it is aimed squarely at the cost of retaining the flood of telemetry that modern and AI heavy applications produce.

The short version is this. The new OpenSearch Service engine delivers up to 4x better price performance, up to 70 percent lower storage costs, up to 2x higher ingestion throughput, and up to 2x faster analytical queries compared with the general purpose engine, and it does this at no additional charge on the same instance and storage pricing. In practical terms you can retain up to 3x more data for the same money, which means the choice between keeping your logs and controlling your bill gets a lot less painful. If your business runs anything meaningful on AWS and you have ever exported or dropped logs purely to save money, this is worth understanding properly.

Why Log Costs Became a Board-Level Problem

Observability spending stopped being a rounding error a while ago. Grafana Labs’ 2025 Observability Survey found that observability now accounts for an average of 17 percent of total compute infrastructure spend, with the median sitting at 10 percent. That is a real slice of the cloud bill, and it is not shrinking. LogicMonitor’s 2026 outlook reports that 96 percent of IT leaders expect observability budgets to hold steady or grow, and 62 percent expect them to increase even while broader IT budgets are being cut. When one category resists cost pressure that everything else is subject to, it earns attention from people who do not usually read a CloudWatch dashboard.

The reason the numbers keep climbing is volume, and the newest driver of volume is AI. In a survey of enterprise log management practices cited by InfoWorld, Dynatrace found that AI workloads drove a 93 percent increase in log volume over the previous year. Faced with that, organisations excluded an average of 86 percent of their log data just to manage cost and system capacity. Read that again. Most teams are now throwing away the overwhelming majority of their telemetry, not because it is worthless, but because keeping it costs too much. That is exactly the kind of forced trade off that comes back to bite you during an incident, when the log line you need turns out to be one of the ones you dropped.

What the New OpenSearch Engine Actually Does

The design idea behind the new engine is that log analytics is not the same workload as general purpose search, so it should not pay for general purpose search. Logs are mostly written once, queried in aggregate, and occasionally searched in full when something breaks. The new engine stores data in a columnar format using Apache Parquet, which is far more efficient for the aggregation queries that dominate log analysis, and it still writes an inverted index for the fields you mark as searchable so you keep full text search for incident investigation. You get the cheap aggregations without losing the ability to grep through a rough week.

AWS published its benchmark figures rather than leaving the claims abstract, which makes them worth quoting. In testing described on the AWS Big Data blog, the team ingested three data sets totalling 24.4 billion documents and 9.5 terabytes of raw JSON. The optimised engine sustained 1.78 million documents per second at matched concurrency, roughly twice the throughput of the Apache Lucene baseline, while using less CPU. It stored the same data at 5x compression versus raw JSON, an 80 percent storage saving, and on unfiltered aggregations such as counting log volume by host it ran up to 70 times faster because the columnar layout only reads the columns a query touches. These are internal AWS benchmarks rather than independent tests, so treat the exact multiples as directional, but the architecture behind them is sound and the direction is clearly down on cost.

The Numbers in One Place

Here is the picture the sources paint when you put the market pressure and the AWS figures side by side.

The through line is simple. Costs are rising because volume is rising, teams are coping by deleting data, and the new engine attacks the storage cost that forced the deletions in the first place.

Who Should Care, and Who Can Wait

If you run your own OpenSearch Service domains for logs, this is directly relevant and you should evaluate it. Teams with high ingestion, painful retention windows, or a habit of dropping logs to control spend have the most to gain, because a 70 percent storage reduction turns a two week retention policy into something closer to six weeks at the same price. That extra history is often the difference between diagnosing an intermittent issue and guessing at it.

If you are on a third party SaaS observability platform rather than OpenSearch, the immediate lever is different, but the pressure is the same and worth naming. The Grafana survey shows most organisations run two or three observability tools, and consolidation is a live theme precisely because fragmentation drives cost. A cheaper OpenSearch based log tier can become the destination for high volume, low value logs that do not justify premium SaaS pricing, while your critical signals stay where your team already works. The point is not to rip everything out. It is to stop paying premium rates to store logs that are, by the industry’s own admission, 86 percent noise.

One caveat before anyone reorganises their logging stack around a launch that is days old. The benchmarks are AWS’s own, the engine is new, and the sensible path is a measured pilot on a representative slice of your log traffic rather than a wholesale migration. Availability is currently limited to 12 regions, including London, Ireland, Frankfurt, and Spain for European teams, so confirm your region is covered before you plan around it.

What to Do About It

Start by finding out what you are actually spending on logs, because most teams cannot answer that to the nearest thousand pounds. Break the OpenSearch or observability bill into ingestion, storage, and query, and identify the retention policies that exist only to control cost rather than to meet a real requirement. Then take a representative sample of your highest volume log sources and run them through the new engine in a non production domain to see whether the storage and price performance gains hold for your data shape, since log characteristics vary and your mileage will differ from a vendor benchmark. If the numbers hold, the win is not just a smaller bill. It is the ability to keep the data you were about to throw away, which is worth far more the next time something breaks at two in the morning.

Getting this right is part cost engineering and part architecture, and it is easy to either leave money on the table or migrate too aggressively and regret it. HAZERCLOUD helps UK, US, and European businesses run leaner, better instrumented infrastructure on AWS, from log and observability cost reviews to full Well Architected assessments. If your cloud bill is climbing faster than your traffic, or you are dropping telemetry you would rather keep, book a free consultation and migration assessment at https://hazercloud.com/contact/ and we will help you find the savings that are actually safe to take.

Jobin Joseph

View all posts by Jobin Joseph

hazercloud.com