How to Reduce AWS CloudTrail Volume Without Losing Security Value

TL;DR

• CloudTrail records often repeat account, identity, and request metadata across multiple fields.
• Removing carefully selected duplicated and low-utility fields can reduce record size by approximately 20–24%.
• Field removal should be validated against your detections, investigations, compliance requirements, and actual event samples.
• The goal is not simply to ingest less. It is to send high-value, contextual data to the SIEM and route the rest to more cost-effective destinations.

The CloudTrail Noise Problem

AWS CloudTrail is one of the most valuable (and mostly free) sources of cloud security data, especially when monitoring an AWS workload, application, or environment. However, it’s also pretty noisy since it includes absolutely EVERYTHING that happens in that AWS environment.

If it’s an organization trail that includes many accounts (AWS’s preferred architecture), it can capture management activity across dozens or hundreds of AWS accounts, which becomes very noisy very quickly. And noise isn't free just because the events themselves are. It still costs you ingest, storage, and query time. AWS management events are free and data events have a cost associated with them, but even if all you use are the management events, there is a lot of value to those logs… As long as they don’t run your analysis/SIEM/storage costs up too high (assuming you’re not using GuardDuty as your SIEM).

Cutting Duplicate Fields

So, what can you do about it? Pluck the low-hanging fruit first, since the fastest way to cut cost is to stop paying to ingest and store what you already have twice. CloudTrail logs contain a LOT of fields, and the values held in those fields are sometimes duplicated. There are edge cases where every single one of those fields are different, but most security teams have to take an 80/20 approach to log analysis, which means that extreme edge cases are sometimes ok to skip. As long as you retain the core information, these duplicated data values can be dropped. Which ones are those? Primarily:

• recipientAccountId

  • Duplicated by userIdentity.accountId
• userIdentity.sessionContext.sessionIssuer.accountId

  • Duplicated or derivable from recipientAccountId, userIdentity.accountId, and embedded in every ARN
• userIdentity.sessionContext.sessionIssuer.userName

  • Embedded in sessionIssuer.arn and userIdentity.arn
• additionalEventData.RequestDetails.awsServingRegion

  • Duplicated by awsRegion
• userIdentity.sessionContext.sessionIssuer.principalId

  • Prefix of userIdentity.principalId
• responseElements.credentials.accessKeyId

  • Duplicated by userIdentity.accessKeyId

The account ID and user name frequently, if not always, show up in multiple places in each CloudTrail record, so understanding the source and destination accounts is trivial without that data being duplicated so many times. Removing these fields can shrink records without removing the identity, account, region, and credential context analysts typically need. But validate the assumption across each event family you collect; a field that is redundant in an AssumeRole event may not be redundant everywhere else.

Cutting Low-Value Fields

What else? Now that we’ve reduced record size by dropping duplicate data keys/values, we can move on to dropping those fields that have low intrinsic value for security purposes. CloudTrail also includes operational and transport metadata that may help AWS troubleshoot a request but rarely contributes to a detection or analyst decision. Unlike duplicated data, they may contain unique information — even if your team rarely uses it. This means that while every field listed here has some value, it’s usually not enough to justify the spend from ingesting into a SIEM. Routing that information to a data lake instead should be strongly considered if forensics or threat-hunting are capabilities your organization can perform.

Short-lived credential metadata

• responseElements.credentials.sessionToken
• responseElements.credentials.expiration

These fields provide metadata about temporary credentials. The token itself provides almost no value outside of tracking the “movements” of an actor’s actions, while expiration has limited value depending on how the team investigates credential misuse.

AWS request-correlation metadata

• requestID
• additionalEventData.ExtendedRequestId
• additionalEventData.x-amz-id-2
• sharedEventID

These identifiers primarily support AWS-side request correlation, troubleshooting, or deduplication. Keep them if your incident-response process regularly escalates events to AWS Support or correlates activity across related records.

Transport and schema metadata

• tlsDetails.clientProvidedHostHeader
• tlsDetails.cipherSuite
• tlsDetails.tlsVersion
• eventVersion

These fields often have low cardinality in a well-controlled environment. Instead of dropping them unconditionally, consider retaining or alerting on changes upstream while excluding repetitive values from higher-cost storage.

Environment-specific tags

requestParameters.incomingTransitiveTags.*

Transitive tags are highly environment-specific. They may be noise in one organization and critical authorization context in another. Review how tags are used in IAM policies and investigations before removing them.

Errors

• errorCode
• errorMessage

Often those have little to no relevance to security operations since they are more about applications, services, and infrastructure malfunctioning, but sometimes they can provide key insight from a forensic perspective.

What This Actually Saves You

The combination of dropping keys (and their associated values) from a record that are either duplicated elsewhere in the record or provide little to no value for security detection purposes results in an approximately 20-24% reduction in individual record size. That can be a pretty significant percentage if you’re dealing with millions of events per day, which is fairly common in AWS.

At that volume, a 20% cut in record size is a 20% cut in what you're paying to move, store, and search that data. Obviously, not every environment is going to have the same low-value or duplicated fields - it all depends on your workload, normal usage, and how services are used. However, this should be a good place to start.

Routing What's Left

And if dropping low-utility and duplicated fields is combined with normalization (which tends to increase record size) and/or contextual enrichment, the size “saved” by pruning the records can be used to store information that allows for intelligent routing of the data to appropriate destinations. Not every byte that comes in from a data source needs to be routed into a SIEM - one of the most expensive destinations in terms of ingest, storage, and enrichment.

A better, more cost-effective, data pipeline architecture is to route the highest-value data (things that a detection can actually fire on) with enough context that an analyst can take action immediately to the highest-cost destination (usually a SIEM), and to route correlated data to something like a data lake, with its associated lower ingest, storage, and processing costs. Finally, any data that needs to be retained due to regulatory, legal, or audit requirements can be routed to the least expensive storage options; something that is long-term, costs next to nothing, and doesn’t matter if the data sits there until and unless it’s needed.

In conclusion, when dealing with any log source (this post is specifically about AWS CloudTrail, but it applies to any log source), the best data pipeline architecture is one that strips out unnecessary bytes from high-cost destinations, retains the necessary context for decision-making, and routes the rest to appropriately-priced destinations based on the importance or immediacy of that data itself. All of that can be contained in logical rules, which is what a platform like Monad is built to do: cut the noise, keep the signal, and put the budget you saved toward ingesting more data instead of storing less of it.

‍