Introducing Schema Drift Detection in Monad

TL;DR:

Monad Schema Drift Detection identifies new fields and data type changes throughout a pipeline, shows where each change first appeared, and sends alerts into the workflows teams already use. This gives teams a chance to respond before drift results in missed detections, rejected records, incomplete investigations, or broken reporting.

Audit logs, findings, alerts, and other security events change shape over time. Vendors add fields, change data types, and update event structures. Changes can also happen as teams adjust parsing, mappings, and transforms inside their own pipelines.

Some changes are documented. Others are not noticed until something downstream starts behaving differently.

Schema drift is what happens when the structure you expect no longer matches the structure you receive.

The catch is that data can keep flowing and the pipeline can stay green even as the assumptions built around that data stop holding.

What makes schema drift especially painful is how many of its failure modes stay silent:

• Detections, filters, correlations, and routing logic can stop matching without producing an error and while data is still ingested.
• Schema-strict destinations (i.e., Snowflake) can reject otherwise valid records while the source continues sending data normally.
• Investigations, reports, and audits can operate on incomplete data without an obvious warning.
• Troubleshooting becomes harder once the change has passed through several pipeline stages and affected multiple downstream systems.

Monad’s Schema Drift Detection learns the expected structure of security data throughout a pipeline, detects new fields and type changes, and shows what changed and where it first appeared.

How schema changes affect downstream logic

Consider a source that has always sent a process ID as a number:

{
  "event": {
    "ProcessId": 4824
  }
}

After an upstream change, the same field starts arriving as a string:

{
  "event": {
    "ProcessId": "4824"
  }
}

At a glance, the difference looks minor. To a type-sensitive query, a number and a string are not equivalent.

Downstream logic written against the numeric value can stop matching:

WHERE event.ProcessId = 4824


The event still arrives. The JSON remains valid. The pipeline may report no error, but the detection no longer finds what it was written to find.

Because nothing necessarily crashes, the issue may not surface until a detection validation exercise, an audit, or an actual incident.

Find where the schema changed

Inside Monad, every ingested event is converted to JSON as it moves through a pipeline, regardless of its original format.

Schema Drift Detection can monitor different stages of that pipeline. Each monitored point learns the structure of the data it receives independently, allowing different stages to maintain different valid schemas.

Each point begins in Learning Mode to establish a reference schema, then moves into Detection Mode. When Monad detects a new field or an existing field changing type, it shows the affected field path, the learned and observed types, and where the difference first appeared.

The comparison uses schema structure rather than event values.

Alert the team or accept the change

Monad automatically creates and manages the Schema Drift Detection alert rule. Detected changes appear in the platform and can be sent out to Slack, PagerDuty, incident.io, a SIEM, webhooks, and other supported destinations.

Not every change requires action. Some schema changes introduce fields that provide security value. If a new field or type is expected, Reset Schema History returns only that monitored point to Learning Mode so Monad can establish a new reference schema.

Schema Drift Detection is now available

Read the Schema Drift Detection documentation for more on learning, detection, alerting, and schema comparison.

Or test Schema Drift Detection on your own data sources with a free trial or demo.

‍