Two-Phase Mutation

We recommend that users deploy two-phase mutation data pipelines for resiliency against infrastructure and configuration errors.

The two-phase mutation design deployed in AWS and used by Brex's Security team is similar to this:

This design allows for:

  • concurrent routing of data to multiple downstream systems
  • long-term retention of raw and processed data
  • recoverability in case of configuration errors
    • Kinesis Data Streams will retry Lambda until the Lambda succeeds or data expires from the stream