Design : Replicator
The replicator component contains the core logic for the replication of DynamoDB table data across regions. It consists of two lambda functions.
Replicator Function - [src]
The replicator function handles the actual replication of table data between regions. An event source mapping is added to the lambda function for each source table, connecting the function to the table's stream. In order for a DynamoDB table to be replicated, it must have streams enabled with a view type of NEW_AND_OLD_IMAGES. The lambda function reads events from the streams in batches of 25 and writes them to the replica table. If any requests are throttled, they are retried with an exponential backoff. If any records cannot be process before the timeout, or a non-retryable error is encountered, the function will fail to ensure that no records are missed or skipped.
The following replication metrics are written to the logs:
- MinutesBehindRecord - The difference (in minutes) between the stream record's ApproximateCreationDateTime and the time of replication.
- RecordsProcessed - The number of records received from the stream function
- RecordsWritten - The number of records written to the replica table
- UnknownEventTypes - The number of unexpected event types encountered (should never be more than 0, metric is meant to monitor for breaking changes to the DynamoDB api in the future)
Metrics Function - [src]
The metrics function subscribes to the replicator function's logs, extracting metric data and posting it to CloudWatch.
Metrics are extracted from the logs using the following filter pattern:
[timestamp, request_id, metric=METRIC, table, level, name, value]