|
| 1 | +--- |
| 2 | +title: Monitor DynamoDB with Amazon Kinesis and OpenObserve |
| 3 | +description: Learn how to capture real-time DynamoDB item-level changes using Amazon Kinesis Data Streams and Firehose, and monitor them in OpenObserve. |
| 4 | +--- |
| 5 | + |
| 6 | +# DynamoDB Integration with OpenObserve |
| 7 | + |
| 8 | +This guide provides step-by-step instructions to capture **DynamoDB table changes** with **Amazon Kinesis** and forward them to OpenObserve for real-time monitoring and analysis. |
| 9 | + |
| 10 | +## Overview |
| 11 | + |
| 12 | +Applications that rely on **Amazon DynamoDB** often require real-time insights into table changes. By enabling **DynamoDB → Kinesis Data Streams → Kinesis Firehose → OpenObserve**, you can ingest and visualize these changes instantly. |
| 13 | + |
| 14 | +## Steps to Integrate |
| 15 | + |
| 16 | +??? "Prerequisites" |
| 17 | + - AWS account with permissions for **DynamoDB**, **Kinesis Data Streams**, and **Kinesis Firehose** |
| 18 | + - OpenObserve account ([Cloud](https://cloud.openobserve.ai/web/) or [Self-Hosted](../../../quickstart/#self-hosted-installation)) |
| 19 | + - Dynamo DB Table |
| 20 | + |
| 21 | +??? "Step 1: Create Kinesis Data Stream" |
| 22 | + |
| 23 | + 1. Open the [Kinesis Console](https://console.aws.amazon.com/kinesis). |
| 24 | + 2. Follow the instructions and create a new data stream. |
| 25 | +  |
| 26 | + |
| 27 | + |
| 28 | +??? "Step 2: Enable Kinesis Data Stream on DynamoDB" |
| 29 | + |
| 30 | + 1. Go to the **DynamoDB Console → Table → Exports and Streams tab**. |
| 31 | + 2. Enable **Kinesis Data Stream** and select your Kinesis data stream. |
| 32 | + |
| 33 | +  |
| 34 | + |
| 35 | + |
| 36 | +??? "Step 3: Create Kinesis Data Firehose Delivery Stream" |
| 37 | + |
| 38 | + 1. Open the [Kinesis Firehose Console](https://console.aws.amazon.com/firehose). |
| 39 | + 2. Create a new **Delivery Stream**. |
| 40 | + 3. Select **Source** → `Kinesis Data Stream` → choose the Kinesis data stream. |
| 41 | + 4. Select **Destination** → `HTTP Endpoint`. |
| 42 | + - **Endpoint URL**: your OpenObserve Firehose endpoint |
| 43 | + - **Authentication**: Access Key (from OpenObserve → Ingestions → Amazon Kinesis Firehose) |
| 44 | + 5. (Optional) Configure an **S3 bucket** for backup of failed records. |
| 45 | + 6. Review and **Create Firehose Stream**. |
| 46 | + |
| 47 | +  |
| 48 | + |
| 49 | +??? "Step 4: Monitor DynamoDB Changes in OpenObserve" |
| 50 | + |
| 51 | + Once the Firehose is active, DynamoDB changes begin flowing to OpenObserve. Access at Logs -> Stream name |
| 52 | +  |
| 53 | + |
| 54 | + |
| 55 | +??? "Troubleshooting" |
| 56 | + |
| 57 | + **No logs appearing in OpenObserve?** |
| 58 | + |
| 59 | + |
| 60 | + - Firehose: |
| 61 | + - Verify the delivery stream is active and error-free. |
| 62 | + - Review the Monitoring tab and look for failed deliveries. |
| 63 | + |
| 64 | + - OpenObserve: |
| 65 | + - Confirm the HTTP ingestion URL and access key are correct. |
| 66 | + - In the Logs view, select the correct stream and expand the time range to view recent data. |
0 commit comments