Skip to content

Commit 4c1d67f

Browse files
authored
Add cdc improved private link document. (#21929)
1 parent e3fecc8 commit 4c1d67f

File tree

4 files changed

+192
-66
lines changed

4 files changed

+192
-66
lines changed

TOC-tidb-cloud.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,7 @@
309309
- [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md)
310310
- [Set Up Self-Hosted Kafka Private Link Service in Azure](/tidb-cloud/setup-azure-self-hosted-kafka-private-link-service.md)
311311
- [Set Up Self-Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md)
312+
- [Set Up Private Endpoint for Changefeeds](/tidb-cloud/set-up-sink-private-endpoint.md)
312313
- Disaster Recovery
313314
- [Recovery Group Overview](/tidb-cloud/recovery-group-overview.md)
314315
- [Get Started](/tidb-cloud/recovery-group-get-started.md)

tidb-cloud/changefeed-sink-to-apache-kafka.md

Lines changed: 33 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -52,23 +52,9 @@ Private Connect leverages **Private Link** or **Private Service Connect** techno
5252

5353
TiDB Cloud currently supports Private Connect only for self-hosted Kafka. It does not support direct integration with MSK, Confluent Kafka, or other Kafka SaaS services. To connect to these Kafka SaaS services via Private Connect, you can deploy a [kafka-proxy](https://github.com/grepplabs/kafka-proxy) as an intermediary, effectively exposing the Kafka service as self-hosted Kafka. For a detailed example, see [Set Up Self-Hosted Kafka Private Service Connect by Kafka-proxy in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md#set-up-self-hosted-kafka-private-service-connect-by-kafka-proxy). This setup is similar across all Kafka SaaS services.
5454

55-
- If your Apache Kafka service is hosted in AWS, follow [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md) to ensure that the network connection is properly configured. After setup, provide the following information in the TiDB Cloud console to create the changefeed:
56-
57-
- The ID in Kafka Advertised Listener Pattern
58-
- The Endpoint Service Name
59-
- The Bootstrap Ports
60-
61-
- If your Apache Kafka service is hosted in Google Cloud, follow [Set Up Self-Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md) to ensure that the network connection is properly configured. After setup, provide the following information in the TiDB Cloud console to create the changefeed:
62-
63-
- The ID in Kafka Advertised Listener Pattern
64-
- The Service Attachment
65-
- The Bootstrap Ports
66-
67-
- If your Apache Kafka service is hosted in Azure, follow [Set Up Self-Hosted Kafka Private Link Service in Azure](/tidb-cloud/setup-azure-self-hosted-kafka-private-link-service.md) to ensure that the network connection is properly configured. After setup, provide the following information in the TiDB Cloud console to create the changefeed:
68-
69-
- The ID in Kafka Advertised Listener Pattern
70-
- The Alias of Private Link Service
71-
- The Bootstrap Ports
55+
- If your Apache Kafka service is hosted on AWS, follow [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md) to configure the network connection and obtain the **Bootstrap Ports** information, and then follow [Set Up Private Endpoint for Changefeeds](/tidb-cloud/set-up-sink-private-endpoint.md) to create a private endpoint.
56+
- If your Apache Kafka service is hosted on Google Cloud, follow [Set Up Self-Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md) to configure the network connection and obtain the **Bootstrap Ports** information, and then follow [Set Up Private Endpoint for Changefeeds](/tidb-cloud/set-up-sink-private-endpoint.md) to create a private endpoint.
57+
- If your Apache Kafka service is hosted on Azure, follow [Set Up Self-Hosted Kafka Private Link Service in Azure](/tidb-cloud/setup-azure-self-hosted-kafka-private-link-service.md) to configure the network connection and obtain the **Bootstrap Ports** information, and then follow [Set Up Private Endpoint for Changefeeds](/tidb-cloud/set-up-sink-private-endpoint.md) to create a private endpoint.
7258

7359
</div>
7460
<div label="VPC Peering">
@@ -139,63 +125,55 @@ The steps vary depending on the connectivity method you select.
139125
<div label="Private Link (AWS)">
140126

141127
1. In **Connectivity Method**, select **Private Link**.
142-
2. Authorize the [AWS Principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#principal-accounts) of TiDB Cloud to create an endpoint for your endpoint service. The AWS Principal is provided in the tip on the web page.
143-
3. Make sure you select the same **Number of AZs** and **AZ IDs of Kafka Deployment**, and fill the same unique ID in **Kafka Advertised Listener Pattern** when you [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md) in the **Network** section.
144-
4. Fill in the **Endpoint Service Name** which is configured in [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md).
145-
5. Fill in the **Bootstrap Ports**. It is recommended that you set at least one port for one AZ. You can use commas `,` to separate multiple ports.
146-
6. Select an **Authentication** option according to your Kafka authentication configuration.
128+
2. In **Private Endpoint**, select the private endpoint that you created in the [Network](#network) section. Make sure the AZs of the private endpoint match the AZs of the Kafka deployment.
129+
3. Fill in the **Bootstrap Ports** that you obtained from the [Network](#network) section. It is recommended that you set at least one port for one AZ. You can use commas `,` to separate multiple ports.
130+
4. Select an **Authentication** option according to your Kafka authentication configuration.
147131

148132
- If your Kafka does not require authentication, keep the default option **Disable**.
149133
- If your Kafka requires authentication, select the corresponding authentication type, and then fill in the **user name** and **password** of your Kafka account for authentication.
150-
151-
7. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
152-
8. Select a **Compression** type for the data in this changefeed.
153-
9. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
154-
10. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
155-
11. TiDB Cloud creates the endpoint for **Private Link**, which might take several minutes.
156-
12. Once the endpoint is created, log in to your cloud provider console and accept the connection request.
157-
13. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
134+
5. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
135+
6. Select a **Compression** type for the data in this changefeed.
136+
7. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
137+
8. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
138+
9. TiDB Cloud creates the endpoint for **Private Link**, which might take several minutes.
139+
10. Once the endpoint is created, log in to your cloud provider console and accept the connection request.
140+
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
158141

159142
</div>
160143
<div label="Private Service Connect (Google Cloud)">
161144

162145
1. In **Connectivity Method**, select **Private Service Connect**.
163-
2. Ensure that you fill in the same unique ID in **Kafka Advertised Listener Pattern** when you [Set Up Self-Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md) in the **Network** section.
164-
3. Fill in the **Service Attachment** that you have configured in [Setup Self Hosted Kafka Private Service Connect in Google Cloud](/tidb-cloud/setup-self-hosted-kafka-private-service-connect.md)
165-
4. Fill in the **Bootstrap Ports**. It is recommended that you provide more than one port. You can use commas `,` to separate multiple ports.
166-
5. Select an **Authentication** option according to your Kafka authentication configuration.
146+
2. In **Private Endpoint**, select the private endpoint that you created in the [Network](#network) section.
147+
3. Fill in the **Bootstrap Ports** that you obtained from the [Network](#network) section. It is recommended that you provide more than one port. You can use commas `,` to separate multiple ports.
148+
4. Select an **Authentication** option according to your Kafka authentication configuration.
167149

168150
- If your Kafka does not require authentication, keep the default option **Disable**.
169151
- If your Kafka requires authentication, select the corresponding authentication type, and then fill in the **user name** and **password** of your Kafka account for authentication.
170-
171-
6. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
172-
7. Select a **Compression** type for the data in this changefeed.
173-
8. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
174-
9. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
175-
10. TiDB Cloud creates the endpoint for **Private Service Connect**, which might take several minutes.
176-
11. Once the endpoint is created, log in to your cloud provider console and accept the connection request.
177-
12. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
152+
5. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
153+
6. Select a **Compression** type for the data in this changefeed.
154+
7. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
155+
8. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
156+
9. TiDB Cloud creates the endpoint for **Private Service Connect**, which might take several minutes.
157+
10. Once the endpoint is created, log in to your cloud provider console and accept the connection request.
158+
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
178159

179160
</div>
180161
<div label="Private Link (Azure)">
181162

182163
1. In **Connectivity Method**, select **Private Link**.
183-
2. Authorize the Azure subscription of TiDB Cloud or allow anyone with your alias to access your Private Link service before creating the changefeed. The Azure subscription is provided in the **Reminders before proceeding** tip on the web page. For more information about the visibility of Private Link service, see [Control service exposure](https://learn.microsoft.com/en-us/azure/private-link/private-link-service-overview#control-service-exposure) in Azure documentation.
184-
3. Make sure you fill in the same unique ID in **Kafka Advertised Listener Pattern** when you [Set Up Self-Hosted Kafka Private Link Service in Azure](/tidb-cloud/setup-azure-self-hosted-kafka-private-link-service.md) in the **Network** section.
185-
4. Fill in the **Alias of Private Link Service** which is configured in [Set Up Self-Hosted Kafka Private Link Service in Azure](/tidb-cloud/setup-azure-self-hosted-kafka-private-link-service.md).
186-
5. Fill in the **Bootstrap Ports**. It is recommended that you set at least one port for one AZ. You can use commas `,` to separate multiple ports.
187-
6. Select an **Authentication** option according to your Kafka authentication configuration.
164+
2. In **Private Endpoint**, select the private endpoint that you created in the [Network](#network) section.
165+
3. Fill in the **Bootstrap Ports** that you obtained in the [Network](#network) section. It is recommended that you set at least one port for one AZ. You can use commas `,` to separate multiple ports.
166+
4. Select an **Authentication** option according to your Kafka authentication configuration.
188167

189168
- If your Kafka does not require authentication, keep the default option **Disable**.
190169
- If your Kafka requires authentication, select the corresponding authentication type, and then fill in the **user name** and **password** of your Kafka account for authentication.
191-
192-
7. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
193-
8. Select a **Compression** type for the data in this changefeed.
194-
9. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
195-
10. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
196-
11. TiDB Cloud creates the endpoint for **Private Link**, which might take several minutes.
197-
12. Once the endpoint is created, log in to the [Azure portal](https://portal.azure.com/) and accept the connection request.
198-
13. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
170+
5. Select your **Kafka Version**. If you do not know which one to use, use **Kafka v2**.
171+
6. Select a **Compression** type for the data in this changefeed.
172+
7. Enable the **TLS Encryption** option if your Kafka has enabled TLS encryption and you want to use TLS encryption for the Kafka connection.
173+
8. Click **Next** to test the network connection. If the test succeeds, you will be directed to the next page.
174+
9. TiDB Cloud creates the endpoint for **Private Link**, which might take several minutes.
175+
10. Once the endpoint is created, log in to the [Azure portal](https://portal.azure.com/) and accept the connection request.
176+
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
199177

200178
</div>
201179
</SimpleTab>

0 commit comments

Comments
 (0)