You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/current/v25.2/cockroachdb-feature-availability.md
-4Lines changed: 0 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -87,10 +87,6 @@ The [`VECTOR`]({% link {{ page.version.version }}/vector.md %}) data type stores
87
87
88
88
[Organizing CockroachDB {{ site.data.products.cloud }} clusters using folders]({% link cockroachcloud/folders.md %}) is in preview. Folders allow you to organize and manage access to your clusters according to your organization's requirements. For example, you can create top-level folders for each business unit in your organization, and within those folders, organize clusters by geographic location and then by level of maturity, such as production, staging, and testing.
89
89
90
-
### Logical data replication (LDR) for CockroachDB {{ site.data.products.core }}
91
-
92
-
**Logical data replication (LDR)** continuously replicates tables between active CockroachDB clusters. Both source and destination cluster can receive application reads and writes, with LDR enabling bidirectional replication for eventual consistency in the replicating tables. The active-active setup between clusters can provide protection against cluster, datacenter, or region failure while still achieving single-region low latency reads and writes in the individual CockroachDB clusters. Setting up LDR between a source and destination CockroachDB {{ site.data.products.core }} cluster is in preview.
93
-
94
90
### Read on standby cluster in physical cluster replication (PCR) for CockroachDB {{ site.data.products.core }}
95
91
96
92
The [`READ VIRTUAL CLUSTER`]({% link {{ page.version.version }}/create-virtual-cluster.md %}#options) option allows you to set up a PCR stream that also creates a read-only virtual cluster on the standby cluster. You can create a PCR job as per the [Set Up Physical Cluster Replication]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}) guide and then add the option to the [`CREATE VIRTUAL CLUSTER`]({% link {{ page.version.version }}/create-virtual-cluster.md %}) statement.
Copy file name to clipboardExpand all lines: src/current/v25.2/create-logical-replication-stream.md
-10Lines changed: 0 additions & 10 deletions
Original file line number
Diff line number
Diff line change
@@ -5,8 +5,6 @@ toc: true
5
5
---
6
6
7
7
{{site.data.alerts.callout_info}}
8
-
{% include feature-phases/preview.md %}
9
-
10
8
Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.
11
9
{{site.data.alerts.end}}
12
10
@@ -66,14 +64,6 @@ Option | Description
66
64
`cursor` | Emits any changes after the specified timestamp. LDR will not perform an initial backfill with the `cursor` option, it will stream any changes after the specified timestamp. The LDR job will encounter an error if you specify a `cursor` timestamp that is before the configured [garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) window for that table. **Warning:** Apply the `cursor` option carefully to LDR streams. Using a timestamp in error could cause data loss.
67
65
<aid="discard-ttl-deletes-option"></a>`discard` | ([**Unidirectional LDR only**]({% link {{ page.version.version }}/logical-data-replication-overview.md %}#use-cases)) Ignore [TTL deletes]({% link {{ page.version.version }}/row-level-ttl.md %}) in an LDR stream with `discard = ttl-deletes`. **Note**: To ignore row-level TTL deletes in an LDR stream, it is necessary to set the [`ttl_disable_changefeed_replication`]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters) storage parameter on the source table. Refer to the [Ignore row-level TTL deletes](#ignore-row-level-ttl-deletes) example.
68
66
`label` | Tracks LDR metrics at the job level. Add a user-specified string with `label`. Refer to [Metrics labels]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}#metrics-labels).
69
-
`mode` | Determines how LDR replicates the data to the destination cluster. Possible values: `immediate`, `validated`. For more details, refer to [LDR modes](#ldr-modes).
70
-
71
-
## LDR modes
72
-
73
-
_Modes_ determine how LDR replicates the data to the destination cluster. There are two modes:
74
-
75
-
-`immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %}
76
-
-`validated`: {% include {{ page.version.version }}/ldr/validated-description.md %}
Copy file name to clipboardExpand all lines: src/current/v25.2/create-logically-replicated.md
+1-11Lines changed: 1 addition & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -5,8 +5,6 @@ toc: true
5
5
---
6
6
7
7
{{site.data.alerts.callout_info}}
8
-
{% include feature-phases/preview.md %}
9
-
10
8
Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.
11
9
{{site.data.alerts.end}}
12
10
@@ -81,14 +79,6 @@ Option | Description
81
79
-------+------------
82
80
`bidirectional on`/`unidirectional` | (**Required**) Specifies whether the LDR stream will be unidirectional or bidirectional. With `bidirectional on` specified, LDR will set up two LDR streams between the clusters. Refer to the examples for [unidirectional](#unidirectional) and [bidirectional](#bidirectional).
83
81
`label` | Tracks LDR metrics at the job level. Add a user-specified string with `label`. For more details, refer to [Metrics labels]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}#metrics-labels).
84
-
`mode` | Determines how LDR replicates the data to the destination cluster. Possible values: `immediate`, `validated`. For more details, refer to [LDR modes](#ldr-modes).
85
-
86
-
## LDR modes
87
-
88
-
_Modes_ determine how LDR replicates the data to the destination cluster. There are two modes:
89
-
90
-
-`immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %}
91
-
-`validated`: {% include {{ page.version.version }}/ldr/validated-description.md %}
92
82
93
83
## Examples
94
84
@@ -100,7 +90,7 @@ From the destination cluster of the LDR stream, run:
100
90
101
91
{% include_cached copy-clipboard.html %}
102
92
~~~ sql
103
-
CREATE LOGICALLY REPLICATED TABLE {database.public.destination_table_name} FROM TABLE {database.public.source_table_name} ON'external://source' WITH unidirectional, mode=validated;
93
+
CREATE LOGICALLY REPLICATED TABLE {database.public.destination_table_name} FROM TABLE {database.public.source_table_name} ON'external://source' WITH unidirectional;
-**Table-level replication**: When you initiate LDR, it will replicate all of the source table's existing data to the destination table. From then on, LDR will replicate the source table's data to the destination table to achieve eventual consistency.
45
45
-**Last write wins conflict resolution**: LDR uses [_last write wins (LWW)_ conflict resolution]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#conflict-resolution), which will use the latest [MVCC]({% link {{ page.version.version }}/architecture/storage-layer.md %}#mvcc) timestamp to resolve a conflict in row insertion.
46
46
-**Dead letter queue (DLQ)**: When LDR starts, the job will create a [DLQ table]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#dead-letter-queue-dlq) with each replicating table in order to track unresolved conflicts. You can interact and manage this table like any other SQL table.
47
-
-**Replication modes**: LDR offers different [_modes_]({% link {{ page.version.version }}/create-logical-replication-stream.md %}#ldr-modes) that apply data differently during replication, which allows you to consider optimizing for throughput or constraints during replication.
48
47
-**Monitoring**: To [monitor]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}) LDR's initial progress, current status, and performance, you can view metrics available in the DB Console, Prometheus, and Metrics Export.
Copy file name to clipboardExpand all lines: src/current/v25.2/manage-logical-data-replication.md
+3-10Lines changed: 3 additions & 10 deletions
Original file line number
Diff line number
Diff line change
@@ -5,8 +5,6 @@ toc: true
5
5
---
6
6
7
7
{{site.data.alerts.callout_info}}
8
-
{% include feature-phases/preview.md %}
9
-
10
8
Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.
11
9
{{site.data.alerts.end}}
12
10
@@ -22,7 +20,7 @@ In LDR, conflicts are detected at both the [KV]({% link {{ page.version.version
22
20
23
21
### KV level conflicts
24
22
25
-
LDR uses _last write wins (LWW)_ conflict resolution based on the [MVCC timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#mvcc) of the replicating write. LDR will resolve conflicts by inserting the row with the latest MVCC timestamp. Conflicts at the KV level are detected in both `immediate` and `validated` mode.
23
+
LDR uses _last write wins (LWW)_ conflict resolution based on the [MVCC timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#mvcc) of the replicating write. LDR will resolve conflicts by inserting the row with the latest MVCC timestamp.
26
24
27
25
Conflicts at the KV level are detected when there is either:
28
26
@@ -31,21 +29,16 @@ Conflicts at the KV level are detected when there is either:
31
29
32
30
### SQL level conflicts
33
31
34
-
In `validated` mode, when a conflict cannot apply due to violating [constraints]({% link {{ page.version.version }}/constraints.md %}), for example, a foreign key constraint or schema constraint, it will be retried for up to a minute and then put in the [DLQ](#dead-letter-queue-dlq) if it could not be resolved.
32
+
When a conflict cannot apply due to violating [constraints]({% link {{ page.version.version }}/constraints.md %}), for example, a foreign key constraint or schema constraint, LDR will send the row to the [DLQ](#dead-letter-queue-dlq).
35
33
36
34
### Dead letter queue (DLQ)
37
35
38
-
When the LDR job starts, it will create a DLQ table with each replicating table so that unresolved conflicts can be tracked. The DLQ will contain the writes that LDR cannot apply after the retry period, which could occur if:
36
+
When the LDR job starts, it will create a DLQ table with each replicating table so that unresolved conflicts can be tracked. The DLQ will contain the writes that LDR cannot apply after the retry period of a minute, which could occur if:
39
37
40
38
- The destination table was dropped.
41
39
- The destination cluster is unavailable.
42
40
- Tables schemas do not match.
43
41
44
-
In `validated` mode, rows are also sent to the DLQ when:
45
-
46
-
-[Foreign key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies are not met where there are foreign key constraints in the schema.
47
-
- Unique indexes and other constraints are not met.
48
-
49
42
{{site.data.alerts.callout_info}}
50
43
LDR will not pause when the writes are sent to the DLQ, you must manage the DLQ manually.
Copy file name to clipboardExpand all lines: src/current/v25.2/set-up-logical-data-replication.md
+2-13Lines changed: 2 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -5,8 +5,6 @@ toc: true
5
5
---
6
6
7
7
{{site.data.alerts.callout_info}}
8
-
{% include feature-phases/preview.md %}
9
-
10
8
Logical data replication is only supported in CockroachDB {{ site.data.products.core }} clusters.
11
9
{{site.data.alerts.end}}
12
10
@@ -39,7 +37,7 @@ If you're setting up bidirectional LDR, both clusters will act as a source and a
39
37
40
38
1. Prepare the clusters with the required settings, users, and privileges according to the LDR setup.
41
39
1. Set up [external connection(s)]({% link {{ page.version.version }}/create-external-connection.md %}) on the destination to hold the connection URI for the source.
42
-
1. Start LDR from the destination cluster with your required modes and syntax.
40
+
1. Start LDR from the destination cluster with your required syntax and options.
43
41
1. Check the status of the LDR job in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}).
44
42
45
43
## Before you begin
@@ -62,16 +60,14 @@ You cannot use LDR on a table with a schema that contains:
62
60
-[Partial indexes]({% link {{ page.version.version }}/partial-indexes.md %}) and [hash-sharded indexes]({% link {{ page.version.version }}/hash-sharded-indexes.md %})
63
61
- Indexes with a [virtual computed column]({% link {{ page.version.version }}/computed-columns.md %})
64
62
- Composite types in the [primary key]({% link {{ page.version.version }}/primary-key.md %})
63
+
-[Foreign key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies
65
64
66
65
Additionally, for the `CREATE LOGICALLY REPLICATED` syntax, you cannot use LDR on a table with a schema that contains:
67
66
68
67
-[User-defined types]({% link {{ page.version.version }}/enum.md %})
69
-
-[Foreign key]({% link {{ page.version.version }}/foreign-key.md %}) dependencies
70
68
71
69
For more details, refer to the LDR [Known limitations]({% link {{ page.version.version }}/logical-data-replication-overview.md %}#known-limitations).
72
70
73
-
When you run LDR in [`immediate` mode](#modes), you cannot replicate a table with [foreign key constraints]({% link {{ page.version.version }}/foreign-key.md %}). In [`validated` mode](#modes), foreign key constraints **must** match. All constraints are enforced at the time of SQL/application write.
74
-
75
71
## Step 1. Prepare the cluster
76
72
77
73
In this step you'll prepare the required settings and privileges for LDR.
@@ -186,11 +182,6 @@ You can use the `cockroach encode-uri` command to generate a connection string c
186
182
187
183
In this step, you'll start the LDR stream(s) from the destination cluster. You can replicate one or multiple tables in a single LDR job. You cannot replicate system tables in LDR, which means that you must manually apply configurations and cluster settings, such as [row-level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) and user permissions on the destination cluster.
188
184
189
-
<a id="modes"></a>_Modes_ determine how LDR replicates the data to the destination cluster. There are two modes:
190
-
191
-
- `immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %}
192
-
- `validated`: {% include {{ page.version.version }}/ldr/validated-description.md %}
193
-
194
185
LDR streams can be started using one of the following sections for instructions on creating an LDR stream. For details on which syntax to use, refer to the [Syntax](#syntax) section at the beginning of this tutorial:
@@ -234,8 +225,6 @@ Ensure you've created the table on the destination cluster with a matching schem
234
225
CREATE LOGICAL REPLICATION STREAM FROM TABLE {database.public.source_table_name} ON 'external://{source_external_connection}' INTO TABLE {database.public.destination_table_name};
235
226
~~~
236
227
237
-
You can change the default `mode` using the `WITH mode = validated` syntax.
238
-
239
228
If you would like to add multiple tables to the LDR job, ensure that the table name in the source table list and destination table list are in the same order:
0 commit comments