Skip to content

DOC-12484 XDCR Conflict Logging feature #3806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 22 commits into
base: release/8.0
Choose a base branch
from

Conversation

rao-shwe
Copy link
Contributor

@rao-shwe rao-shwe commented May 9, 2025

DOC-12484

Link to the preview doc: https://preview.docs-test.couchbase.com/DOC-12484/server/current/learn/clusters-and-availability/xdcr-conflict-logging-feature.html

Preview pages:

PR pages:
New page: XDCR Conflict Logging.

Updated the following pages for "XDCR Conflict Logging":

Don't review the following files: The following are 7.6.6 release docs which were missing in the release/8.0 branch.

Copy link

@sumukhbhat2701 sumukhbhat2701 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some review feedback which hold true for all pages:

  1. For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.
  2. There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.
  3. The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.
  4. There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.
  5. Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc.
  6. There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

@sumukhbhat2701
Copy link

I think I missed one of the pages from reviewing, so if somethings are already done from last comment, please ignore.

Copy link
Contributor Author

@rao-shwe rao-shwe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sumukhbhat2701

I've implemented most of your review inputs and closed the comments.

@rao-shwe
Copy link
Contributor Author

rao-shwe commented Jul 21, 2025

@sumukhbhat2701

Points 1 and 2 are fixed.
Point 3: Already has examples and descriptions. Not okay to repeat the same content in multiple locations. So I've added a link to examples wherever necessary.
Point 4, 5, and 6: Already exists.

Edit: All fixed in draft-2.

Some review feedback which hold true for all pages:

  1. For this feature specifically, we need to use the term "true conflicts" more than just mentioning "conflicts". That means we need to first define what a true conflict is and set the expectation.
  2. There should be a warning that this feature is best effort (and that true conflicts is assumed to be very low). Everything that's in this slide - https://couchbase.slack.com/archives/C0963TSUU0N/p1752763776316649.
  3. The setting is quite complex to understand just from textual description. An example will do a lot of help to someone new reading this.
  4. There should be a mention that on every true conflict detected, XDCR will log 3 documents to the conflict collection - CRD (Conflict record document - contains metadata of detected true conflict), source document in conflict & target document in conflict. It should be mentioned that the CRD will contain the document IDs of source and target documents logged. Maybe an example of source and target document IDs in CRD.
  5. Continuation of (3), I think there should be some examples on how to make use of the detected and logged conflicts. Eg: Use SDK, N1QL, range scan, eventing etc. @hyunjuV I think you had a document prepared for this, was that for public docs?
  6. There should be a mention that the logged documents will not be replicated by XDCR if conflict collection is a source collection of any XDCR.

If you try to use the feature _XDCR Active-Active with Sync Gateway_ when you have more than 10 user xattrs in your document, the XDCR replication **silently skips** replicating that document.
As a result, the data in the replication-skipped document will not be consistent between the target and source clusters.
The only way you will know this skip occured is because the Prometheus stat `subdoc_cmd_docs_skipped` will be incremented and the document will _not_ be consistent between the target and source.
* Eventing Service cannot be used with Sync Gateway in bi-directional XDCR.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • If you are using Eventing Service functions that update documents in the XDCR replicated buckets, you must take care that the deployed Eventing functions do not cause XDCR to ping-pong and never stop replicating.

As a result, the data in the replication-skipped document will not be consistent between the target and source clusters.
The only way you will know this skip occured is because the Prometheus stat `subdoc_cmd_docs_skipped` will be incremented and the document will _not_ be consistent between the target and source.
* Eventing Service cannot be used with Sync Gateway in bi-directional XDCR.
If used with the _Sync Gateway in a bi-directional, active-active XDCR_ environment, the updates of Eventing Service metadata in the source and the target clusters causes XDCR to ping-pong and never stop replicating.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are using Eventing functions that update the documents in the XDCR replicated buckets (also referred to as Eventing source bucket mutations), ensure that the deployed functions behave as desired in the replication environment. Within a bi-directional, active-active XDCR environment, the deployed Eventing functions can cause XDCR to ping-pong and never stop replicating if you do not include logic to prevent the infinite loop. In general, for active-active, avoid redundant updates with appropriate logic within the Eventing functions. See XDCR Active-Active and Eventing for more information.

Note for @rao-shwe :
Fortune Ikechi is working on DOC-13300, which will add a page called "XDCR Active-Active and Eventing" in 7.6.x documentation. One of the changes for that work is to update this note in lines 25-26.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants