Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txn support IS_EXIST and NOT_EXIST compare #10769

Closed
wants to merge 3 commits into from
Closed

txn support IS_EXIST and NOT_EXIST compare #10769

wants to merge 3 commits into from

Conversation

yylt
Copy link

@yylt yylt commented May 29, 2019

txn support IS_EXIST and NOT_EXIST compare

Fix #10566

cc @purpleidea

@codecov-io
Copy link

codecov-io commented May 29, 2019

Codecov Report

Merging #10769 into master will increase coverage by 0.64%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #10769      +/-   ##
==========================================
+ Coverage   62.47%   63.12%   +0.64%     
==========================================
  Files         392      392              
  Lines       37158    37175      +17     
==========================================
+ Hits        23215    23466     +251     
+ Misses      12381    12132     -249     
- Partials     1562     1577      +15
Impacted Files Coverage Δ
etcdserver/apply.go 78.3% <0%> (+0.23%) ⬆️
clientv3/compare.go 60.75% <0%> (-14.25%) ⬇️
pkg/logutil/zap_grpc.go 47.61% <0%> (-4.77%) ⬇️
pkg/testutil/recorder.go 77.77% <0%> (-3.71%) ⬇️
auth/simple_token.go 83.73% <0%> (-3.26%) ⬇️
clientv3/leasing/txn.go 88.09% <0%> (-3.18%) ⬇️
etcdserver/api/v3election/election.go 61.11% <0%> (-2.78%) ⬇️
pkg/adt/interval_tree.go 85.28% <0%> (-2.71%) ⬇️
clientv3/concurrency/election.go 79.68% <0%> (-2.35%) ⬇️
etcdserver/api/rafthttp/msgappv2_codec.go 69.56% <0%> (-1.74%) ⬇️
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23a89b0...0d113f8. Read the comment docs.

@purpleidea
Copy link
Contributor

wow, awesome! I'd recommend we add a test with this patch to keep it stable! Also, do we want to guarantee the behaviour of checking if a value when that value doesn't exist? Should it be an error instead?

@jingyih
Copy link
Contributor

jingyih commented May 29, 2019

Could you provide an example usage? Is this trying to decide whether a key exist? I am trying to understand the difference between this and something like: clientv3.Compare(clientv3.ModRevision("key"), ">", 0)

@yylt
Copy link
Author

yylt commented May 30, 2019

@jingyih yes, this trying to decide whether a key exist or not exist. I think ModRevision may be used as Optimistic lock mostly .

ctx, cancel := context.WithTimeout(context.Background(), requestTimeout)
_, err = kvc.Txn(ctx).
// txn value comparisons are lexical
If(clientv3.CompareExist(clientv3.Value("key"), "ne")).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I am still confused. If we replace this line with the following line, isn't it the same?
If(clientv3.Compare(clientv3.ModRevision("key"), "=", 0))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I don't think this PR solves the original issue #10566. The original issue is that when we use clientv3.Compare(clientv3.Value(key), "=", value), the compare always return false if key does not exist, regardless of "=" or "!=".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your suggestions.

1: There are the same, i just think ModRevision make user realize this API should be use as Optimistic lock in mvcc storage

2: clientv3.Compare(clientv3.Value(key), "=", value) always return false, the reason just is

if c.Target == pb.Compare_VALUE {

This PR had delete these codes.
I am also confused why add these codes, and think value compare result should be decided by bytes.Compare

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your suggestions.

1: There are the same, i just think ModRevision make user realize this API should be use as Optimistic lock in mvcc storage

Thanks for confirming. I think we should not break API to implement a feature that already exists.

2: clientv3.Compare(clientv3.Value(key), "=", value) always return false, the reason just is

if c.Target == pb.Compare_VALUE {

This PR had delete these codes.
I am also confused why add these codes, and think value compare result should be decided by bytes.Compare

Sorry I missed the deleting part. Removing the code block means treating non-exist-key's value as an empty string (or its byte presentation). This is probably fine, but we should more carefully think through this.

@purpleidea
Copy link
Contributor

So there's this: https://godoc.org/github.com/coreos/etcd/clientv3/clientv3util

Secondly, I think this relates to the whole design of what should happen if you try to read the value of a key that doesn't exist. I think this should propagate upwards as an error. This is the equivalent of a nil in golang. I can explain this better if it would help.

@yylt
Copy link
Author

yylt commented May 30, 2019

Wow, I will close this PR
Thank your explanation @jingyih @purpleidea

@yylt yylt closed this May 30, 2019
@jingyih
Copy link
Contributor

jingyih commented May 30, 2019

@purpleidea Thanks for the link. I agree the behavior should be carefully designed. The discussion can be continued in the original issue.

@jingyih
Copy link
Contributor

jingyih commented May 30, 2019

@yylt Thanks for trying to fix this!

@purpleidea
Copy link
Contributor

@yylt I think there was some promise in the PR, in particular when I previously read through the code, I noticed the: https://github.com/etcd-io/etcd/pull/10769/files#diff-f141df18bc5c4d2e821b7d9dfb484f95L449 which you removed was probably a bug which was masking weird key not exists handling as discussed in #10566
So I think you're on the right track.

Perhaps you'd like to take what you learned digging through this part of the code and look at #10571 ?

@yylt yylt deleted the dev branch May 30, 2019 05:58
@yylt
Copy link
Author

yylt commented May 30, 2019

@purpleidea yes, I can try the feature #10571

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Compare not equal operator is broken in etcd
4 participants