feat(common): Add cross-process lock dirtying #5672

Hywan · 2025-09-15T16:54:38Z

This set of patches updates the cross-process lock to introduce “dirtiness”. The lock is considered “dirty” when the lock is acquired from another holder:

holder A takes the lock for the first time: not dirty,
holder A takes the lock again, not dirty,
holder B takes the lock for the first time: not dirty,
holder A takes the lock again: dirty

To achieve so, a generation value is added in the stores. This generation is a u64. Every time the lock is acquired from a previous holder, this generation is incremented by one. That's basically it. The first value for the generation is 0, which acts as a guard value.

The TryLock::try_lock method now returns an Option<CrossProcessLockGeneration> instead of bool.

The CrossProcessLock::try_lock_once method now returns a CrossProcessLockResult instead of an Option<CrossProcessLockGuard>. It's a new type that looks like this:

enum CrossProcessLockResult {
    Clean(CrossProcessLockGuard),
    Dirty(CrossProcessLockGuard),
    Unobtained,
}

The CrossProcessLock::spin_lock method doesn't change for the moment. Let's do that in another PR where we will use the new API to decide how to invalidate the various states here and there. I wanted to keep this PR as small as possible first.

Address [meta] _Dirty_ cross-process locks #4874

codecov · 2025-09-16T09:38:41Z

Codecov Report

❌ Patch coverage is 86.85446% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.55%. Comparing base (81ff96d) to head (d5c68c0).
⚠️ Report is 22 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
crates/matrix-sdk-common/src/cross_process_lock.rs	80.89%	10 Missing and 7 partials ⚠️
crates/matrix-sdk-sqlite/src/crypto_store.rs	88.88%	1 Missing and 2 partials ⚠️
crates/matrix-sdk-sqlite/src/event_cache_store.rs	88.00%	1 Missing and 2 partials ⚠️
crates/matrix-sdk-sqlite/src/media_store.rs	88.00%	1 Missing and 2 partials ⚠️
crates/matrix-sdk/src/encryption/mod.rs	50.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5672   +/-   ##
=======================================
  Coverage   88.54%   88.55%           
=======================================
  Files         361      361           
  Lines      101533   101629   +96     
  Branches   101533   101629   +96     
=======================================
+ Hits        89904    89998   +94     
+ Misses       7418     7415    -3     
- Partials     4211     4216    +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codspeed-hq · 2025-09-16T09:49:37Z

CodSpeed Performance Report

Merging #5672 will not alter performance

_{Comparing Hywan:feat-cross-process-lock-poisoning (d5c68c0) with main (81ff96d)}

Summary

✅ 50 untouched

Hywan · 2025-11-04T13:41:55Z

Must be merged after #5829.

This patch adds `CrossProcessLockGeneration`. A lock generation is an integer incremented each time the lock is taken by another holder. If the generation changes, it means the lock is _dirtied_. This _dirtying_ aspect is going to be expanded in the next patches. This patch focuses on the introduction of this _generation_. The `CrossProcessLock::try_lock_once` method, and the `TryLock::try_lock` method, both returns a `Option<CrossProcessLockGeneration>` instead of a `bool`: `true` is replaced by `Some(_)`, `false` by `None`.

…cache stores. This patch adds `Lease::generation` support in the crypto, media and event cache stores. For the crypto store, we add the new `lease_locks` object store/table. Previously, `Lease` was stored in `core`, but without any prefix, it's easy to overwrite another records, it's dangerous. The sad thing is that it's hard to delete the existing leases in `core` because the keys aren't known. See the comment in the code explaining the tradeoff. For media and event cache stores, the already existing `leases` object store/table is cleared so that we can change the format of `Lease` easily.

crates/matrix-sdk-common/src/cross_process_lock.rs

This patch detects when the cross-process lock has been dirtied. A new `CrossProcessLockResult` enum is introduced to simplify the returned value of `try_lock_once` and `spin_lock`. It flattens the previous `Result<Option<_>>` by providing 3 variants: `Clean`, `Dirty` and `Unobtained`.

poljar

Alright, I think all of this makes sense to me and it looks good.

poljar · 2025-11-06T12:48:20Z

crates/matrix-sdk-sqlite/src/crypto_store.rs

+                        UPDATE SET
+                            holder = excluded.holder,
+                            expiration = excluded.expiration,
+                            generation =
+                                CASE holder
+                                    WHEN excluded.holder THEN generation
+                                    ELSE generation + 1
+                                END
+                        WHERE
+                            holder = excluded.holder
+                            OR expiration < ?4
+                    RETURNING generation


It feels a bit wrong to have this three times, but moving this to a common DB seems like it would cause other kind of trouble.

I don't have a better idea here 😅

We could refactor that later to have a common implementation.

poljar · 2025-11-06T13:27:21Z

crates/matrix-sdk-common/src/cross_process_lock.rs

+impl CrossProcessLockResult {
+    /// Convert from [`CrossProcessLockResult`] to
+    /// [`Option<T>`] where `T` is [`CrossProcessLockGuard`].
+    pub fn ok(self) -> Option<CrossProcessLockGuard> {
+        match self {
+            Self::Clean(guard) | Self::Dirty(guard) => Some(guard),
+            Self::Unobtained => None,
+        }
+    }
+
+    /// Return `true` if the lock has been obtained, `false` otherwise.
+    pub fn is_ok(&self) -> bool {
+        matches!(self, Self::Clean(_) | Self::Dirty(_))
+    }
+}


Something we mentioned in chat but for archeological reasons I'll repeat it here.

I don't think it's worth it to reimplement the Result type to avoid having Result<Result<T>>.

But I think we left the modification of the return type to another PR.

So nothing to do here.

Thanks for the future us.

Hywan force-pushed the feat-cross-process-lock-poisoning branch from 3c168ad to 976f768 Compare September 16, 2025 09:22

Hywan force-pushed the feat-cross-process-lock-poisoning branch 3 times, most recently from 39b6f56 to b29747d Compare September 16, 2025 16:33

Hywan force-pushed the feat-cross-process-lock-poisoning branch 5 times, most recently from 891db14 to 985032c Compare September 26, 2025 12:51

Hywan changed the title ~~feat(common): Add cross-process lock poison~~ feat(common): Add cross-process lock dirtying Sep 26, 2025

Hywan force-pushed the feat-cross-process-lock-poisoning branch 7 times, most recently from 33f45c4 to 960bc89 Compare November 4, 2025 13:22

Hywan marked this pull request as ready for review November 4, 2025 13:40

Hywan requested review from a team as code owners November 4, 2025 13:40

Hywan requested review from poljar, richvdh and stefanceriu and removed request for a team and richvdh November 4, 2025 13:40

chore(indexeddb): Run rustfmt.

5663f4d

Hywan added 2 commits November 4, 2025 15:13

Hywan force-pushed the feat-cross-process-lock-poisoning branch from 960bc89 to d444e68 Compare November 4, 2025 14:14

stefanceriu reviewed Nov 4, 2025

View reviewed changes

crates/matrix-sdk-common/src/cross_process_lock.rs Outdated Show resolved Hide resolved

Hywan added 3 commits November 5, 2025 09:01

doc(common): Update CHANGELOG.md.

6a02e88

doc(common): Fix a link in the CHANGELOG.md.

d5c68c0

Hywan force-pushed the feat-cross-process-lock-poisoning branch from 21f3014 to d5c68c0 Compare November 5, 2025 08:02

This was referenced Nov 5, 2025

[meta] _Dirty_ cross-process locks #4874

Open

refactor(common): Provide a unified, solid API between try_lock_once and spin_lock on CrossProcessLock #5837

Merged

poljar approved these changes Nov 6, 2025

View reviewed changes

Hywan merged commit 3b14184 into matrix-org:main Nov 7, 2025
54 checks passed

richvdh mentioned this pull request Nov 10, 2025

Building matrix-rust-sdk-crypto-wasm against the latest matrix-sdk Rust is failing matrix-org/matrix-sdk-crypto-wasm#277

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(common): Add cross-process lock dirtying #5672

feat(common): Add cross-process lock dirtying #5672

Uh oh!

Hywan commented Sep 15, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

Hywan commented Nov 4, 2025

Uh oh!

Uh oh!

poljar left a comment

Uh oh!

poljar Nov 6, 2025

Uh oh!

Hywan Nov 7, 2025

Uh oh!

poljar Nov 6, 2025

Uh oh!

Hywan Nov 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat(common): Add cross-process lock dirtying #5672

feat(common): Add cross-process lock dirtying #5672

Uh oh!

Conversation

Hywan commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #5672 will not alter performance

Summary

Uh oh!

Hywan commented Nov 4, 2025

Uh oh!

Uh oh!

poljar left a comment

Choose a reason for hiding this comment

Uh oh!

poljar Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Hywan Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

poljar Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Hywan Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Hywan commented Sep 15, 2025 •

edited

Loading

codecov bot commented Sep 16, 2025 •

edited

Loading

codspeed-hq bot commented Sep 16, 2025 •

edited

Loading