Skip to content

feat(api): add shared source-author and distributed-lock helpers - #17086

Open
AtefBellaaj wants to merge 1 commit into
devfrom
atef/ai-2335-shared-sync-helpers
Open

AtefBellaaj wants to merge 1 commit into
devfrom
atef/ai-2335-shared-sync-helpers

Conversation

@AtefBellaaj

@AtefBellaaj AtefBellaaj commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Pull Request

Summary

The upcoming Langfuse prompt sync needs a deterministic author id for records owned by an external source and a cross-instance lock so two admins or an admin and a scheduler cannot run the same sync at once. Today the first exists only as a private function inside the GitHub skill sync (packages/api/src/skills/sync/github.ts) and the second only as a feature-local Redis lock in the principals cache (packages/api/src/cache/principals.ts). This PR lifts both into shared @librechat/api helpers: makeSourceAuthorId(provider, sourceId, tenantId?) in packages/api/src/utils/author.ts and createDistributedLock({ redisClient?, now? }) in packages/api/src/cache/lock.ts. Behavior is unchanged: the skill sync now calls the shared helper and produces byte-identical author ids (the unchanged github.spec.ts proves it), and nothing switches to the new lock yet. The Langfuse prompt stack (starting at PR #16854) will retarget onto this branch.

Related to #16854

How it works

Ownership:

packages/api/src/
├── utils/author.ts        # makeSourceAuthorId: sha256(provider:sourceId[:tenantId]) sliced to a 24-hex id
├── utils/index.ts          # re-exports the new author helper
├── cache/lock.ts           # createDistributedLock: Redis-backed or in-process fallback
├── cache/index.ts          # re-exports the new lock helper
└── skills/sync/github.ts   # call site now imports makeSourceAuthorId instead of defining it locally

makeSourceAuthorId joins its components with : and does not escape them, so it throws when provider, sourceId, or a provided tenantId contains : — otherwise two distinct inputs could collide on the same seed. For valid inputs the seed (provider:sourceId or provider:sourceId:tenantId) is unchanged from the ids the GitHub skill sync already stores, so the encoding cannot change without re-keying existing records.

-- EXTEND_SCRIPT
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('PEXPIRE', KEYS[1], ARGV[2])
end
return 0

-- RELEASE_SCRIPT
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0

The Redis path uses SET key token PX ttl NX to acquire, plus the two owner-checked scripts above (extend and release) run through the existing evalScript helper (EVALSHA with NOSCRIPT fallback); without a Redis client, an in-process Map with an injectable clock is used, which is complete rather than degraded because a LibreChat deployment without Redis is single-node. acquire(key, ttlMs) and extend(key, token, ttlMs) throw RangeError unless ttlMs is a positive integer. release(key, token) returns true when the caller still owned the key and false when the lease had already expired or another holder owns it. The in-process backend removes an expired entry when its key is touched again and sweeps the whole table once it grows past a threshold that doubles after each sweep, so acquire stays O(1) per call and keys that are never touched again do not accumulate.

The module lives in cache/ next to redisScript.ts, whose evalScript it reuses, and principals.ts, whose owner-token lock shape it generalizes. The GitHub skill sync under packages/api/src/skills/sync does not use the Mongo lease in packages/data-schemas/src/methods/skillSync.ts (tryAcquireSkillSyncLock); it needs a lock that coordinates admin-triggered and scheduled runs without a SkillSyncStatus document round trip. The repository already has owner-token Redis locks with the same Lua shape in cache/principals.ts, agents/files.ts, cluster/LeaderElection.ts, and mcp/catalog/store.ts, and multi-replica LibreChat already requires shared Redis (.env.example, scheduled chats comment), so the in-process backend only needs to cover the single-process case.

Type of change

  • Refactor
  • Feature

Testing

Tested environments/configuration:

  • packages/api Jest with a mocked ioredis client; no browser or provider involved.

Automated tests:

  • packages/api: src/utils/author.spec.ts, src/cache/lock.spec.ts, src/skills/sync/github.spec.ts (3 suites, all passing)
  • npx tsc --noEmit in packages/api
  • Prettier, ESLint, and import-sort checks on the changed files
  • Added tests: author.spec.ts (12 tests), lock.spec.ts (36 tests: 20 plain cases plus 4 test.each blocks of 4 ttl values each)
  • Not run: manual Redis sanity check

Screenshots / recordings

No user-facing change.

Risk / compatibility

Low. The only runtime change is the github.ts call site, with parity proven by the unchanged spec. No schema, index, or migration change; no new Mongo-only operators; Redis commands are single-key, so this is Redis Cluster safe. Safe to deploy and roll back in any order.

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors
  • User-facing or complex behavior is documented where necessary
  • Required dependency changes have been merged/published
  • Required documentation PR: N/A

🤖 Generated with Claude Code

@AtefBellaaj

Copy link
Copy Markdown
Collaborator Author

@codex review 0493a54

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T13:52:55.215080Z 4b6d8bb Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0493a54880

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/cache/lock.ts
Comment thread packages/api/src/utils/author.ts
@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from 0493a54 to a26b9fa Compare October 14, 2026 11:29
@AtefBellaaj

Copy link
Copy Markdown
Collaborator Author

@codex review a26b9fa

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: a26b9fa451

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codegraph-librechat codegraph-librechat Bot added the 🗺️ Redis Infra codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9) label Oct 14, 2026
@AtefBellaaj
AtefBellaaj added this pull request to stack #17090 October 14, 2026 12:55
@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from a26b9fa to 74d3ca3 Compare October 14, 2026 12:55
@AtefBellaaj

Copy link
Copy Markdown
Collaborator Author

@codex review 74d3ca3

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 74d3ca31dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from 74d3ca3 to 1a38bf5 Compare October 14, 2026 13:38
@AtefBellaaj

Copy link
Copy Markdown
Collaborator Author

@codex review 1a38bf5

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a38bf54fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/cache/lock.ts
@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from 1a38bf5 to 4b6d8bb Compare October 14, 2026 13:49
@AtefBellaaj

Copy link
Copy Markdown
Collaborator Author

@codex review 4b6d8bb

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 4b6d8bb188

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@AtefBellaaj
AtefBellaaj marked this pull request as ready for review October 14, 2026 13:53
@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from 4b6d8bb to 8033c78 Compare October 15, 2026 08:22
Lift makeSourceAuthorId out of the GitHub skill sync into
packages/api/src/utils/author.ts and add createDistributedLock in
packages/api/src/cache/lock.ts (Redis SET NX PX with owner-checked Lua
extend/release, in-process Map fallback without Redis). The skill sync
call site uses the shared helper; author ids are byte-identical.
@AtefBellaaj
AtefBellaaj force-pushed the atef/ai-2335-shared-sync-helpers branch from 8033c78 to b4d2bf0 Compare October 15, 2026 10:49
@codegraph-librechat codegraph-librechat Bot added the 🗺️ DB Models & Ops codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9) label Oct 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🗺️ DB Models & Ops codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9) 🗺️ Redis Infra codegraph: the taxonomy area this belongs to (classifier, confidence ≥ 0.9)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant