feat(gitlab): Cache results acl project membership #2280
Conversation
Summary of ChangesHello @chmouel, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant performance optimization for GitLab integration by implementing a caching layer for project membership checks. By storing the results of Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Pull Request Overview
This PR implements caching functionality for GitLab project membership checks to reduce redundant API calls when verifying the same user's membership status multiple times during event processing.
Key changes:
- Added a
memberCachemap to the GitLab Provider struct to cache membership results by user ID - Modified the
checkMembershipmethod to check cache first and store results for successful API calls - Implemented logic to avoid caching API failures to allow retry on transient errors
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/provider/gitlab/gitlab.go | Added memberCache field to Provider struct with documentation |
| pkg/provider/gitlab/acl.go | Implemented caching logic in checkMembership method with cache initialization and result storage |
| pkg/provider/gitlab/test/test.go | Added helper function for testing caching behavior with API call counting |
| pkg/provider/gitlab/acl_test.go | Added comprehensive tests for membership caching and failure handling scenarios |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
pipelines-as-code
bot
added
enhancement
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces caching for GitLab project membership checks to reduce API calls, which is a great performance improvement. The implementation is mostly solid, with good test coverage for the happy path and for uncached failures. However, I've found a potential issue in the error handling logic where a successful fallback check during an API failure could lead to incorrect caching, preventing future API checks for that user. My review includes a suggestion to fix this.
chmouel
force-pushed
the
SRVKP-9056-feat-gitlab-cache-results-acl-project-membership
branch
from
3e6bad3 to
af38bdb
Compare
| v.memberCache = map[int]bool{} | ||
| } | ||
|
|
||
| if allowed, ok := v.memberCache[userid]; ok { |
There was a problem hiding this comment.
@chmouel like in Konflux, what if event comes User A is member of Repository R and it is cached but same user does something for Repository B and there User A is not member or an approved user, but due to cache A will be allowed. wdyt?
There was a problem hiding this comment.
shouldn't it be mapping to repository URL
There was a problem hiding this comment.
that should not happen because it works by event not across repository tho,
What's the reason we make multiple call to check the membership multiple times when processing an event? If caching the first response works then only one call should be necessary. Reducing the call volume seems preferable, especially since the Provider object only lives for the duration of one event; it would only ever check one username FWICT |
it's basically how the code is structured.. when we check from a comment we need to do a ACL Check from the submitted sending /ok-to-test with all the logic coming if allowed from repo/org or OWNERS files, it's a bit of a recursive function |
|
I do want in the future that we do some caching via TTL of git api fetch of objects when requested via SHA since those should be immutable, we have a jira story for it.. |
| if err != nil { | ||
| // If the API call fails, fall back without caching the result so a | ||
| // transient failure can be retried on the next invocation. | ||
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | ||
| return isAllowed | ||
| } | ||
|
|
||
| if member.ID != 0 && member.ID == userid { | ||
| v.memberCache[userid] = true | ||
| return true | ||
| } | ||
|
|
||
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | ||
| v.memberCache[userid] = isAllowed |
There was a problem hiding this comment.
Small change but simplifies a bit
| if err != nil { | |
| // If the API call fails, fall back without caching the result so a | |
| // transient failure can be retried on the next invocation. | |
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | |
| return isAllowed | |
| } | |
| if member.ID != 0 && member.ID == userid { | |
| v.memberCache[userid] = true | |
| return true | |
| } | |
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | |
| v.memberCache[userid] = isAllowed | |
| member, _, apiErr := v.Client().ProjectMembers.GetInheritedProjectMember(v.targetProjectID, userid) | |
| if apiErr == nil && member.ID != 0 && member.ID == userid { | |
| v.memberCache[userid] = true | |
| return true | |
| } | |
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | |
| // don't cache result if GetMembership API call errored | |
| if apiErr == nil { | |
| v.memberCache[userid] = isAllowed | |
| } |
There was a problem hiding this comment.
tbh, i know it sounds harmless code change, but i rather not not too much refactoring before we get full unittests coverage.. I learned the hard way on github when working on its acl.go there
| } | ||
| } | ||
|
|
||
| func TestMembershipCaching(t *testing.T) { |
There was a problem hiding this comment.
Can you also add a test to check the caching on
- user is allowed via owners file
- user is not allowed
There was a problem hiding this comment.
weird that they were not added previously, i'll add a github issue so we can properly coverage acl.go in gitlab like the way we do in github
There was a problem hiding this comment.
here is a jira ticket for it https://issues.redhat.com/browse/SRVKP-9074
Implemented caching in gitlab for results returned by `checkMembership`. This was done to reduce repeated calls to the GitLab API when checking the membership status of the same user multiple times during processing of an event. Co-authored-by: Claude <[email protected]> Jira: https://issues.redhat.com/browse/SRVKP-9056 Signed-off-by: Chmouel Boudjnah <[email protected]>
chmouel
force-pushed
the
SRVKP-9056-feat-gitlab-cache-results-acl-project-membership
branch
from
af38bdb to
731bea8
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces caching for GitLab project membership checks, which is a great way to reduce redundant API calls and improve performance. The implementation is logical and includes new tests to verify the caching behavior, including how it handles API failures. My main feedback is to address a potential race condition in the cache implementation. The current code is not thread-safe, which could cause issues if used concurrently. I've provided suggestions to add a mutex and refactor the caching logic to ensure thread safety.
| triggerEvent string | ||
| // memberCache caches membership/permission checks by user ID within the | ||
| // current provider instance lifecycle to avoid repeated API calls. | ||
| memberCache map[int]bool |
There was a problem hiding this comment.
The memberCache is accessed and modified in checkMembership without any synchronization. If the Provider instance is ever used concurrently, this will lead to a race condition. To make the cache thread-safe, you should add a sync.RWMutex to the Provider struct. You'll also need to add import "sync" to the file.
I'll provide a suggestion for how to use this mutex in pkg/provider/gitlab/acl.go.
memberCache map[int]bool
memberCacheMutex sync.RWMutex| func (v *Provider) checkMembership(ctx context.Context, event *info.Event, userid int) bool { | ||
| // Initialize cache lazily | ||
| if v.memberCache == nil { | ||
| v.memberCache = map[int]bool{} | ||
| } | ||
|
|
||
| if allowed, ok := v.memberCache[userid]; ok { | ||
| return allowed | ||
| } | ||
|
|
||
| member, _, err := v.Client().ProjectMembers.GetInheritedProjectMember(v.targetProjectID, userid) | ||
| if err == nil && member.ID != 0 && member.ID == userid { | ||
| if err != nil { | ||
| // If the API call fails, fall back without caching the result so a | ||
| // transient failure can be retried on the next invocation. | ||
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | ||
| return isAllowed | ||
| } | ||
|
|
||
| if member.ID != 0 && member.ID == userid { | ||
| v.memberCache[userid] = true | ||
| return true | ||
| } | ||
|
|
||
| isAllowed, _ := v.IsAllowedOwnersFile(ctx, event) | ||
| v.memberCache[userid] = isAllowed | ||
| return isAllowed | ||
| } |
There was a problem hiding this comment.
The current implementation of checkMembership is not thread-safe. The lazy initialization and subsequent reads/writes to memberCache can cause a race condition if this function is called from multiple goroutines on the same Provider instance. Here is a refactored version that uses the RWMutex (suggested in gitlab.go) to ensure thread safety. This version also avoids holding a lock during the network API call, which is important for performance.
func (v *Provider) checkMembership(ctx context.Context, event *info.Event, userid int) bool {
// Check cache with a read lock.
v.memberCacheMutex.RLock()
if v.memberCache != nil {
if allowed, ok := v.memberCache[userid]; ok {
v.memberCacheMutex.RUnlock()
return allowed
}
}
v.memberCacheMutex.RUnlock()
// If not in cache, perform the check. The API call is done outside of any lock.
member, _, err := v.Client().ProjectMembers.GetInheritedProjectMember(v.targetProjectID, userid)
if err != nil {
// If the API call fails, fall back without caching the result so a
// transient failure can be retried on the next invocation.
isAllowed, _ := v.IsAllowedOwnersFile(ctx, event)
return isAllowed
}
var isAllowed bool
if member.ID != 0 && member.ID == userid {
isAllowed = true
} else {
isAllowed, _ = v.IsAllowedOwnersFile(ctx, event)
}
// Acquire a write lock to update the cache.
v.memberCacheMutex.Lock()
defer v.memberCacheMutex.Unlock()
// Initialize the cache if it's the first time.
if v.memberCache == nil {
v.memberCache = map[int]bool{}
}
v.memberCache[userid] = isAllowed
return isAllowed
}3 participants
Implemented caching in gitlab for results returned by
checkMembership.This was done to reduce repeated calls to the GitLab API when checking
the membership status of the same user multiple times during processing
of an event.
Co-authored-by: Claude [email protected]
Signed-off-by: Chmouel Boudjnah [email protected]
📝 Description of the Change
👨🏻 Linked Jira
Jira: https://issues.redhat.com/browse/SRVKP-9056
🔗 Linked GitHub Issue
Fixes #
🚀 Type of Change
fix:)feat:)feat!:,fix!:)docs:)chore:)refactor:)enhance:)deps:)🧪 Testing Strategy
🤖 AI Assistance
If you have used AI assistance, please provide the following details:
Which LLM was used?
Extent of AI Assistance:
Important
If the majority of the code in this PR was generated by an AI, please add a
Co-authored-bytrailer to your commit message.For example:
Co-authored-by: Gemini [email protected]
Co-authored-by: ChatGPT [email protected]
Co-authored-by: Claude [email protected]
Co-authored-by: Cursor [email protected]
Co-authored-by: Copilot [email protected]
**💡You can use the script
./hack/add-llm-coauthor.shto automatically addthese co-author trailers to your commits.
✅ Submitter Checklist
fix:,feat:) matches the "Type of Change" I selected above.make testandmake lintlocally to check for and fix anyissues. For an efficient workflow, I have considered installing
pre-commit and running
pre-commit installtoautomate these checks.