Skip to content

feat: adding support for commit refs in CAS#6010

Merged
yhakbar merged 2 commits into
mainfrom
feat/adding-support-for-commit-refs-in-cas
May 12, 2026
Merged

feat: adding support for commit refs in CAS#6010
yhakbar merged 2 commits into
mainfrom
feat/adding-support-for-commit-refs-in-cas

Conversation

@yhakbar

@yhakbar yhakbar commented May 1, 2026

Copy link
Copy Markdown
Collaborator

Description

Right now, the CAS only supports branch/tag refs. This updates the implementation to also support commit refs.

TODOs

Read the Gruntwork contribution guidelines.

  • I authored this code entirely myself
  • I am submitting code based on open source software (e.g. MIT, MPL-2.0, Apache)
  • I am adding or upgrading a dependency or adapted code and confirm it has a compatible open source license
  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added / Removed / Updated [X].

Migration Guide

Summary by CodeRabbit

  • New Features

    • CAS now accepts commit SHAs (full or abbreviated) in addition to branch/tag names for Git sources.
  • Documentation

    • Added clarification on how Terragrunt handles commit SHA references in CAS, including cold and warm clone behavior.
    • Updated changelog documentation for commit SHA resolution and caching behavior.
  • Tests

    • Added comprehensive test coverage for commit reference cloning, caching, and offline behavior.

Review Change Stack

@vercel

vercel Bot commented May 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
terragrunt-docs Ready Ready Preview, Comment May 12, 2026 5:32pm

Request Review

@coderabbitai

coderabbitai Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR extends the CAS system to support cloning Git repositories by full or abbreviated commit SHA via ref=<sha> parameters. The changes refactor GitStore to use locked handles, introduce ref-type abstractions to distinguish symbolic refs (branches/tags) from commit refs, and add comprehensive fallback logic for offline caching and temporary clones when the central store is unavailable.

Changes

CAS Commit-SHA Reference Support

Layer / File(s) Summary
Git Primitives
internal/git/errors.go, internal/git/git.go, internal/git/server.go, internal/cas/testserver_test.go
Introduces ErrUnknownRevision error variable, RevParseCommit method to resolve local commit SHAs via git rev-parse, and test server helpers (Head, Tag, Branch) plus newEmptyTestServer fixture builder for test setup.
GitStore Locking and Repository Handles
internal/cas/gitstore.go, internal/cas/gitstore_test.go
Refactors GitStore.EnsureRef and EnsureCommit to return locked GitStoreRepo handles instead of bare-path/unlocker pairs. Adds session-based lock acquisition/release via repoSession, case-insensitive commit-hash prefix checking, and support for full-history fetches with canonicalization. Test updates validate the new handle-based API and error scenarios.
CAS Reference Resolution and Population
internal/cas/cas.go, internal/cas/cas_test.go
Adds FS backing validation on CAS.New; refactors CAS.Clone to resolve refs into an internal resolvedRef abstraction (symbolicRef/commitRef), then dispatches to ref-kind-aware population helpers (populateTreeFromSymbolicRef, populateTreeFromCommitRef) that first attempt GitStore then fall back to temporary clones with cleanup warnings on failure.
Deterministic CAS Keys from Resolved Refs
internal/cas/stacks.go
Updates ProcessStackComponent to derive deterministic CAS keys from resolvedRef.CommitHash(), ensuring consistent hashing for both symbolic and commit refs.
Commit-SHA Clone and Caching Tests
internal/cas/commitref_test.go
Comprehensive test suite with 15 new tests covering full/abbreviated commit-SHA cloning, caching behavior (offline resolution after first fetch, known-hash fast paths), edge cases (non-tip commits, 40-char hex branch names), tag refs, fallback clone behavior when GitStore fails, and concurrent racing scenarios.
End-to-End Commit-SHA Download Test
test/fixtures/download/remote-commit-ref/terragrunt.hcl, test/integration_download_test.go
New Terragrunt fixture with a source pinned to a commit SHA, and integration test (TestDownloadWithCASCommitRef) that applies the fixture with CAS enabled and verifies the output.
Feature Documentation and Changelog
docs/src/content/docs/03-features/07-caching/04-cas.mdx, docs/src/data/changelog/v1.0.5/cas-commit-refs.mdx
Documents commit-SHA support: explains full/abbreviated SHA acceptance, describes cold-clone full-history fetch behavior and warm-clone cache reuse, provides HCL examples, and clarifies differences from branch/tag refs. Includes v1.0.5 changelog entry with resolution behavior details.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • gruntwork-io/terragrunt#6003: Implements the core GitStore-based CAS workflow with per-URL bare Git stores and ref/commit resolution that this PR extends with commit-SHA support.
  • gruntwork-io/terragrunt#5911: Touches overlapping CAS implementation and Git ref resolution paths around stack/source processing.

Suggested reviewers

  • denis256
  • thisguycodes
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The PR description covers the main change but is incomplete against the template. It lacks specific details about what was updated, is missing structured release notes content, and the migration guide section remains a placeholder. However, the core description explains the objective adequately. Complete the 'Release Notes (draft)' section with a one-line summary of what was added/changed. If no migration guide is needed, explicitly state that this change is backward compatible and requires no migration.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat: adding support for commit refs in CAS' clearly summarizes the main change: adding commit ref support to the CAS system. It is specific, concise, and directly reflects the primary objective of the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/adding-support-for-commit-refs-in-cas

Comment @coderabbitai help to get the list of available commands and usage tips.

@yhakbar yhakbar force-pushed the feat/adding-support-for-commit-refs-in-cas branch from 2c10fbd to d634f65 Compare May 1, 2026 13:28
@yhakbar yhakbar marked this pull request as ready for review May 4, 2026 12:56
@yhakbar yhakbar requested a review from denis256 as a code owner May 4, 2026 12:56
@yhakbar yhakbar force-pushed the feat/adding-support-for-commit-refs-in-cas branch from d634f65 to 26e2414 Compare May 8, 2026 15:42
@yhakbar yhakbar force-pushed the feat/adding-central-git-store branch 2 times, most recently from 3a8acae to b04a540 Compare May 9, 2026 13:44
@yhakbar yhakbar force-pushed the feat/adding-support-for-commit-refs-in-cas branch from 26e2414 to 7b1bb75 Compare May 9, 2026 13:55
Comment thread internal/cas/stacks.go
Comment thread internal/cas/cas.go
Comment thread docs/src/data/changelog/v1.0.5/cas-commit-refs.mdx Outdated
Comment thread test/integration_download_test.go Outdated
Comment thread internal/cas/gitstore.go
Comment thread internal/cas/cas.go Outdated
denis256
denis256 previously approved these changes May 12, 2026
Base automatically changed from feat/adding-central-git-store to main May 12, 2026 17:26
@yhakbar yhakbar dismissed denis256’s stale review May 12, 2026 17:26

The base branch was changed.

@yhakbar yhakbar force-pushed the feat/adding-support-for-commit-refs-in-cas branch from f75cf82 to 4255f05 Compare May 12, 2026 17:28
@yhakbar yhakbar requested a review from denis256 May 12, 2026 17:29
@yhakbar yhakbar force-pushed the feat/adding-support-for-commit-refs-in-cas branch from 4255f05 to 3f73aa2 Compare May 12, 2026 17:30

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/src/content/docs/03-features/07-caching/04-cas.mdx`:
- Around line 224-227: The steps currently imply two network fetches for
commit-SHA cold-clones; update the wording so Step 1 ("Terragrunt resolves the
Git reference to a commit hash" and mention of "git ls-remote") is strictly
resolution-focused (note that ls-remote only resolves named refs and will not
fetch SHA objects), and move all fetching behavior into Step 3 (the description
that opens the bare repo under cas/store/git/, takes the per-URL lock, and
fetches—shallow for branch/tag refs, full history for commit SHAs) so that Step
1 only detects the ref type and Step 3 performs the actual network fetch; keep
references to "commit SHA", "git ls-remote", and "cas/store/git/" to locate the
lines to edit.

In `@internal/cas/gitstore.go`:
- Around line 210-212: The fallback fetch only pulls branch refs, so commits
reachable only via tags can still be missing; update the two
session.runner.Fetch calls to fetch tags as well (e.g. add
"+refs/tags/*:refs/tags/*" or use a broader refspec like "+refs/*:refs/*") so
tag-only commits are fetched and no longer return ErrNoMatchingReference; modify
the refspec strings passed to session.runner.Fetch accordingly in gitstore.go
(both occurrences of session.runner.Fetch).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3b2b4358-3e2f-4384-a2f3-deba371a9c9f

📥 Commits

Reviewing files that changed from the base of the PR and between 5aad26f and 3f73aa2.

📒 Files selected for processing (14)
  • docs/src/content/docs/03-features/07-caching/04-cas.mdx
  • docs/src/data/changelog/v1.0.5/cas-commit-refs.mdx
  • internal/cas/cas.go
  • internal/cas/cas_test.go
  • internal/cas/commitref_test.go
  • internal/cas/gitstore.go
  • internal/cas/gitstore_test.go
  • internal/cas/stacks.go
  • internal/cas/testserver_test.go
  • internal/git/errors.go
  • internal/git/git.go
  • internal/git/server.go
  • test/fixtures/download/remote-commit-ref/terragrunt.hcl
  • test/integration_download_test.go

Comment on lines +224 to 227
1. Terragrunt resolves the Git reference to a commit hash. Branch and tag refs resolve via `git ls-remote`. `ls-remote` lists named refs and does not resolve commit SHAs, so for SHA refs Terragrunt fetches the full history of every branch into the central Git store and resolves the SHA locally.
2. The tree related to the commit hash is not found in the CAS
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref. Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref (shallow for branch/tag refs, full history for commit SHAs). Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
4. All blobs and trees required to reproduce the repository are extracted from the bare repository

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify cold-clone step sequencing to avoid implying two SHA fetches.

Line 224 and Line 227 both describe fetching for commit-SHA refs; the current phrasing can read as two network fetches in one cold-clone path. Consider making Step 1 resolution-focused and keeping fetch details in Step 3.

✏️ Suggested doc tweak
-1. Terragrunt resolves the Git reference to a commit hash. Branch and tag refs resolve via `git ls-remote`. `ls-remote` lists named refs and does not resolve commit SHAs, so for SHA refs Terragrunt fetches the full history of every branch into the central Git store and resolves the SHA locally.
+1. Terragrunt resolves the Git reference to a commit hash. Branch and tag refs resolve via `git ls-remote`. For commit-SHA refs, resolution is performed locally from the central Git store (because `ls-remote` does not resolve commit SHAs).

-3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref (shallow for branch/tag refs, full history for commit SHAs). Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
+3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and performs the fetch needed for the ref kind (shallow for branch/tag refs, full history for commit SHAs). Subsequent misses against the same URL reuse existing pack files and only transfer new objects.

As per coding guidelines: "Review the documentation for clarity, grammar, and spelling. Make sure that the documentation is easy to understand and follow."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
1. Terragrunt resolves the Git reference to a commit hash. Branch and tag refs resolve via `git ls-remote`. `ls-remote` lists named refs and does not resolve commit SHAs, so for SHA refs Terragrunt fetches the full history of every branch into the central Git store and resolves the SHA locally.
2. The tree related to the commit hash is not found in the CAS
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref. Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and fetches the requested ref (shallow for branch/tag refs, full history for commit SHAs). Subsequent misses against the same URL reuse the existing pack files and only transfer new objects.
4. All blobs and trees required to reproduce the repository are extracted from the bare repository
1. Terragrunt resolves the Git reference to a commit hash. Branch and tag refs resolve via `git ls-remote`. For commit-SHA refs, resolution is performed locally from the central Git store (because `ls-remote` does not resolve commit SHAs).
2. The tree related to the commit hash is not found in the CAS
3. Terragrunt opens the bare repository for the remote URL under `cas/store/git/` (initializing it on first use), takes a per-URL lock, and performs the fetch needed for the ref kind (shallow for branch/tag refs, full history for commit SHAs). Subsequent misses against the same URL reuse existing pack files and only transfer new objects.
4. All blobs and trees required to reproduce the repository are extracted from the bare repository
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/src/content/docs/03-features/07-caching/04-cas.mdx` around lines 224 -
227, The steps currently imply two network fetches for commit-SHA cold-clones;
update the wording so Step 1 ("Terragrunt resolves the Git reference to a commit
hash" and mention of "git ls-remote") is strictly resolution-focused (note that
ls-remote only resolves named refs and will not fetch SHA objects), and move all
fetching behavior into Step 3 (the description that opens the bare repo under
cas/store/git/, takes the per-URL lock, and fetches—shallow for branch/tag refs,
full history for commit SHAs) so that Step 1 only detects the ref type and Step
3 performs the actual network fetch; keep references to "commit SHA", "git
ls-remote", and "cas/store/git/" to locate the lines to edit.

Comment thread internal/cas/gitstore.go
Comment on lines +210 to +212
if err := session.runner.Fetch(ctx, url, "+refs/heads/*:refs/heads/*", 0); err != nil {
return nil, err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Commit fallback fetch misses tag-only commits

The cache-miss path fetches only branch refs. A commit reachable only via tags can still fail resolution and be returned as ErrNoMatchingReference, even though it exists remotely.

💡 Suggested fix
-	if err := session.runner.Fetch(ctx, url, "+refs/heads/*:refs/heads/*", 0); err != nil {
+	if err := session.runner.Fetch(ctx, url, "+refs/heads/*:refs/heads/*", 0); err != nil {
+		return nil, err
+	}
+
+	if err := session.runner.Fetch(ctx, url, "+refs/tags/*:refs/tags/*", 0); err != nil {
 		return nil, err
 	}
-	if err := session.runner.Fetch(ctx, url, "+refs/heads/*:refs/heads/*", 0); err != nil {
+	if err := session.runner.Fetch(ctx, url, "+refs/heads/*:refs/heads/*", 0); err != nil {
+		return nil, err
+	}
+
+	if err := session.runner.Fetch(ctx, url, "+refs/tags/*:refs/tags/*", 0); err != nil {
 		return nil, err
 	}

Also applies to: 253-255

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/cas/gitstore.go` around lines 210 - 212, The fallback fetch only
pulls branch refs, so commits reachable only via tags can still be missing;
update the two session.runner.Fetch calls to fetch tags as well (e.g. add
"+refs/tags/*:refs/tags/*" or use a broader refspec like "+refs/*:refs/*") so
tag-only commits are fetched and no longer return ErrNoMatchingReference; modify
the refspec strings passed to session.runner.Fetch accordingly in gitstore.go
(both occurrences of session.runner.Fetch).

@yhakbar yhakbar merged commit 06c5246 into main May 12, 2026
51 of 52 checks passed
@yhakbar yhakbar deleted the feat/adding-support-for-commit-refs-in-cas branch May 12, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants