Fix filestore Retrieve() failing when local cache directories are missing

[concaf]

Summary

• Retrieve() downloads files from S3 when the local cache is empty, but os.WriteFile fails if the parent directory doesn't exist (e.g. after a PVC delete and pod restart)
• Adds os.MkdirAll before the write, matching what Store() already does
• Without this fix, all three pipeline stages (DocumentProcessor, ChunksGenerator, VectorEmbeddingsGenerator) enter a retry loop with ~12K errors because they can't write S3 data back to the local cache

Test plan

• go build ./pkg/filestore/ passes
• go vet ./pkg/filestore/ passes
• Delete PVC, restart pod, verify controller recovers by re-downloading from S3

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a runtime error in the filestore component where file retrieval fails if the local cache directory structure is missing, such as after a pod restart. By ensuring the directory path exists before attempting to write downloaded data, the system now correctly recovers from missing local state.

Highlights

• Directory Creation: Added os.MkdirAll to ensure parent directories exist before writing files to the local cache.
• Error Prevention: Resolved an issue where Retrieve() would fail when local cache directories were missing, preventing unnecessary retry loops in pipeline stages.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates the file retrieval process in pkg/filestore/filestore.go to automatically create parent directories if they do not exist before writing the file locally. The reviewer identified a critical path traversal vulnerability where the target path could escape the root directory, and provided a code suggestion to validate the path and simplify a redundant directory check.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The path parameter is joined with fs.root using filepath.Join, which does not prevent directory traversal if path contains relative directory segments (e.g., ../../). This could allow writing files or creating directories outside the intended fs.root directory (Path Traversal).

To mitigate this, we should validate that the resolved localPath is indeed within fs.root before creating directories or writing the file. Additionally, filepath.Dir never returns an empty string (it returns . if empty), so the dir != "" check is redundant and can be simplified.

Suggested change

	if dir := filepath.Dir(localPath); dir != "" {
	if err := os.MkdirAll(dir, 0755); err != nil {
	return nil, err
	}
	}
	// Ensure the local path is within the root directory to prevent path traversal
	rel, err := filepath.Rel(fs.root, localPath)
	if err != nil || (len(rel) >= 2 && rel[0] == '.' && rel[1] == '.') {
	return nil, fmt.Errorf("path escapes root directory: %s", path)
	}
	// write to local filesystem — create parent directories if they don't exist
	dir := filepath.Dir(localPath)
	if err := os.MkdirAll(dir, 0755); err != nil {
	return nil, err
	}