Migrate Flux controllers to use runtime/secrets package

[cappyzawa]

Migrate Flux controllers to use runtime/secrets package

Background

We have successfully consolidated secret handling patterns in the Flux ecosystem by creating a new runtime/secrets package in fluxcd/pkg. This package provides unified functions for handling Kubernetes secrets across all Flux components.

Key developments:

• Issue fluxcd/pkg#949: Consolidate the handling of secrets
• PR fluxcd/pkg#950: Add runtime/secrets package (merged)
• Issue fluxcd/pkg#953: Add legacy field logging
• PR fluxcd/pkg#955: Add legacy field logging to TLS functions (merged)
• Release: fluxcd/pkg/runtime v0.62.0 now available for use across Flux

New runtime/secrets Package

The runtime/secrets package provides:

Core Functions

• TLSConfigFromSecret(ctx, client, name, namespace, logger) - Creates TLS config with legacy field support
• ProxyURLFromSecret(ctx, client, name, namespace) - Extracts proxy configuration
• BasicAuthFromSecret(ctx, client, name, namespace) - Retrieves basic auth credentials
• PullSecretsFromServiceAccount(ctx, client, name, namespace) - Resolves image pull secrets

Helper Functions

• MakeTLSSecret(), MakeProxySecret(), MakeBasicAuthSecret(), etc.

Key Features

• Legacy Field Support: Always supports deprecated field names (certFile, keyFile, caFile) for backward compatibility
• Standard Field Priority: Standard Kubernetes field names (tls.crt, tls.key, ca.crt) take precedence
• Migration Logging: Warns when legacy fields are used to encourage migration
• Consistent API: Same behavior across all Flux components

Migration Goals

1. Consistency: Ensure all Flux controllers handle secrets the same way
2. Maintainability: Reduce code duplication across repositories
3. User Experience: Provide consistent secret field support and clear migration paths
4. Future-proofing: Prepare for eventual v2 API transition

Target Repositories

The following repositories contain secret handling functionality that can benefit from migrating to runtime/secrets:

• source-controller: Repository-type-specific migration with authentication patterns
  • HelmRepository controller migration (Migrate HelmRepository to runtime/secrets source-controller#1849)
  • OCIRepository controller migration (Migrate OCIRepository controller to runtime/secrets source-controller#1851)
  • Bucket controller migration (Migrate Bucket controller to runtime/secrets source-controller#1852)
  • GitRepository controller migration
    • Migration not required: Git authentication uses the established pkg/git module with git.NewAuthOptions(), which provides equivalent functionality to runtime/secrets. The pkg/git module is specifically designed for Git authentication and is used consistently across the Flux ecosystem (source-controller, image-automation-controller) for Git operations.
• kustomize-controller: TLS certificates for webhook validation, SOPS decryption keys
  • Migration not required: kustomize-controller only handles secrets for SOPS decryption, which uses file extension-based processing (.asc, .agekey, sops.aws-kms, etc.) rather than standard Kubernetes secret field names (tls.crt, tls.key, ca.crt, username, password). This unique requirement makes the controller incompatible with the runtime/secrets package's field-based approach, as SOPS expects specific file names for different encryption methods rather than standardized secret fields.
• helm-controller: TLS certificates for Helm repositories, authentication secrets
  • Migration not required: helm-controller delegates all authentication (TLS/BasicAuth/Proxy) to source-controller and only consumes authenticated artifacts. The controller handles KubeConfig secrets and values injection, which are outside the scope of runtime/secrets package.
• notification-controller: Core runtime/secrets migration completed
  • ProxySecretRef implementation (Add ProxySecretRef field to Provider API notification-controller#1133)
  • mTLS support for postMessage-based notifiers (Add mTLS support for postMessage-based notifiers notification-controller#1137)
  • BasicAuth processing unification (Unify BasicAuth processing using pkg/runtime/secrets notification-controller#1139)
  • Proxy URL parsing optimization (planned for Flux v1)
    • We'll address this in Remove deprecated proxy fields from Provider API notification-controller#1144.
  • Git-based notifiers mTLS support (Add mTLS support for git-based notifiers notification-controller#1146)
  • Other notifiers (DataDog, Sentry)
• image-automation-controller: Git authentication secrets
  • Migration not required: Git authentication already uses the established pkg/git module with git.NewAuthOptions(), which provides equivalent functionality to runtime/secrets for Git-specific authentication patterns. The pkg/git module is specifically designed for Git authentication and is used consistently across the Flux ecosystem for Git operations.
• image-reflector-controller: Registry authentication for image scanning
  • Core pkg/runtime/secrets migration (Migrate secrets handling to pkg/runtime/secrets image-reflector-controller#791)
    • Migrated TLS certificate handling to TLSConfigFromSecret()
    • Migrated proxy configuration to ProxyURLFromSecret()
  • ServiceAccount ImagePullSecrets migration (Migrate secrets handling to pkg/runtime/secrets image-reflector-controller#791)
    • Migrate ServiceAccount ImagePullSecrets handling to PullSecretsFromServiceAccount()
    • Maintain integration with go-containerregistry k8schain
    • Complete cleanup of any remaining custom secret handling code
• flux2: CLI utilities for creating and managing secrets
  • Git authentication commands (flux bootstrap, flux create secret git)
    • Migration not required: These commands already use the established pkg/git module for Git authentication, providing equivalent functionality to runtime/secrets for Git operations.
  • Other secret management utilities (TLS, registry, etc.) (Migrate sourcesecret package to runtime/secrets APIs #5462)

mTLS Terminology and Consistency

Based on the implementation in notification-controller PR #1137, we've established important guidelines for mTLS terminology:

Historical Context

• Before Flux 2.6: The term "mTLS" or "mutual" was not used in documentation

  • Example: imagerepositories/#certificate-secret-reference
  • Example: ocirepositories/#cert-secret-reference
  • Example: buckets/#cert-secret-reference
  • Example: helmrepositories/#cert-secret-reference

• Flux 2.6: First mention of "mutual TLS"

  • gitrepositories/#https-mutual-tls-authentication

Decision

We will maintain consistency with pkg/runtime/secrets definition of mTLS:

• mTLS configuration: tls.crt + tls.key + optional ca.crt
• This definition should be used consistently across all controllers when implementing TLSConfigFromSecret

[cappyzawa]

I'd like to take on this migration work. Could someone please assign this issue to me?

[cappyzawa]

Update: mTLS Terminology and GitRepository Considerations

Based on the review of notification-controller PR #1137, I'm updating this issue with important findings:

mTLS Terminology and Consistency

Based on the implementation in notification-controller PR #1137, we've established important guidelines for mTLS terminology:

Historical Context

• Before Flux 2.6: The term "mTLS" or "mutual" was not used in documentation

  • Example: imagerepositories/#certificate-secret-reference
  • Example: ocirepositories/#cert-secret-reference
  • Example: buckets/#cert-secret-reference
  • Example: helmrepositories/#cert-secret-reference

• Flux 2.6: First mention of "mutual TLS"

  • gitrepositories/#https-mutual-tls-authentication

Decision

We will maintain consistency with pkg/runtime/secrets definition of mTLS:

• mTLS configuration: tls.crt + tls.key + optional ca.crt
• This definition should be used consistently across all controllers when implementing TLSConfigFromSecret

GitRepository Special Considerations

GitRepository requires special handling:

• Unique pattern: Unlike other APIs, GitRepository uses spec.secretRef instead of spec.certSecretRef
• Reason: It's the only API that supports both HTTPS and SSH authentication
• Migration approach: Leave GitRepository for last to get maintainer input (@stefanprodan) on whether to:
  1. Introduce spec.certSecretRef for consistency
  2. Keep the current spec.secretRef approach
• Note: We may not be able to fully leverage TLSConfigFromSecret in this API due to its unique requirements

cc: @matheuscscp

[matheuscscp]

I realized that for some cases (e.g. auth handling in the NC Provider API and SC GitRepository API), spec.secretRef is fetched and then a bunch of auth keys are attempted. This does not work well for the reader functions we added in runtime/secrets, as each of those functions expect to fetch the secret themselves and then read a specific set of keys. For those APIs we have two alternatives:

• Do not use secrets in those specific cases (but, of course, still use it for .spec.certSecretRef and .spec.proxySecretRef where applicable).
• Adapt runtime/secrets to support those cases as well, e.g. introducing *FromSecretData variants that simply receive map[string][]byte and return (thing, bool, error) instead of (thing, error) with the boolean telling if the keys that function was expecting were there. The variant still needs to return an error in case the keys are not consistent e.g. tls.crt is present but tls.key is not.

[matheuscscp]

One more use case has arrived for runtime/secrets:

I'd like to use secrets.TLSConfigFromSecret() to implement this feature:

#5447

This, however, will be implemented in the github.com/fluxcd/pkg/auth package, and it cannot depend on github.com/fluxcd/pkg/runtime/secrets, so we need to move the secrets library outside runtime and make it its own Go module.

[stefanprodan]

I'd like to use secrets.TLSConfigFromSecret() to implement this feature:

Why don't you just take the tls config as input the auth package?

[matheuscscp]

auth/utils has a utility function for converting meta.KubeConfigReference into []auth.Option (auth can take *tls.Config as input, yes)

[cappyzawa]

Following up on the discussion around spec.secretRef patterns and the double Secret fetch issue - after implementing fluxcd/notification-controller#1142, I've been thinking about the broader implications for the migration effort.

@matheuscscp's earlier proposal for *FromSecretData variants is spot on for addressing the performance concerns. But since we're doing a comprehensive migration across all controllers AND planning to move the secrets package outside of runtime (as mentioned for #5447), I wonder if we should consider taking this a step further and redesign the runtime/secrets interface entirely.

What if we make Secret pointer-based functions the primary interface? This would:

1. Eliminate performance issues: No risk of double fetches - callers control Secret fetching
2. Clean separation of concerns: Secret fetching vs. data processing are completely separated
3. Better optimization opportunities: Controllers can batch Secret fetches, implement caching, etc.
4. Access to full Secret context: SecretType validation, metadata, annotations for future extensibility

The API would look like:

// New module (e.g., github.com/fluxcd/pkg/secrets)
// Primary interface - Secret object processing
func TLSConfigFromSecret(secret *corev1.Secret, opts ...Option) (*tls.Config, bool, error)
func BasicAuthFromSecret(secret *corev1.Secret) (string, string, bool, error)
func ProxyURLFromSecret(secret *corev1.Secret) (*url.URL, bool, error)

// Convenience wrappers (if still needed)
func TLSConfigFromSecretRef(ctx context.Context, c client.Client, name, namespace string, opts ...Option) (*tls.Config, error)

Since we're planning both a major migration AND moving the package outside of runtime, this might be the perfect opportunity to get the design right from the start. The key insight is that with a new module path, we can use the same function names but with a better interface design - no breaking changes for existing code, but a clean foundation for the future.

This approach would also naturally solve the spec.secretRef patterns we've been discussing without needing separate *FromSecretData variants.

Thoughts?

[matheuscscp]

@cappyzawa This looks very good to me 👍

Just one question about opts ...Option in TLS: I think we got rid of options for this entirely? What's the use case?

And about this:

no breaking changes for existing code

No need to worry about this at all, as you know runtime is not GA, we are at v0.x right now 👍 And depending on me this package will never be GA

[cappyzawa]

@matheuscscp Thanks for your positive feedback!

You're right, opts was removed, so it's not needed! Sorry about that.

Let me start working on the new secrets module! 💪