Skip to content

Interrupt pod references a metadata ConfigMap name that is never created #412

Description

@lockwobr

Problem

The interrupt pod spec declares a ConfigMap volume whose LocalObjectReference.Name is built by hand, while the ConfigMap is actually created under a generateSafeName name. The two forms can never match, because generateSafeName unconditionally appends -<8 hex chars> of a sha256 over the joined parts.

Created (operator/internal/controller/skyhook_controller.go:1610), and the same form is used by the prune set at :2396:

configMapName := generateSafeName(253, skyhook.Name, node.Name, "metadata")
// -> "<skyhook>-<node>-metadata-<8hex>"

Referenced on the interrupt path (operator/internal/controller/job_builder.go:409 on feature/package-as-jobs, operator/internal/controller/skyhook_controller.go:2362 on main):

Name: strings.ReplaceAll(fmt.Sprintf("%s-%s-metadata", skyhook.Name, nodeName), ".", "-"),
// -> "<skyhook>-<node>-metadata"   (no hash suffix)

The normal package path gets this right (job_builder.go:196) and uses generateSafeName(253, skyhook.Name, nodeName, "metadata"), matching creation.

Not a regression, and currently latent

This predates the Jobs migration. It exists on main today at skyhook_controller.go:2362; commit 87c8092 ("Job builders and stage-timeout/TTL options", #301) carried the expression into job_builder.go unchanged, so feature/package-as-jobs inherits it rather than introducing it. git log -S traces it back through fix: volume names getting longer than DNS_LABEL to begin reorg.

It appears to be latent rather than actively breaking: the interrupt pod declares this volume but no container mounts it. The interrupt path's volumeMounts contains only volumeNameRootMount, so nothing resolves the reference, which is consistent with the interrupt chainsaw suite passing today.

I did not verify on a live cluster whether kubelet materializes a declared-but-unreferenced ConfigMap volume, so "latent" is an inference from the passing e2e suite plus the missing mount, not a direct observation.

Suggested fix

Either use the same generated name the ConfigMap is created under:

Name: generateSafeName(253, skyhook.Name, nodeName, "metadata"),

or drop the volume from the interrupt pod spec entirely, since no container mounts it. The second is probably the honest fix if the interrupt stage genuinely has no use for node metadata, and it removes a broken reference rather than repairing one nothing reads.

Note the volume name also differs between the two paths, though each is internally consistent so neither is broken today: the package path uses generateSafeName(63, "metadata", nodeName) (job_builder.go:195) and the interrupt path uses generateSafeName(63, skyhook.Name, nodeName, "metadata") (job_builder.go:405), a different argument set.

Provenance

Found by CodeRabbit during a multi-reviewer cross-review of feature/package-as-jobs at a3dd68c8. Verified by inspection; the "pre-existing" and "latent" qualifications above are corrections to the original report, which framed it as a live fault introduced by the branch.

Split out of #411 so it does not gate the Jobs migration merge.

Metadata

Metadata

Assignees

No one assigned

    Labels

    component/operatorSkyhook operator (controller-manager)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions