Recursive Read-only (RRO) mounts

[AkihiroSuda]

Enhancement Description

• One-line enhancement description (can be used as a release note): Recursive Read-only (RRO) mounts

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/3857-rro-mounts

• Discussion Link: none

• Primary contact (assignee): @AkihiroSuda

• Responsible SIGs: SIG-node

• Enhancement target (which target equals to which milestone):

  • Alpha release target (x.y): v1.30
  • Beta release target (x.y): v1.31
  • Stable release target (x.y): v1.33

• Alpha

  • KEP (k/enhancements) update PR(s): KEP-3857: Recursive Read-only (RRO) mounts #3858
  • Code (k/k) update PR(s):
    • cri-api: KEP-3857: Recursive Read-only (RRO) mounts [Split from #123180 for ease of review] kubernetes#123272
    • KEP-3857: Recursive Read-only (RRO) mounts  kubernetes#123180
    • KEP-3857: Recursive Read-only (RRO) mounts kubernetes-sigs/cri-tools#1344
    • KEP-3857: Recursive Read-only (RRO) mounts containerd/containerd#9787
  • Docs (k/website) update PR(s):
    • [dev-1.30] KEP-3857: Recursive Read-only (RRO) mounts website#45159
    • blog post for KEP-3857: Recursive Read-only (RRO) mounts website#45238

• Beta

  • KEP (k/enhancements) update PR(s): KEP-3857: Recursive Read-only (RRO) mounts: promote to Beta #4668
  • Code (k/k) update PR(s): KEP-3857: promote RecursiveReadOnlyMounts feature to beta kubernetes#125475
  • Docs (k/website) update PR(s):
    • KEP-3857: promote RecursiveReadOnlyMounts feature to beta website#48615

• Stable

  • KEP (k/enhancements) update PR(s): KEP-3857: Recursive Read-only (RRO) mounts: promote to GA #5157
  • Code (k/k) update PR(s): KEP-3857: Recursive Read-only (RRO) mounts: promote to GA kubernetes#130116
  • Docs (k/website) update(s): KEP-3857: Recursive Read-only (RRO) mounts: promote to GA website#49866

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[AkihiroSuda]

/sig node

[SergeyKanzhelev]

adding sig storage as participating sig

/sig storage

[SergeyKanzhelev]

@AkihiroSuda do you want to target this KEP for 1.28?

[AkihiroSuda]

@AkihiroSuda do you want to target this KEP for 1.28?

Not necessary.

[AkihiroSuda]

@kubernetes/sig-node-leads
Any chance to get this lead-opted-in for v1.29?

[AkihiroSuda]

ping @kubernetes/sig-node-leads 🙏

[AkihiroSuda]

@kubernetes/sig-node-leads
Will this be considered for lead-opted-in in the v1.30 milestone?

[AkihiroSuda]

cc @mrunalp @derekwaynecarr @dchen1107 @SergeyKanzhelev for the question above about v1.30 milestone

[pacoxu]

+1 for v1.30

• We may tag this as v1.30 and get some early input.

adding sig storage as participating sig

@SergeyKanzhelev do we have to check with sig storage leads about this KEP?

[AkihiroSuda]

@mrunalp @derekwaynecarr @dchen1107 @SergeyKanzhelev

PTAL.
If this KEP isn't going to be accepted, I'll just modify the default behavior of containerd to treat "readonly" as "recursive readonly".
Small portion of users may find this to be a breaking change, but probably this is subtle enough.
#3858 (comment)

[carlory]

@SergeyKanzhelev added sig storage as participating sig for this KEP. The proposal makes sense to me, but I still want to hear from the sig storage members. cc @jsafrane @msau42 @xing-yang Can you please review the proposal and provide your feedback? Thanks! If accepted, this KEP may be able to have an alpha implement in 1.30.

[AkihiroSuda]

Didn't get lead-opted-in. Giving up.

Alternative:

• [Reverted in #9747] cri: make read-only mounts recursively read-only containerd/containerd#9713

[AkihiroSuda]

Reopening

#3858 (comment)

[mrunalp]

/milestone 1.30
/stage alpha
/label lead-opted-in

[k8s-ci-robot]

@mrunalp: The provided milestone is not valid for this repository. Milestones in this repository: [v1.25, v1.27, v1.28, v1.29, v1.30, v1.31]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.30
/stage alpha
/label lead-opted-in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tjons]

Hi, enhancements lead here - I inadvertently added this to the 1.32 tracking board 😀. Please readd it if you wish to progress this enhancement in 1.32.

/remove-label lead-opted-in

[haircommander]

@AkihiroSuda are you planning on pushing this forward in 1.33?

[AkihiroSuda]

@AkihiroSuda are you planning on pushing this forward in 1.33?

I'm not sure.

The original criteria to graduate this from Beta was:

enhancements/keps/sig-node/3857-rro-mounts/README.md

Lines 712 to 714 in 62cad7d

	#### GA
	- Two beta releases of Kubernetes at least
	- containerd, CRI-O, and cri-dockerd supports the feature with their GA releases

However, cri-dockerd with this feature is still not released, although the feature has been implemented in its master branch.

• KEP-3857: Recursive Read-only (RRO) mounts Mirantis/cri-dockerd#370

Can we just skip cri-dockerd?

[haircommander]

Can we just skip cri-dockerd?

yeah our guidance now is containerd and cri-o are required kubernetes/contributor-site#566 . SIG node would be open to additional runtimes but the expectation is they participate more in the SIG to block the progress of features

[AkihiroSuda]

The test (https://github.com/kubernetes/kubernetes/blob/v1.33.0-alpha.1/test/e2e_node/mount_rro_linux_test.go) seems still skipped in ci-kubernetes-e2e-kind-beta-features, although the kind nodes has been updated to use containerd v2. 🤔

https://storage.googleapis.com/kubernetes-ci-logs/logs/ci-kubernetes-e2e-kind-beta-features/1887785121608634368/artifacts/junit_01.xml

The graduation might need to be postponed to v1.34.

[AkihiroSuda]

Opened an issue:

• ci-kubernetes-e2e-kind-beta-features seems silently skipping RecursiveReadOnlyMounts (beta) tests test-infra#34303

[AkihiroSuda]

Proposal: promote to GA

• KEP-3857: Recursive Read-only (RRO) mounts: promote to GA #5157

[soltysh]

Proposal: promote to GA

* [KEP-3857: Recursive Read-only (RRO) mounts: promote to GA #5157](https://github.com/kubernetes/enhancements/pull/5157)

I'll take a look from PRR perspective, but you need to ping sig-node leads to get this included in 1.33 release, it is not currently.

[AkihiroSuda]

ping @kubernetes/sig-node-leads

[haircommander]

/label lead-opted-in
/milestone v1.33

[jenshu]

Hello @AkihiroSuda 👋, v1.33 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th February 2025 / 19:00 PDT Thursday 13th February 2025.

This enhancement is targeting stage stable for v1.33 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.33.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 6th February 2025 so that the PRR team has enough time to review your KEP.

For this KEP, we would just need to update the following:

• In the kep.yaml, the latest-milestone needs to be updated to v1.33
• In the PRR file, an approver for stable needs to be listed. Once your PR merges, both of these should be resolved.

The status of this enhancement is marked as At risk for enhancements freeze. Please keep the issue description up-to-date with appropriate stages as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[jenshu]

Hello @AkihiroSuda 👋, v1.33 Enhancements team here.

With all the requirements fulfilled, this enhancement is now marked as tracked for the upcoming enhancements freeze 🚀

[rayandas]

Hello @AkihiroSuda 👋, v1.33 Docs Lead here.

Does this enhancement work planned for v1.33 require any new docs or modification to existing docs?

If so, please follow the steps here to open a PR against dev-1.33 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 27th February 2025 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[AkihiroSuda]

Docs

Here:

• KEP-3857: Recursive Read-only (RRO) mounts: promote to GA website#49866

[rayandas]

Thanks @AkihiroSuda