KEP for kfp helm charts#12842
Conversation
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
Signed-off-by: XploY04 <2004agarwalyash@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @XploY04. Thanks for your PR. I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
There was a problem hiding this comment.
Pull request overview
Adds a KEP-style proposal document describing an approach to introducing official Helm charts for Kubeflow Pipelines (KFP), including goals around parity with existing Kustomize manifests, chart structure, and testing strategy.
Changes:
- Introduces a new proposal document for KFP Helm charts (motivation, goals/non-goals, design details, risks, and test plan).
| @@ -0,0 +1,195 @@ | |||
| # KEP-XXXX: Helm Charts for Kubeflow Pipelines | |||
There was a problem hiding this comment.
The KEP title still uses the placeholder KEP-XXXX. Other proposals in this repo use the numeric KEP ID in the header (and typically in the proposal directory name as well), so this should be updated to the correct number (likely matching the issue/KEP ID) to align with existing proposal conventions and make the document easier to reference.
| # KEP-XXXX: Helm Charts for Kubeflow Pipelines | |
| # KEP-1234: Helm Charts for Kubeflow Pipelines |
|
|
||
| ## Summary | ||
|
|
||
| This proposal is to build a minimalistic, and maintainable Helm charts for Kubeflow Pipelines (KFP). These charts will serve as an alternative installation method to Kustomize, reflecting Kustomize defaults 1:1 to ensure consistency across deployment options. |
There was a problem hiding this comment.
Grammar issue: “a minimalistic, and maintainable Helm charts” mixes singular/plural and has an unnecessary comma. Consider rephrasing to either singular (“a minimalistic and maintainable Helm chart”) or plural (“minimalistic and maintainable Helm charts”).
| This proposal is to build a minimalistic, and maintainable Helm charts for Kubeflow Pipelines (KFP). These charts will serve as an alternative installation method to Kustomize, reflecting Kustomize defaults 1:1 to ensure consistency across deployment options. | |
| This proposal is to build minimalistic and maintainable Helm charts for Kubeflow Pipelines (KFP). These charts will serve as an alternative installation method to Kustomize, reflecting Kustomize defaults 1:1 to ensure consistency across deployment options. |
|
|
||
| ### Notes/Constraints/Caveats | ||
|
|
||
| Beyond the surface-level YAMLs, we have identified three critical implementation challenges that the Helm chart must address to be viable: |
There was a problem hiding this comment.
This section says “three critical implementation challenges” but the list contains four items (1–4). Please update the count or adjust the list so the text matches the enumerated items.
| Beyond the surface-level YAMLs, we have identified three critical implementation challenges that the Helm chart must address to be viable: | |
| Beyond the surface-level YAMLs, we have identified four critical implementation challenges that the Helm chart must address to be viable: |
| KFP has strict external dependencies, specifically **Object Storage (Minio/S3)** and **Relational Database (MySQL)**. | ||
| - The Helm chart MUST provide a mechanism to: | ||
| 1. Provision in-cluster instances of Minio/MySQL (for easy testing/dev) - possibly via subcharts like `bitnami/mysql`. | ||
| 2. Configure connection details for external managed services (RDS/S3) - using a clean `values.yaml` structure for secrets/hosts. |
There was a problem hiding this comment.
The “Dependency Management” section frames the relational DB dependency as MySQL-only, but later the proposal maps Kustomize patches from base/postgresql/... and the repo includes both MySQL and PostgreSQL install paths. To avoid confusion, clarify whether the Helm charts will support both DB backends (and which is default), or update the later examples to match the stated dependency.
| KFP has strict external dependencies, specifically **Object Storage (Minio/S3)** and **Relational Database (MySQL)**. | |
| - The Helm chart MUST provide a mechanism to: | |
| 1. Provision in-cluster instances of Minio/MySQL (for easy testing/dev) - possibly via subcharts like `bitnami/mysql`. | |
| 2. Configure connection details for external managed services (RDS/S3) - using a clean `values.yaml` structure for secrets/hosts. | |
| KFP has strict external dependencies, specifically **Object Storage (Minio/S3)** and **Relational Database (MySQL or PostgreSQL, defaulting to the same backend as the existing Kustomize manifests—currently MySQL)**. | |
| - The Helm chart MUST provide a mechanism to: | |
| 1. Provision in-cluster instances of Minio plus a relational database (MySQL/PostgreSQL) for easy testing/dev — for example via subcharts like `bitnami/mysql` or a PostgreSQL equivalent. | |
| 2. Configure connection details for external managed services (e.g., managed MySQL/PostgreSQL such as RDS, Cloud SQL, or Aurora, and object storage such as S3/GCS) — using a clean `values.yaml` structure for secrets/hosts. |
|
Thanks for putting this together this is a very thoughtful and well-structured proposal. I especially like the explicit definition of 1:1 parity and the CI diffing approach to prevent drift between Helm and Kustomize. One thought, since we’ve been emphasizing “minimal and maintainable” for the first iteration, would it make sense to explicitly define a V1 boundary (for example, single-user mode with one default DB backend) and defer multi-user + additional backends to follow-up phases? It might help us validate the Helm testing and release workflow with a smaller surface area before expanding into the more complex overlays. Curious to hear what others think about phasing this incrementally. |
|
I see multiple ones. 1. the general one kubeflow/community#832 2. KFP specific ones #12842 #12787 #12723 and some requirements from GSOC https://www.kubeflow.org/events/upcoming-events/gsoc-2026/#project-5-helm-charts " This project will touch most components and continue the helm chart initiative started by Kunal Dugar who also helped a lot with the testing infrastructure. This will therefore also include working with maintainers of other components such as KFP maintainersfor the KFP helm charts, security and scalability topic or Katib maintainers for Katib helm charts. Some have already open PRs and there was a formal vote by the KSC (Kubeflow steering Committee) that we are moving forward with offering Kubeflow platform and standalone components as helm charts. Therefore it is not just the technical part, but also the coordination effort. The goal is to make minimalistic helm charts that are easy to maintain next to kustomize and only expose sensible settings relevant to most users. For the time being the rendered chart default values must replicate kustomize 1:1. The testing infrastructure has already been set up in the GSOC 2025 efforts in kubeflow/manifests where we already have a few helm charts. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
/close |
|
@juliusvonkohout: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Description of your changes:
Fixes #12728
Checklist: