K8SPS-567: automatic yaml generation #1136

pooknull · 2025-10-22T12:37:38Z

https://perconadev.atlassian.net/browse/K8SPS-567

DESCRIPTION

This PR introduces automatic generation of the example cr.yaml, backup.yaml, restore.yaml files.

How it works:

A new binary cmd/example-gen generates a populated PerconaServerMySQL/PerconaServerMySQLBackup/PerconaServerMySQLRestore resources.
The script cmd/example-gen/generate.sh:
- sorts fields in a predefined order,
- removes unnecessary fields,
- comments out predefined values.

When a new field is added to PerconaServerMySQL/PerconaServerMySQLBackup/PerconaServerMySQLRestore, it will automatically appear in the example file after running make manifests.

By default, the field will:

appear at the end of its parent structure,
and contain a zero value (even if omitempty is used).

To adjust it:

Order: add the field to the sort_yaml function.
Default value: update example-gen/pkg/defaults/manual.go.
Shared defaults for multiple types: modify example-gen/pkg/defaults/preset.go.

Additional changes:

VolumeSpec was moved from PodSpec to MySQLSpec as it is only used in the MySQL
Removed StorageClass from BackupStorageS3Spec as it is not used

CHECKLIST

Jira

Is the Jira ticket created and referenced properly?
Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

Is an E2E test/test case added for the new feature/change?
Are unit tests added where appropriate?

Config/Logging/Testability

Are all needed new/changed options added to default YAML files?
Are all needed new/changed options added to the Helm Chart?
Did we add proper logging messages for operator actions?
Did we ensure compatibility with the previous version or cluster upgrade process?
Does the change support oldest and newest supported PS version?
Does the change support oldest and newest supported Kubernetes version?

Copilot

Pull Request Overview

This PR introduces automated generation of the example deploy/cr.yaml file for the PerconaServerMySQL operator. The solution consists of a Go binary (cmd/example-gen) that generates a populated resource and a bash script that formats, sorts, and comments the output. This automation ensures that new API fields automatically appear in the example configuration after running make manifests.

Key changes include:

New generation tooling with custom marshaling to ignore omitempty tags
Removal of deprecated StorageClass field from S3 backup specifications
Relocation of VolumeSpec from PodSpec to MySQLSpec to fix structural issues

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
cmd/example-gen/main.go	Entry point for CR generation binary
cmd/example-gen/generate.sh	Bash script to format, sort, and comment generated YAML
cmd/example-gen/defaults/*.go	Default value configuration for manual and preset fields
cmd/example-gen/internal/marshal/marshal.go	Custom JSON marshaler ignoring omitempty tags
cmd/example-gen/internal/fill/bytype.go	Type-based field filling logic
api/v1/perconaservermysql_types.go	API changes: VolumeSpec relocation and StorageClass removal
deploy/cr.yaml	Regenerated example configuration
Multiple CRD files	Updated schemas reflecting API changes
Test files	Updated test fixtures for VolumeSpec relocation
Makefile	Added generate-cr-yaml target to manifest generation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

api/v1/perconaservermysql_types.go

cmd/example-gen/generate.sh

cmd/example-gen/defaults/preset.go

https://perconadev.atlassian.net/browse/K8SPS-567

Co-authored-by: Copilot <[email protected]>

Copilot

Pull Request Overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

hors · 2025-10-22T15:26:46Z

deploy/cr.yaml

 #    proxy: false
 #    proxySize: false
-#  pause: false
+#  pause: true


@pooknull, we need to have default values in commented options

Copilot

Pull Request Overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

cmd/example-gen/defaults/manual.go

cmd/example-gen/internal/fill/defaults.go

Copilot

Pull Request Overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull Request Overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

cmd/example-gen/scripts/lib/util.sh:1

This complex awk script that transforms diff hunks lacks inline comments explaining its logic. Add comments describing: (1) what 'has_lt', 'has_gt', 'has_sep' track, (2) why delete hunks are converted to change hunks, and (3) how lines are prefixed with '> #' for commenting. This would significantly improve maintainability.

#!/usr/bin/env bash

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

cmd/example-gen/internal/marshal/marshal.go

cmd/example-gen/pkg/defaults/manual.go

Copilot

Pull Request Overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

api/v1/perconaservermysql_types.go

cmd/example-gen/scripts/lib/util.sh

Copilot

Pull Request Overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull Request Overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

egegunes · 2025-10-28T10:12:01Z

I think this PR should also add a job to github actions to check if example cr.yaml is updated like we do for manifests.

pooknull · 2025-10-28T12:03:51Z

@egegunes it is already done as part of manifests job: https://github.com/percona/percona-server-mysql-operator/actions/runs/18874012303/job/53859112773?pr=1136

This reverts commit ed6a0a0.

Copilot

Pull Request Overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings October 22, 2025 12:37

pull-request-size bot added the size/XXL 1000+ lines label Oct 22, 2025

Copilot AI reviewed Oct 22, 2025

View reviewed changes

api/v1/perconaservermysql_types.go Show resolved Hide resolved

cmd/example-gen/generate.sh Outdated Show resolved Hide resolved

cmd/example-gen/defaults/preset.go Outdated Show resolved Hide resolved

K8SPS-567: automatic cr.yaml generation

88305af

https://perconadev.atlassian.net/browse/K8SPS-567

pooknull force-pushed the K8SPS-567 branch from e3c2be6 to 88305af Compare October 22, 2025 12:39

Update cmd/example-gen/defaults/preset.go

afa24ce

Co-authored-by: Copilot <[email protected]>

Copilot AI review requested due to automatic review settings October 22, 2025 12:40

Copilot AI reviewed Oct 22, 2025

View reviewed changes

hors requested changes Oct 22, 2025

View reviewed changes

fix commenting

a4d6c96

Copilot AI review requested due to automatic review settings October 27, 2025 12:32

Copilot AI reviewed Oct 27, 2025

View reviewed changes

cmd/example-gen/defaults/manual.go Outdated Show resolved Hide resolved

cmd/example-gen/defaults/manual.go Outdated Show resolved Hide resolved

cmd/example-gen/internal/fill/defaults.go Outdated Show resolved Hide resolved

small fixes

a576772

pooknull force-pushed the K8SPS-567 branch from b3fdb3d to a576772 Compare October 27, 2025 13:36

reduce size of manual.go

48ef7b8

Copilot AI review requested due to automatic review settings October 27, 2025 14:47

Copilot AI reviewed Oct 27, 2025

View reviewed changes

pooknull added 2 commits October 28, 2025 11:02

generation of backups and restores

75a86cb

fix shellcheck

31368ed

Copilot AI review requested due to automatic review settings October 28, 2025 09:14

Copilot AI reviewed Oct 28, 2025

View reviewed changes

cmd/example-gen/internal/marshal/marshal.go Show resolved Hide resolved

cmd/example-gen/pkg/defaults/manual.go Show resolved Hide resolved

pooknull added 2 commits October 28, 2025 11:25

fix shellcheck

231116d

fix shellcheck

de4116b

Copilot AI review requested due to automatic review settings October 28, 2025 09:38

Copilot AI reviewed Oct 28, 2025

View reviewed changes

api/v1/perconaservermysql_types.go Show resolved Hide resolved

cmd/example-gen/scripts/lib/util.sh Show resolved Hide resolved

pooknull added 2 commits October 28, 2025 11:44

fix shellcheck

168ea42

fix shellcheck

918120e

Copilot AI review requested due to automatic review settings October 28, 2025 09:49

Copilot AI reviewed Oct 28, 2025

View reviewed changes

Merge branch 'main' into K8SPS-567

708e8ff

pooknull requested a review from hors October 28, 2025 09:57

pooknull marked this pull request as ready for review October 28, 2025 09:57

pooknull requested a review from egegunes as a code owner October 28, 2025 09:57

Copilot AI review requested due to automatic review settings October 28, 2025 09:57

pooknull requested review from gkech and nmarukovich as code owners October 28, 2025 09:57

pooknull changed the title ~~K8SPS-567: automatic cr.yaml generation~~ K8SPS-567: automatic yaml generation Oct 28, 2025

Copilot AI reviewed Oct 28, 2025

View reviewed changes

check

ed6a0a0

Revert "check"

3c20bca

This reverts commit ed6a0a0.

Copilot AI review requested due to automatic review settings October 28, 2025 12:04

Copilot AI reviewed Oct 28, 2025

View reviewed changes

Uh oh!

K8SPS-567: automatic yaml generation #1136

Are you sure you want to change the base?

K8SPS-567: automatic yaml generation #1136

Conversation

pooknull commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DESCRIPTION

How it works:

Additional changes:

CHECKLIST

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

hors Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

egegunes commented Oct 28, 2025

Uh oh!

pooknull commented Oct 28, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pooknull commented Oct 22, 2025 •

edited

Loading