CLOUDP-317663: Archive snippets outputs instead of comitting

[lsierant]

Summary

With this PR we will no longer commit snippets outputs back to the repo and just archive them to s3 instead.

Additionally, scripts/dev/update_docs_snippets.sh scripts was updated. But automated snippets updates from CI runs are not part of this PR.

When reviewing collapse changes to tests/output directory - all those files (previously comitted outputs) were just deleted.

Proof of Work

Archiving outputs to s3
manual evg run

locally executed update script

$ MCK_DIR=$(pwd) MCK_BRANCH=lsierant/archive-snippets-outputs DOCS_DIR=~/mdb/docs-mongodb-internal OUTPUTS_VERSION_ID=69038706b0fce50007f25a9d scripts/dev/update_docs_snippets.sh

~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
~/mdb/docs-mongodb-internal ~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
HEAD is now at 308db2bd35 DOCSP-54633 MongoDB Search product name change-main-backport (15252) (#15805)
fatal: a branch named 'MCK-snippets-update-69038706b0fce50007f25a9d' already exists
Switched to branch 'MCK-snippets-update-69038706b0fce50007f25a9d'
~/mdb/lsierant ~/mdb/lsierant/lsierant_archive-snippets-outputs
Downloading snippets outputs from s3 to /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_community_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_community_snippets.tgz
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_external_mongod_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_external_mongod_snippets.tgz
download: s3://operator-e2e-artifacts/snippets_outputs/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_enterprise_snippets.tgz to ../../../../var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/snippets_outputs_69038706b0fce50007f25a9d_test_kind_search_enterprise_snippets.tgz
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0110_wait_for_operator_deployment.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0315_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0330_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/01_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_community_snippets/03_0455_execute_vector_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0310_wait_for_database_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/02_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets/03_0455_execute_vector_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0090_helm_add_mogodb_repo.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0100_install_operator.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0315_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0330_wait_for_community_resource.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/04_0335_show_running_pods.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0420_import_movies_mflix_database.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0440_wait_for_search_index_ready.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0444_list_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0445_list_vector_search_indexes.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0450_execute_search_query.out
x scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets/03_0455_execute_vector_search_query.out
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_community_snippets
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_enterprise_snippets
Replacing outputs for test: /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs/test_kind_search_external_mongod_snippets
/var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs
/var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d/scripts/code_snippets/tests/outputs
├── test_kind_search_community_snippets
│   ├── 01_0090_helm_add_mogodb_repo.out
│   ├── 01_0100_install_operator.out
│   ├── 01_0110_wait_for_operator_deployment.out
│   ├── 01_0315_wait_for_community_resource.out
│   ├── 01_0330_wait_for_community_resource.out
│   ├── 01_0335_show_running_pods.out
│   ├── 03_0420_import_movies_mflix_database.out
│   ├── 03_0440_wait_for_search_index_ready.out
│   ├── 03_0444_list_search_indexes.out
│   ├── 03_0445_list_vector_search_indexes.out
│   ├── 03_0450_execute_search_query.out
│   └── 03_0455_execute_vector_search_query.out
├── test_kind_search_enterprise_snippets
│   ├── 02_0090_helm_add_mogodb_repo.out
│   ├── 02_0100_install_operator.out
│   ├── 02_0310_wait_for_database_resource.out
│   ├── 02_0335_show_running_pods.out
│   ├── 03_0420_import_movies_mflix_database.out
│   ├── 03_0440_wait_for_search_index_ready.out
│   ├── 03_0444_list_search_indexes.out
│   ├── 03_0445_list_vector_search_indexes.out
│   ├── 03_0450_execute_search_query.out
│   └── 03_0455_execute_vector_search_query.out
└── test_kind_search_external_mongod_snippets
    ├── 03_0420_import_movies_mflix_database.out
    ├── 03_0440_wait_for_search_index_ready.out
    ├── 03_0444_list_search_indexes.out
    ├── 03_0445_list_vector_search_indexes.out
    ├── 03_0450_execute_search_query.out
    ├── 03_0455_execute_vector_search_query.out
    ├── 04_0090_helm_add_mogodb_repo.out
    ├── 04_0100_install_operator.out
    ├── 04_0315_wait_for_community_resource.out
    ├── 04_0330_wait_for_community_resource.out
    └── 04_0335_show_running_pods.out

4 directories, 33 files
~/mdb/docs-mongodb-internal /var/folders/_l/xlckyyss4nsf1hd_0wf1pt3r0000gp/T/tmp.f9z7UeJ2Te/69038706b0fce50007f25a9d ~/mdb/lsierant/lsierant_archive-snippets-outputs
[MCK-snippets-update-69038706b0fce50007f25a9d 5198ea9981] Update sample files from MCK
 18 files changed, 78 insertions(+), 623 deletions(-)
 delete mode 100644 content/kubernetes/upcoming/source/includes/code-examples/search/04-search-external-mongod/README.md
Enumerating objects: 57, done.
Counting objects: 100% (57/57), done.
Delta compression using up to 16 threads
Compressing objects: 100% (34/34), done.
Writing objects: 100% (34/34), 4.72 KiB | 4.72 MiB/s, done.
Total 34 (delta 30), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (30/30), completed with 21 local objects.
remote:
remote: Create a pull request for 'MCK-snippets-update-69038706b0fce50007f25a9d' on GitHub by visiting:
remote:      https://github.com/10gen/docs-mongodb-internal/pull/new/MCK-snippets-update-69038706b0fce50007f25a9d
remote:
remote: GitHub found 264 vulnerabilities on 10gen/docs-mongodb-internal's default branch (25 critical, 106 high, 86 moderate, 47 low). To find out more, visit:
remote:      https://github.com/10gen/docs-mongodb-internal/security/dependabot
remote:
To github.com:10gen/docs-mongodb-internal.git
 * [new branch]            MCK-snippets-update-69038706b0fce50007f25a9d -> MCK-snippets-update-69038706b0fce50007f25a9d

Branch was automatically pushed with all gathered snippets and outputs:
https://github.com/10gen/docs-mongodb-internal/compare/MCK-snippets-update-69038706b0fce50007f25a9d?expand=1

[github-actions]

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.6.0 Release Notes

New Features

• MongoDBCommunity: Added support to configure custom cluster domain via newly introduced spec.clusterDomain resource field. If spec.clusterDomain is not set, environment variable CLUSTER_DOMAIN is used as cluster domain. If the environment variable CLUSTER_DOMAIN is also not set, operator falls back to cluster.local as default cluster domain.
• Helm Chart: Introduced two new helm fields operator.podSecurityContext and operator.securityContext that can be used to configure securityContext for Operator deployment through Helm Chart.

Bug Fixes

• Fixed parsing of the customEnvVars Helm value when values contain = characters.
• ReplicaSet: Blocked disabling TLS and changing member count simultaneously. These operations must now be applied separately to prevent configuration inconsistencies.

Other Changes

• kubectl-mongodb plugin: cosign, the signing tool that is used to sign kubectl-mongodb plugin binaries, has been updated to version 3.0.2. With this change, released binaries will be bundled with .bundle files containing both signature and certificate information. For more information on how to verify signatures using new cosign version please refer to -> https://github.com/sigstore/cosign/blob/v3.0.2/doc/cosign_verify-blob.md

[MaciejKaras]

Great change!

Couple of questions though:

• I see in the code that we are uploading code_snippets output for every patch, including PRs etc. Do we actually need to upload code snippets for PRs? Do we have a cleanup mechanism in S3 for this?
• update_docs_snippets.sh needs to be run manually, right?

[lsierant]

@MaciejKaras

• I see in the code that we are uploading code_snippets output for every patch, including PRs etc. Do we actually need to upload code snippets for PRs? Do we have a cleanup mechanism in S3 for this?

Good catch! Indeed, we should only upload in public flavors!

• update_docs_snippets.sh needs to be run manually, right?

Yes, for now it's for manual run. I assume automation will be done as part of Atomic Releases.

[lsierant]

@MaciejKaras

• I see in the code that we are uploading code_snippets output for every patch, including PRs etc. Do we actually need to upload code snippets for PRs? Do we have a cleanup mechanism in S3 for this?

Good catch! Indeed, we should only upload in public flavors!

No, it's correct. Outputs are uploaded to s3 as part of evergreen artifacts for every run and this is by design. Wrt cleanup - it's being cleaned up according to the artifacts bucket - after 90 days or so.