CLOUDP-341025: Refactor of existing snippets on master #366

lsierant · 2025-08-24T14:30:01Z

Summary

This PR refactors how we generate and manage outputs from snippet tests.

Before, every snippet module has its own output/ directory. But because snippet modules were reused across different tests we had no way of storing the outputs from different test runs (the files were overwritten).

Changes were needed to accommodate upcoming Search snippets, but due to the large scale of this refactoring it was extracted onto master.

Only changes to public/architectures has been made performed in this PR. Existing search snippets are left intact and will be refactored on the search feature branch. After this PR is approved, a followup PR in the docs repo will be created adjusting for the new include paths.

This PR incorporated changes from #150 to fix diagnostic dumps from GKE clusters.

What this change will help

committing back snippet output will be easy as adding changes from one directory scripts/code_snippets/tests/outputs/test_* instead of traversing the tree and dealing with the conflicts pushed from different test runs.
more clarity what snippets and outputs we're using in docs
ability to share snippet modules and gather different outputs from the same snippet module for different testing scenarios.

Glossary

Snippet module - directory containing test.sh and scripts in a code_snippets sub directory. It's a reusable set of scripts, e.g. public/architectures/setup-multi-cluster/ra-01-setup-gke.
Snippet test - automated test running in evg that runs multiple snippet modules. It's a top level entrypoint that runs all the snippet modules. Examples: scripts/code_snippets/tests/test_gke_multi_cluster_no_mesh_snippets.sh,
scripts/code_snippets/test_gke_multi_cluster_snippets.sh.
- Previously we named these files a "task" and they were directly in scripts/code_snippets dir (e.g. scripts/code_snippets/task_gke_multi_cluster_no_mesh_snippets_test.sh).
Snippet outputs - output files (.out) gathered from a Snippet test.

Changes

Snippet modules don't store their outputs in the snippet directory now, but in scripts/code_snippets/tests/outputs/<test name> directory, which is gathering all outputs from a given snippet test.
Every file name in any of the code_snippets directories in the codebase must have a globally unique name. Existing file names has been refactored to match the new convention (in public/architecture).
- Validation/linter has been added (scripts/code_snippets/validate_snippets.py) added to pre-commit that enforces this convention.
The naming convention is the following:
Snippet module directory must be prefixed by some arbitrary prefix. Snippet modules in reference architectures has been prefixed with "ra-NN, e.g. ra-02-setup-operator. Every snippet file belonging to this module should be also prefixed (e.g. ra-02_0046_create_image_pull_secrets.sh`)
This allows for storing all the snippets outputs in a flat directory, guaranteeing we won't overwrite outputs.
This also helps quickly verifying the paths that will be used in the docs to include snippets and outputs. Having those prefixes will help identifying if any included file is coming from a correct snippet test and module.
Improved diagnostic gathering from the test runs
- GKE clusters are correctly handled now
- Every file name is prefixed with the kube context name and the namespace from which the files were downloaded. This will impact all the e2e test runs, but it will improve diagnostic archive in cases where we dump more than one namespace or k8s cluster.
All evergreen yamls related to snippets have been moved into .evergreen-snippets.yml (included from the main .evergreen-snippets.yml)

Here is the fragment of the current directory structure:

public/architectures
├── ra-06-ops-manager-multi-cluster
│   ├── code_snippets
│   │   ├── ra-06_0250_generate_certs.sh
│   │   ├── ra-06_0300_ops_manager_create_admin_credentials.sh
            [...]
│   │   ├── ra-06_9100_delete_backup_namespaces.sh
│   │   └── ra-06_9200_delete_om.sh
│   ├── env_variables.sh
│   ├── teardown.sh
│   └── test.sh
├── ra-07-mongodb-replicaset-multi-cluster
│   ├── code_snippets
│   │   ├── ra-07_1050_generate_certs.sh
│   │   ├── ra-07_1100_mongodb_replicaset_multi_cluster.sh
            [...]
│   │   └── ra-07_9000_delete_resources.sh
│   ├── env_variables.sh
│   ├── teardown.sh
│   └── test.sh

scripts/code_snippets
├── tests
│   ├── outputs
│   │   ├── test_gke_multi_cluster_no_mesh_snippets
│   │   ├── test_gke_multi_cluster_snippets
│   │   │   ├── ra-01_0030_verify_access_to_clusters.out
│   │   │   ├── ra-02_0200_kubectl_mongodb_configure_multi_cluster.out
│   │   │   ├── ra-02_0205_helm_configure_repo.out
            [...]
│   │   │   ├── ra-06_0321_ops_manager_wait_for_pending_state.out
│   │   │   ├── ra-06_0322_ops_manager_wait_for_running_state.out
│   │   │   └── ra-06_0522_ops_manager_wait_for_running_state.out
│   │   └── test_kind_search_community_snippets
│   │       ├── 01_0090_helm_add_mogodb_repo.out
│   │       ├── 01_0100_install_operator.out
            [...]
│   │       ├── 03_0450_execute_search_query.out
│   │       └── 03_0455_execute_vector_search_query.out
│   ├── test_gke_multi_cluster_no_mesh_snippets.sh
│   ├── test_gke_multi_cluster_snippets.sh
│   └── test_kind_search_community_snippets.sh

Proof of Work

Green EVG run for all snippets.

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you added changelog file?
- use skip-changelog label if not needed
- refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

github-actions · 2025-08-24T14:30:41Z

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.3.1 Release Notes

scripts/funcs/operator_deployment

public/architectures/ra-10-ops-manager-mc-no-mesh/test.sh

lucian-tosa · 2025-08-26T09:37:34Z

scripts/code_snippets/code_snippets_cleanup.sh

@@ -1,16 +1,36 @@
-#!/usr/bin/env bash
+#!/bin/bash


oh, thanks, it's a debugging leftover!

lucian-tosa · 2025-08-26T10:17:17Z

scripts/evergreen/e2e/dump_diagnostic_information.sh

+      # Capture output first to check if it contains actual resources
+      local temp_output
+      temp_output=$(kubectl --context="${context}" -n "${namespace}" ${action} "${object}" 2>&1)
+
+      # Check if output contains actual resources (not just empty list)
+      # Skip if it's an empty YAML list (contains "items: []")
+      if printf '%s\n' "${temp_output}" | grep -Fq "items: []"; then
+          # Empty list, don't create file
+          return
+      fi
+
+      if [[ -n "${out_file}" ]]; then
+        {
+          header "${msg}"
+          echo "${temp_output}"
+        } > "${out_file}"
+      else
+        header "${msg}"
+        kubectl --context="${context}" -n "${namespace}" ${action} "${object}" 2>&1
+      fi


nit: This indentation looks off in github.

anandsyncs · 2025-09-01T20:17:48Z

public/architectures/ra-06-ops-manager-multi-cluster/teardown.sh


-run 9100_delete_backup_namespaces.sh &
-run 9200_delete_om.sh &
+run ra-06_9100_delete_backup_namespaces.sh &


What is the significance of this ra prefix?
Edit: I released it probably stands for reference architecture(s) 😄

anandsyncs · 2025-09-01T20:21:26Z

...tup-multi-cluster/ra-02-setup-operator/code_snippets/ra-02_0211_check_operator_deployment.sh

@@ -1,4 +1,4 @@
-kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OPERATOR_NAMESPACE}" rollout status deployment/mongodb-kubernetes-operator-multi-cluster
+kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OPERATOR_NAMESPACE}" rollout status --timeout=2m deployment/mongodb-kubernetes-operator-multi-cluster


Was it getting stuck?

Yes, but I don't have any proof anymore 🙈
I mean - not stuck, sometimes it took longer than default timeout on GKE clusters. Not sure why.

lsierant · 2025-09-08T20:03:10Z

evergreen refresh

lsierant self-assigned this Aug 24, 2025

lsierant force-pushed the lsierant/snippets-refactor branch 2 times, most recently from 3420ed1 to be64ad8 Compare August 26, 2025 07:09

lsierant changed the title ~~Refactor of existing snippets on master~~ CLOUDP-341025: Refactor of existing snippets on master Aug 26, 2025

lsierant mentioned this pull request Aug 26, 2025

Dump diagnostic logs from snippets runs #150

Closed

lsierant force-pushed the lsierant/snippets-refactor branch from d5fd23e to b08575c Compare August 26, 2025 08:24

lsierant marked this pull request as ready for review August 26, 2025 08:24

lsierant requested a review from a team as a code owner August 26, 2025 08:24

lsierant requested review from anandsyncs and viveksinghggits August 26, 2025 08:24

lsierant force-pushed the lsierant/snippets-refactor branch from b08575c to d6f389d Compare August 26, 2025 08:26

lsierant requested a review from lucian-tosa August 26, 2025 08:26

lucian-tosa approved these changes Aug 26, 2025

View reviewed changes

lsierant force-pushed the lsierant/snippets-refactor branch from bc7095f to 9380a5a Compare August 26, 2025 11:46

lsierant added the skip-changelog Use this label in Pull Request to not require new changelog entry file label Aug 26, 2025

lsierant force-pushed the lsierant/snippets-refactor branch 2 times, most recently from 674c669 to 6d985d9 Compare August 28, 2025 20:23

anandsyncs reviewed Sep 1, 2025

View reviewed changes

anandsyncs approved these changes Sep 1, 2025

View reviewed changes

lsierant force-pushed the lsierant/snippets-refactor branch 3 times, most recently from d63c5c1 to 56dbb8b Compare September 8, 2025 20:02

lsierant force-pushed the lsierant/snippets-refactor branch from 56dbb8b to ed39d15 Compare September 8, 2025 20:04

lsierant added 3 commits September 10, 2025 21:00

Refactor of existing snippets on master and fix GKE diagnostics

d7948ca

Review fixes

8c88c26

De-flaked removing GKE service account

d9d020a

lsierant force-pushed the lsierant/snippets-refactor branch from 3588fa1 to d9d020a Compare September 10, 2025 19:42

lsierant added 2 commits September 10, 2025 22:14

Archiving snippets outputs

af6b704

Fixed DNS sa identifier

9f64ee6

lsierant enabled auto-merge (squash) September 11, 2025 09:15

lsierant merged commit 1ad24cf into master Sep 11, 2025
6 of 7 checks passed

lsierant deleted the lsierant/snippets-refactor branch September 11, 2025 09:19

This was referenced Sep 11, 2025

Improve diagnostic dumps in evg #424

Closed

Update code snippets outputs #437

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CLOUDP-341025: Refactor of existing snippets on master #366

CLOUDP-341025: Refactor of existing snippets on master #366

Uh oh!

lsierant commented Aug 24, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 24, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

lucian-tosa Aug 26, 2025

Uh oh!

lsierant Aug 26, 2025

Uh oh!

lucian-tosa Aug 26, 2025

Uh oh!

anandsyncs Sep 1, 2025 •

edited

Loading

Uh oh!

lsierant Sep 1, 2025

Uh oh!

anandsyncs Sep 1, 2025

Uh oh!

lsierant Sep 1, 2025 •

edited

Loading

Uh oh!

lsierant commented Sep 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -1,4 +1,4 @@
		kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OPERATOR_NAMESPACE}" rollout status deployment/mongodb-kubernetes-operator-multi-cluster
		kubectl --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${OPERATOR_NAMESPACE}" rollout status --timeout=2m deployment/mongodb-kubernetes-operator-multi-cluster

CLOUDP-341025: Refactor of existing snippets on master #366

CLOUDP-341025: Refactor of existing snippets on master #366

Uh oh!

Conversation

lsierant commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What this change will help

Glossary

Changes

Proof of Work

Checklist

Uh oh!

github-actions bot commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MCK 1.3.1 Release Notes

Uh oh!

Uh oh!

Uh oh!

lucian-tosa Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

lsierant Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

lucian-tosa Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

anandsyncs Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lsierant Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

anandsyncs Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

lsierant Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lsierant commented Sep 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lsierant commented Aug 24, 2025 •

edited

Loading

github-actions bot commented Aug 24, 2025 •

edited

Loading

anandsyncs Sep 1, 2025 •

edited

Loading

lsierant Sep 1, 2025 •

edited

Loading