Switch context by test #204

lsierant · 2025-06-19T15:35:35Z

Summary

This PR helps in switching to a proper context/variant by specifying a test mark or full task's evergreen url.

It solves a source of friction in our day-to-day workflows, which is finding which context to switch to in order to run a particular test. And by being able to switch context by specifying evergreen task's url we're a step closer to having automation that reruns a failed script locally.

Current workflow

To find a context to run the test locally we often start from two places:

Starting from Evergreen UI (easier): look at the evergreen variant in the evergreen UI, copy it and run make switch context=<copied variant> or use interactive fzf-powered make switch (without args), paste or type the variant and press enter.
Starting from an e2e test file (harder): find a @pytest.mark in the file, copy it, go to evergreen.yml, find a task group containing that file, copy its name, then find a variant where that task group is used, copy variant name, run make switch context=<copied variant>...

Proposed workflow

make switcht is a new tool which is a supercharged make switch. It's named similarly (typing make switch is kind of a muscle memory now) and works in two modes:

With explicit test mark/url on test= argument:

make switcht test=<test>, where <test> is a pytest mark (e.g. e2e_replica_set) - it will list all the variants that this test is running in to be picked with fzf
make switcht test=<task url>, where <task url> is a full url to the evergreen test - it will switch directly to the variant from that test

Without specifying test argument, which works in a similar interactive way like make switch, but lists first all test files with their marks to fuzzy-find the test, which then is used to list pick its variants: make switcht

Requirements

It requires:

fzf - for the interactive fuzzy finder (we already use it for make switch)
ripgrep - to quickly list all the pytest marks in all e2e test files
Having evg credentials in env vars, e.g. in your scripts/dev/contexts/private-context

EVERGREEN_USER=$(cat ~/.evergreen.yml | yq .user)
EVERGREEN_API_KEY=$(cat ~/.evergreen.yml | yq .api_key)

Changes

Majority of the implementation is in two places:

scripts/dev/switch_context_by_test.sh - it's mostly for fzf, finding pytest marks using rg. It calls scripts/python/find_test_variants.py
scripts/python/find_test_variants.py - a new python script that lists all the variants of a pytest's mark (--task-name) or a variant from a full evg task url (--task-url)
introduced scripts/python directory, to limit proliferation of python scripts in random directories. Having them in one place makes it easier to import/run them as modules
refactored some evergreen API helpers into scripts/python/evergreen_api.py
some minor, random changes improving debuggability of bash scripts

Proof of Work

Here are some examples for the most common use cases:

With explicit test= parameter

Switch to one of test's context by pytest mark: make switcht test=e2e_replica_set
Switch to context by test's url: make switcht test='https://spruce.mongodb.com/task/[...]/logs?execution=0'

Without any parameters (fully interactive)

Switch to context by fuzzy-searching a test first: make switcht
Switch to context by pasting full file path into test picker: make switcht
Switch to context by pasting full url into fzf picker: make switcht

Afterwards, if the context needs to be regenerated again just run make switch (without t) + enter - it will remember last switched context and be selected first.

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you checked for release_note changes?

Reminder (Please remove this when merging)

Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible
Our Short Guide for PRs: Link
Remember the following Communication Standards - use comment prefixes for clarity:
- blocking: Must be addressed before approval.
- follow-up: Can be addressed in a later PR or ticket.
- q: Clarifying question.
- nit: Non-blocking suggestions.
- note: Side-note, non-actionable. Example: Praise
- --> no prefix is considered a question

lsierant · 2025-06-19T17:05:27Z

config/manifests/bases/mongodb-kubernetes.clusterserviceversion.yaml

-    description: The MongoDB Controllers for Kubernetes enable easy deploys of 
-      MongoDB into Kubernetes clusters, using our management, monitoring and 
-      backup platforms, Ops Manager and Cloud Manager.
+    description: The MongoDB Controllers for Kubernetes enable easy deploys of MongoDB


for some reason it started to format it differently despite being up-to-date on master

try updating your python packages

yeah, that did the trick, thanks!

this fixes it: #210

lsierant · 2025-06-19T17:06:23Z

scripts/dev/set_env_context.sh


 # shellcheck disable=1091
 source scripts/funcs/errors

 script_name=$(readlink -f "${BASH_SOURCE[0]}")
 script_dir=$(dirname "${script_name}")
-context_file="${script_dir}/../../.generated/context.export.env"
+context_file="$(realpath "${script_dir}/../../.generated/context.export.env")"


this resolves to the physical path and avoids huge paths with multiple /../../ when debugging with -x

Julien-Ben

Awesome PR ! I like that it doesn't break anything to existing workflows

Nice demo gif as well
Thanks for this improvement 🙏

nammn

Awesome work -
lets merge mine first #210

nammn · 2025-06-25T12:26:50Z

scripts/dev/contexts/root-context

@@ -116,3 +116,8 @@ export MDB_SEARCH_COMMUNITY_VERSION

 export MDB_SEARCH_COMMUNITY_NAME="mongodb-search-community"
 export MDB_SEARCH_COMMUNITY_REPO_URL="quay.io/mongodb"
+
+
+if [[ ${MDB_BASH_DEBUG:-0} == 1 ]]; then


can you add a small comment? I actually have no clue what this does :D

nammn · 2025-06-25T12:27:06Z

scripts/dev/launch_e2e.sh

@@ -49,5 +50,3 @@ else
 fi

 title "E2e test ${test} is finished"
-


eof new line?

nammn · 2025-06-25T12:27:23Z

scripts/dev/launch_e2e.sh

@@ -2,6 +2,7 @@

 set -Eeou pipefail

+test "${MDB_BASH_DEBUG:-0}" -eq 1 && set -x


a small comment for the non bash users?

I don't want to put comment to each that line, because that line should be present in all of our bash scripts.
It's allowing us to debug deeply nested shell scripts, e.g.
MDB_BASH_DEBUG=1 make e2e ... will just propagate set -x in all scripts that declare this

lsierant force-pushed the lsierant/switcht branch from 07417a7 to 9e97791 Compare June 19, 2025 16:55

lsierant marked this pull request as ready for review June 19, 2025 16:56

lsierant requested a review from a team as a code owner June 19, 2025 16:56

lsierant requested review from m1kola and viveksinghggits June 19, 2025 16:56

lsierant force-pushed the lsierant/switcht branch from 9e97791 to 7e77ef4 Compare June 19, 2025 17:03

lsierant commented Jun 19, 2025

View reviewed changes

lsierant force-pushed the lsierant/switcht branch 2 times, most recently from fb08643 to 9a76e72 Compare June 19, 2025 17:10

Julien-Ben approved these changes Jun 20, 2025

View reviewed changes

nammn approved these changes Jun 25, 2025

View reviewed changes

lsierant added 2 commits June 26, 2025 09:07

Switch context by test

5e70aac

lint

ca7d2d3

lsierant force-pushed the lsierant/switcht branch from 9a76e72 to ca7d2d3 Compare June 26, 2025 07:09

lsierant merged commit fe81867 into master Jun 26, 2025
35 checks passed

lsierant deleted the lsierant/switcht branch June 26, 2025 11:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Switch context by test #204

Switch context by test #204

Uh oh!

lsierant commented Jun 19, 2025 •

edited

Loading

Uh oh!

lsierant Jun 19, 2025

Uh oh!

lucian-tosa Jun 20, 2025

Uh oh!

lsierant Jun 20, 2025

Uh oh!

nammn Jun 25, 2025

Uh oh!

lsierant Jun 19, 2025

Uh oh!

Julien-Ben left a comment

Uh oh!

nammn left a comment •

edited

Loading

Uh oh!

nammn Jun 25, 2025

Uh oh!

nammn Jun 25, 2025

Uh oh!

nammn Jun 25, 2025

Uh oh!

lsierant Jun 25, 2025

Uh oh!

Uh oh!

Uh oh!

		@@ -49,5 +50,3 @@ else
		fi

		title "E2e test ${test} is finished"

		@@ -2,6 +2,7 @@

		set -Eeou pipefail

		test "${MDB_BASH_DEBUG:-0}" -eq 1 && set -x

Switch context by test #204

Switch context by test #204

Uh oh!

Conversation

lsierant commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Current workflow

Proposed workflow

Requirements

Changes

Proof of Work

Checklist

Reminder (Please remove this when merging)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Julien-Ben left a comment

Choose a reason for hiding this comment

Uh oh!

nammn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lsierant commented Jun 19, 2025 •

edited

Loading

nammn left a comment •

edited

Loading