Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new configs to support inter AZ stage-to-stage routing #755

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

tcai-stripe
Copy link
Contributor

@tcai-stripe tcai-stripe commented Feb 25, 2025

Context

We want to reduce cross AZ traffic during scalar stage to stage communication so adding the following two configs:

  • mantis.taskexecutor.router.scalar-stage-to-stage to support custom stage to stage router, it will fall back to the default round robin router on instantiation failure
  • mantis.availabilityZoneUtils.class to allow users to provide availability zone for each worker

Checklist

  • ./gradlew build compiles code correctly
  • Added new tests where applicable
  • ./gradlew test passes all tests
  • Extended README or added javadocs where applicable

@tcai-stripe tcai-stripe marked this pull request as ready for review February 25, 2025 23:32
Copy link
Collaborator

@crioux-stripe crioux-stripe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty straightforward and as though we could use it to reduce costs stage to stage. I wanted @Andyz26 to take a look as well considering the impact it could have on Netflix.

Though I like that we're defaulting to the old behavior so the base effect is no change for Netflix.

@tcai-stripe tcai-stripe force-pushed the stage-to-stage-routing branch from 60588e1 to 3771468 Compare March 11, 2025 20:15
@tcai-stripe
Copy link
Contributor Author

@Andyz26 could you please help review this when you get a chance? As @codyrioux mentioned earlier that this shouldn't have any impact on Netflix and we want to plugin our own router to reduce cross AZ traffic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants