Skip to content

[Nexthop][fboss2-dev] Let fboss2_integration_test list and filter by production features - #2023

Open
hillol-nexthop wants to merge 1 commit into
meta:mainfrom
nexthop-ai:let-fboss2-integration-test-list-and
Open

hillol-nexthop wants to merge 1 commit into
meta:mainfrom
nexthop-ai:let-fboss2-integration-test-list-and

Conversation

@hillol-nexthop

Copy link
Copy Markdown
Contributor

Pre-submission checklist

  • I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
  • pre-commit run

Summary

Lets fboss2_integration_test be filtered by ASIC production features, the way the agent, qsfp and link tests are. This PR is plumbing only: no fixture declares a feature yet, so nothing is filtered. The first declarations follow in a second PR stacked on this one.

Test binary

  • Fboss2IntegrationTest::addProductionFeatures({...}), called from a fixture's constructor. A derived fixture adds to what its base declared.
  • --list_production_feature prints Feature List: A,B per test, the same format the other test binaries use.
  • In list mode the base constructor skips the test, so gtest never calls SetUp() or TearDown(). This is why features are declared in the constructor instead of a virtual getProductionFeatures(): many fixtures override TearDown(), several of them restore config with an agent restart, and a skip raised in the base SetUp() would not stop those from running during listing.
  • ConfigVlanCreateTest::SetUpTestSuite() returns early in list mode, since it queries the agent.

run_test.py

  • The production feature filter (arguments, SDK gating, per-test listing) moves from SaiAgentTestRunner to ProductionFeatureFilterMixin in runners/production_features.py. sai_agent behaviour is unchanged.
  • fboss2_integration uses the mixin and accepts --enable-production-features <asic>, --production-features <file>, --list-tests-for-features and --test-config-options.

Test plan

  • pytest fboss/oss/scripts/run_scripts/fboss_test_runner/unittests on this branch: 454 passed. That includes 5 new cases for the fboss2 runner (SDK gating and --list-tests-for-features among them); the existing sai_agent filter and SDK gating cases pass against the moved code.
  • cmake build of this branch in our CI (clang): fboss2_integration_test compiles and links. The run as a whole did not finish, because fboss/agent/hw/sai/api/bcm/SwitchApi.cpp from current main does not compile against the prebuilt SAI our CI uses (SAI_SWITCH_ATTR_EXT_STATIC_FDB_REPLACE_DYNAMIC undeclared); this PR does not touch that file. The C++ unit test step did not run for the same reason.
  • In our tree, where the same change is merged, fboss2_integration_test --list_production_feature was run with no agent running: it exits 0 in well under a second, and every test prints a Feature List: line and is reported skipped.

Review Findings

Pre-publication review of the final diff found no issues.

…n features

Let fboss2_integration_test be filtered by ASIC production features, the
way the agent, qsfp and link tests are. This change is plumbing only: no
fixture declares a feature yet, so nothing is filtered.

Test binary:

- Fboss2IntegrationTest::addProductionFeatures({...}), called from a
  fixture's constructor. A derived fixture adds to what its base
  declared.
- --list_production_feature prints "Feature List: A,B" per test, the
  same format the other test binaries use.
- In list mode the base constructor skips the test, so gtest never calls
  SetUp() or TearDown(). This is why features are declared in the
  constructor instead of a virtual getProductionFeatures(): many
  fixtures override TearDown(), several of them restore config with an
  agent restart, and a skip raised in the base SetUp() would not stop
  those from running during listing.
- ConfigVlanCreateTest::SetUpTestSuite() returns early in list mode,
  since it queries the agent.

run_test.py:

- The production feature filter (arguments, SDK gating, per-test
  listing) moves from SaiAgentTestRunner to ProductionFeatureFilterMixin
  in runners/production_features.py. sai_agent behaviour is unchanged.
- fboss2_integration uses the mixin and accepts
  --enable-production-features <asic>, --production-features <file>,
  --list-tests-for-features and --test-config-options.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants