Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: GitHub-hosted runners for benchmarks #29955

Merged
merged 9 commits into from
Feb 18, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/add-release-label.yml
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ jobs:
fetch-depth: 0 # This is needed to checkout all branches

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Get the next semver version
id: get-next-semver-version
2 changes: 1 addition & 1 deletion .github/workflows/build-beta.yml
Original file line number Diff line number Diff line change
@@ -42,7 +42,7 @@ jobs:

- name: Setup environment
if: ${{ steps.needs-beta-build.outputs.NEEDS_BETA_BUILD == 'true' }}
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Run beta build
if: ${{ steps.needs-beta-build.outputs.NEEDS_BETA_BUILD == 'true' }}
2 changes: 1 addition & 1 deletion .github/workflows/build-storybook.yml
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Build storybook
run: yarn storybook:build
2 changes: 1 addition & 1 deletion .github/workflows/check-attributions.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Check attributions changes
run: yarn attributions:check
2 changes: 1 addition & 1 deletion .github/workflows/check-pr-labels.yml
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Check PR has required labels
env:
2 changes: 1 addition & 1 deletion .github/workflows/check-template-and-add-labels.yml
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Check template and add labels
id: check-template-and-add-labels
2 changes: 1 addition & 1 deletion .github/workflows/close-bug-report.yml
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Close release bug report issue
env:
20 changes: 0 additions & 20 deletions .github/workflows/codespaces.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/fitness-functions.yml
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ jobs:
fetch-depth: 0 # This is needed to checkout all branches

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Run fitness functions
env:
58 changes: 56 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -6,6 +6,7 @@ on:
- main
- master
- Version-v*
- trigger-ci*
pull_request:
types:
- opened
@@ -16,7 +17,7 @@ on:
jobs:
lint-workflows:
name: Lint workflows
uses: metamask/github-tools/.github/workflows/lint-workflows.yml@c534f265e02af2f2422a3c686bb09a11bfbf4cc2
uses: metamask/github-tools/.github/workflows/lint-workflows.yml@1d657e262aea7e3f216754febb624831527d2565

test-lint-shellcheck:
name: Test lint shellcheck
@@ -28,7 +29,16 @@ jobs:

test-circular-deps:
name: Test circular deps
uses: ./.github/workflows/test-circular-deps.yml
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Check circular dependencies
run: yarn circular-deps:check

test-lint-changelog:
name: Test lint changelog
@@ -66,10 +76,52 @@ jobs:
name: Validate lavamoat policy webapp
uses: ./.github/workflows/validate-lavamoat-policy-webapp.yml

prep-build-test-browserify:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- run: yarn build:test

- name: Upload artifact prep-build-test-browserify
uses: actions/upload-artifact@v4
with:
name: prep-build-test-browserify
path: ./dist/

prep-build-test-webpack:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- run: yarn webpack --test --no-lavamoat --no-cache --browser=chrome --lockdown --sentry --snow --env production
env:
SEGMENT_PROD_WRITE_KEY: '-' # A fake key just to make the build succeed

- name: Upload artifact prep-build-test-webpack
uses: actions/upload-artifact@v4
with:
name: prep-build-test-webpack
path: ./dist/

run-tests:
name: Run tests
uses: ./.github/workflows/run-tests.yml

run-benchmarks:
uses: ./.github/workflows/run-benchmarks.yml
needs:
- prep-build-test-browserify
- prep-build-test-webpack

wait-for-circleci-workflow-status:
name: Wait for CircleCI workflow status
uses: ./.github/workflows/wait-for-circleci-workflow-status.yml
@@ -112,6 +164,8 @@ jobs:
secrets:
PR_COMMENT_TOKEN: ${{ secrets.PR_COMMENT_TOKEN }}

# Explanation for all-jobs-completed and all-jobs-pass being separate:
# https://github.com/MetaMask/metamask-module-template/pull/151
all-jobs-completed:
name: All jobs completed
runs-on: ubuntu-latest
2 changes: 1 addition & 1 deletion .github/workflows/publish-prerelease.yml
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ jobs:
fetch-depth: 0 # This is needed to get merge base to calculate bundle size diff

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Get merge base commit hash
id: get-merge-base
41 changes: 41 additions & 0 deletions .github/workflows/run-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
on:
workflow_call:

env:
COMMANDS: |
{
pageload: 'yarn benchmark:chrome --out test-artifacts/chrome/benchmark/pageload.json --retries 2',
userActions: 'yarn user-actions-benchmark:chrome --out test-artifacts/chrome/benchmark/user_actions.json --retries 2',
}

jobs:
benchmarks:
runs-on: ubuntu-22.04
strategy:
matrix:
buildType: [browserify, webpack]
testType: [pageload, userActions]
name: ${{ matrix.buildType }}-${{ matrix.testType }}
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Download artifact prep-build-test-${{ matrix.buildType }}
uses: actions/download-artifact@v4
with:
path: ./dist/
pattern: prep-build-test-${{ matrix.buildType }}
merge-multiple: true

- name: Run the benchmark
# Choose a benchmark command from env.COMMANDS
run: ${{ fromJson(env.COMMANDS)[matrix.testType] }}

- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
name: benchmark-${{ matrix.buildType }}-${{ matrix.testType }}
path: test-artifacts/chrome/benchmark/
9 changes: 5 additions & 4 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: test:unit:coverage
run: yarn test:unit:coverage --shard=${{ matrix.shard }}/${{ strategy.job-total }}
@@ -43,7 +43,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: test:unit:webpack:coverage
run: yarn test:unit:webpack:coverage
@@ -64,7 +64,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: test:integration:coverage
run: yarn test:integration:coverage
@@ -92,12 +92,13 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: coverage
pattern: coverage-*
merge-multiple: true

- name: Merge coverage reports
18 changes: 0 additions & 18 deletions .github/workflows/test-circular-deps.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/test-deps-audit.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Run audit
run: yarn audit
2 changes: 1 addition & 1 deletion .github/workflows/test-deps-depcheck.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Run depcheck
run: yarn depcheck
2 changes: 1 addition & 1 deletion .github/workflows/test-lint-changelog.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Validate changelog
# For a `pull_request` event, the branch is `github.head_ref``.
2 changes: 1 addition & 1 deletion .github/workflows/test-lint-lockfile.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Lint lockfile
run: yarn lint:lockfile
2 changes: 1 addition & 1 deletion .github/workflows/test-lint.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Lint
run: yarn lint
2 changes: 1 addition & 1 deletion .github/workflows/test-storybook.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Install Playwright browsers
run: yarn exec playwright install chromium
2 changes: 1 addition & 1 deletion .github/workflows/test-yarn-dedupe.yml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565

- name: Detect yarn lock deduplications
run: yarn dedupe --check
6 changes: 3 additions & 3 deletions .github/workflows/update-attributions.yml
Original file line number Diff line number Diff line change
@@ -59,7 +59,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.issue.number }}
- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565
- name: Get commit SHA
id: commit-sha
run: echo "COMMIT_SHA=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
@@ -81,7 +81,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.issue.number }}
- name: Setup environment
uses: metamask/github-tools/.github/actions/setup-environment@main
uses: metamask/github-tools/.github/actions/setup-environment@1d657e262aea7e3f216754febb624831527d2565
- name: Generate Attributions
run: yarn attributions:generate
- name: Cache attributions file
@@ -182,4 +182,4 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.LAVAMOAT_UPDATE_TOKEN }}
PR_NUMBER: ${{ github.event.issue.number }}
ACTION_RUN_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
ACTION_RUN_URL: '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}'
Loading

Unchanged files with check annotations Beta

// TODO: Re think how to test this without exposing internal state
// it('should replace ethers instance when called with a different chainId than was current when the controller was instantiated', async function () {

Check warning on line 1265 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// fetchTradesInfoStub.mockReset();
// const _swapsController = getSwapsController();
// expect(currentEthersInstance).not.toStrictEqual(newEthersInstance);
// });
// it('should not replace ethers instance when called with the same chainId that was current when the controller was instantiated', async function () {

Check warning on line 1286 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// const _swapsController = new SwapsController({
// getBufferedGasLimit: MOCK_GET_BUFFERED_GAS_LIMIT,
// provider,
// expect(currentEthersInstance).toStrictEqual(newEthersInstance);
// });
// it('should replace ethers instance, and _ethersProviderChainId, twice when called twice with two different chainIds, and successfully set the _ethersProviderChainId when returning to the original chain', async function () {

Check warning on line 1306 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// const _swapsController = new SwapsController({
// getBufferedGasLimit: MOCK_GET_BUFFERED_GAS_LIMIT,
// provider,
});
});
// it('clears polling timeout', function () {

Check warning on line 1390 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// swapsController._pollingTimeout = setTimeout(() => {
// throw new Error('Polling timeout not cleared');
// }, POLLING_TIMEOUT);
describe('stopPollingForQuotes', function () {
// TODO: Re think how to test this without exposing internal state
// it('clears polling timeout', function () {

Check warning on line 1406 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// swapsController._pollingTimeout = setTimeout(() => {
// throw new Error('Polling timeout not cleared');
// }, POLLING_TIMEOUT);
describe('resetPostFetchState', function () {
// TODO: Re think how to test this without exposing internal state
// it('clears polling timeout', function () {

Check warning on line 1429 in app/scripts/controllers/swaps/swaps.test.ts

GitHub Actions / Test lint / Test lint

Some tests seem to be commented
// swapsController._pollingTimeout = setTimeout(() => {
// throw new Error('Polling timeout not cleared');
// }, POLLING_TIMEOUT);
image_large_url: '',
image_opengraph_url: '',
blurhash: 'U=Io~ufQ9_jtJTfQsTfQ0*fQ$$fQ#nfQX7fQ',
predominant_color: '#fb9f18',

Check warning on line 91 in test/e2e/flask/solana/common-solana.ts

GitHub Actions / Test lint / Test lint

'#fb9f18' Hex color values are not allowed. Consider using design tokens instead. For support reach out to the design system team #metamask-design-system on Slack
},
image_url: '',
image_properties: {
return null;
}
const t = useI18nContext();

Check warning on line 209 in ui/components/app/alert-system/alert-modal/alert-modal.tsx

GitHub Actions / Test lint / Test lint

React Hook "useI18nContext" is called conditionally. React Hooks must be called in the exact same order in every component render. Did you accidentally call a React Hook after an early return?
const severityStyle = getSeverityStyle(selectedAlert.severity);
return (
<Box
const isTestNetwork = useMemo(() => {
return (TEST_CHAINS as string[]).includes(currentNetwork.chainId);
}, [currentNetwork.chainId, TEST_CHAINS]);

Check warning on line 83 in ui/components/app/assets/asset-list/asset-list-control-bar/asset-list-control-bar.tsx

GitHub Actions / Test lint / Test lint

React Hook useMemo has an unnecessary dependency: 'TEST_CHAINS'. Either exclude it or remove the dependency array. Outer scope values like 'TEST_CHAINS' aren't valid dependencies because mutating them doesn't re-render the component
const allOpts: Record<string, boolean> = {};
Object.keys(allNetworks || {}).forEach((chainId) => {
} else {
dispatch(setTokenNetworkFilter({ [currentNetwork.chainId]: true }));
}
}, []);

Check warning on line 109 in ui/components/app/assets/asset-list/asset-list-control-bar/asset-list-control-bar.tsx

GitHub Actions / Test lint / Test lint

React Hook useEffect has missing dependencies: 'allOpts', 'currentNetwork.chainId', 'dispatch', and 'tokenNetworkFilter'. Either include them or remove the dependency array
// When a network gets added/removed we want to make sure that we switch to the filtered list of the current network
// We only want to do this if the "Current Network" filter is selected