V16 QA cherry picked acceptance tests with different configuration #20106
+1,362
−730
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mplates (#20049) * Added setup for external login * Started on yaml * Added test file * Updated pipeline * Use env vars * Added env variables and commented out test we don't need to run * Removed list from matrix * Updated condition * Updated package path * Updated testFolder * double slash * Updated condition * Updated condition again * Added port * Removed redundant values * Set as env vars * Added env vars beneath matrix * Get env * Updated naming * Updated usage of values * Added a check for client id, to see if value set * Moved env out of pool * Tried moving env * Trying to fix the env being empty * Removed env * Updated name of variable * Fixed A cyclical reference * Updated typo * More logging * Reverted change * Added env * Added env to tests * Cleaned up * Added yaml template files * Updated nighly pipepline to use templates * Updated sln * Split yaml into templates for e2e setup * Updated pipeline * Updated solution file * Set value * Added if statement * Added variables * Set default values * Updated values * Updated condition * Run multiple tests * Added env * Updated from parameter to variable * Fixed condition * Fixed condition to use actual value * Updated npx wait on command * Updated pwsh * Updated port again * Updated port value * Updated wait on * Updated condition * Restructured * Updated var * Updated run application steps * Added echo * Updated to boolean * Updated conditions * Updated test template usage * Added databaseType * Added another databaseType * Split up templates * Fixed indentation * Updated condition * updated path * removed build from path * Updated conditions for azureAd * Fixed indentation * Updated to single qoutes * Cleaned up * Removed unused file * Clarified namin * Moved * Updated pipeline, not done * Updated locator * Updated pipelines * Updated test helpers package * Skipped build stage for default app settings tests * Updated password var * Updated locators * Updated defaultconfig build setup * Split E2E stage in two * Added parameter for skipping integration tests * Cleaned up * Added ASPNETCORE_URLS
* Start of appsetting * Updated setup of playwright * Adjusted the pipeline * Updated appsetting * Added install test * Added comments * Updated pipeline * Updated development app settings * Commented tests out * comment * Added if statement * Updated pipeline * Fixed condition * Changed to production * Added a log * Updated copy item * Added * Updated app settings * Updated pipeline * Moved playwright login * Updated pipeline * Updated app setting * Updated nightly * Updated appsettings * Updated get * Updated wait on * Updated appsettings * Updated connection string * Updates * Skips code * Updated variable * Updated pipeline * We want to always retain the trace, to see if the test runs as expected on the pipeline * Added a temporary wait till port is open * Fixed condition * Added missing tcp for wait on * Updated URL env * Updated setup * Fixed string * Updated locator * Split tests into SQLite and SQLServer * Updated pipeline to run all tests * Retain trace on failure * Added testFolder var * Added appsettings and program for delivery api tests * Updated playwright config * Split test runners into defaultconfig and different app settings * Added delivery api tests * Cleaned up tests * Bumped version * Updated pipeline * Small fixes * Added password * Updated connection string * Fixed * Removed quotes * Removed unnecessary connection string * Added missing password * Cleaned up * Cleaned up * Cleaned up * Updated to use helpers * Bumped version * Updated helper usage * Added password to variables and a condition * Added check * Indented value * Fixed condition * More updates * Updated variable
There was a problem hiding this comment.
Pull Request Overview
This PR cherry-picks changes from v15 to add acceptance tests with different configuration setups and refactors the pipeline structure by extracting reusable templates. The changes enable testing various Umbraco configurations including different databases, delivery API settings, and external authentication providers.
Key Changes:
- Adds acceptance test configurations for delivery API, unattended install, and Azure AD B2C external login
- Extracts duplicate pipeline code into reusable YAML templates
- Updates playwright configuration to support multiple test projects with different settings
Reviewed Changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| umbraco.sln | Adds new solution folder for E2E pipeline templates |
| tests/Umbraco.Tests.AcceptanceTest/tests/UnattendedInstallConfig/* | Adds test scenarios for different database installation configurations |
| tests/Umbraco.Tests.AcceptanceTest/tests/ExternalLogin/AzureADB2C/* | Adds Azure AD B2C external login test setup with required classes and configuration |
| tests/Umbraco.Tests.AcceptanceTest/tests/DeliveryApi/* | Adds delivery API test configuration and test cases |
| tests/Umbraco.Tests.AcceptanceTest/playwright.config.ts | Updates config to support multiple test projects with different configurations |
| tests/Umbraco.Tests.AcceptanceTest/package.json | Updates test helper dependency version |
| build/nightly-E2E-test-pipelines.yml | Major refactor adding new parameters and stages for different test configurations |
| build/nightly-E2E-*-template.yml | New reusable pipeline templates for setup, build, run application, and run tests |
| build/azure-pipelines.yml | Updates to use new template structure for E2E tests |
Files not reviewed (1)
- tests/Umbraco.Tests.AcceptanceTest/package-lock.json: Language not supported
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR cherry picks changes from v15. Focuses on adding acceptance tests with different appsettings and configurations. This PR also splits the pipeline into templates, as we had quite a bit of reused code.