ci(embedded): Patrol E2E with local mock auth and CI matrix#119
Open
dianaKhortiuk-frontegg wants to merge 33 commits intomasterfrom
Open
ci(embedded): Patrol E2E with local mock auth and CI matrix#119dianaKhortiuk-frontegg wants to merge 33 commits intomasterfrom
dianaKhortiuk-frontegg wants to merge 33 commits intomasterfrom
Conversation
…ards - Add E2E test mode MethodChannel (Android/iOS) and demo UI semantics - Patrol integration tests + Dart LocalMockAuthServer mirroring native E2E - scenario-catalog.json with matrix generation and demo-e2e workflow Made-with: Cursor
- Fix _waitForSemantics → waitForSemantics (undefined method error) - Remove unused imports (dart:convert, dart:async, dart:math) - Remove unused _random field from LocalMockAuthServer - Pin patrol_cli to 3.6.0 (compatible with patrol ^3.13.1) - Remove double cd into embedded/ in shard script - Drop redundant default argument values (resetState, method, count) - Add required trailing commas - Replace deprecated withOpacity with withValues Made-with: Cursor
- Patrol CLI requires test files to end with _test.dart - Remove remaining avoid_redundant_argument_values (delayMs, timeout defaults) Made-with: Cursor
patrol_cli 3.5–3.6 pairs with patrol 3.14–3.15; ^3.13.1 tripped the CLI version check even when the lock resolved to 3.15.1. Made-with: Cursor
Hardcoded iPhone 16 Pro is not present on GitHub macos-15 runners; resolve the first available iPhone from simctl JSON and boot it. Made-with: Cursor
Patrol UI tests link Pods_Runner_RunnerUITests; the target used Runner Debug/Release xcconfigs that only include Pods-Runner, breaking the integration-test xcodebuild (exit 65). Add Flutter/RunnerUITests.*.xcconfig wrappers and drop CLANG_WARN override that conflicted with Pods. Made-with: Cursor
…iOS manualInit - Replace initializeEmbeddedForLocalE2E (not in Maven 1.3.24) with FronteggAppService + companion singleton reflection - android.enableJetifier=false to fix Patrol JetifyTransform on Flutter engine - iOS: use FronteggApp.shared.manualInit + DEBUG testing hooks (matches demo-embedded Swift); remove nonexistent initEmbeddedForLocalE2E Made-with: Cursor
Made-with: Cursor
…for Runner - GitHub-hosted macOS runners cannot use the embedded demo team ID for signing; blank team enables automatic simulator signing. - Point Runner Profile at Flutter/Profile.xcconfig including Pods-Runner.profile (CocoaPods expects profile xcconfig, not Release). Made-with: Cursor
- Set DEVELOPMENT_TEAM to AM6NK96AX6 like demo-embedded in frontegg-ios-swift - Remove CI sed that cleared the team; Swift E2E relies on the same project signing - Prefer iPhone 16 Pro simulator, matching Swift demo-e2e destination - Run patrol test with --verbose and upload tee log as an artifact for failures Made-with: Cursor
- Implement E2E_TEST_FILTER in embedded_e2e_test.dart (e2ePatrolTest wrapper) - iOS job: pass matrix test-methods via E2E_METHODS and reuse run_embedded_e2e_shard.sh with IOS_DEVICE + PATROL_LOG_FILE + PATROL_VERBOSE (was running full suite per shard) - Shard script: optional -d for iOS, tee when PATROL_LOG_FILE set, remove || true so failed patrol runs fail the job - Update generate_e2e_matrix.js to discover e2ePatrolTest registrations Made-with: Cursor
- E2E_TEST_FILTER accepts comma-separated test names so iOS/Android shards build and install once per job instead of four times - Wait for simulator boot (bootstatus -b) and slightly longer settle before tests Made-with: Cursor
Root cause: frontegg_flutter's podspec omitted FronteggSwift dependency (SPM-first design), but the embedded demo's Xcode project never had the required SPM package reference wired up. Locally, cached SPM resolution hid the problem; on CI xcodebuild failed with "no such module FronteggSwift". Fix: - Add s.dependency 'FronteggSwift' to frontegg_flutter.podspec so CocoaPods properly provides the module to the plugin pod target - Add local FronteggSwift.podspec (1.2.79) in embedded/ios/ because only 1.2.76 is on CocoaPods trunk and the plugin needs loadEntitlements (1.2.77+) - Reference it in the Podfile with :podspec => 'FronteggSwift.podspec' - Remove stale SPM Package.resolved files (no longer needed) Verified: local xcodebuild build succeeds for Runner scheme on simulator. Made-with: Cursor
Enable Flutter's native Swift Package Manager plugin integration so frontegg_flutter is built via SPM instead of CocoaPods. This lets the plugin's Package.swift resolve FronteggSwift 1.2.79 transitively, eliminating the need for local podspecs or CocoaPods dependency hacks. - Enable --enable-swift-package-manager in CI workflow - Patch generated FlutterGeneratedPluginSwiftPackage platform to iOS 14.0 - Remove local FronteggSwift.podspec and Podfile overrides - Revert frontegg_flutter.podspec (no s.dependency FronteggSwift needed) - Add SPM Package.resolved with pinned FronteggSwift 1.2.79 Made-with: Cursor
The Frontegg SDK keeps background timers alive (connectivity probes, token refresh) which prevents pumpAndSettle from ever completing. Try pumpAndSettle with a short timeout and gracefully fall back to pump, then rely on explicit waitForSemantics/waitForText in each test. Also replace pumpAndSettle after tapSemantics with a fixed pump to avoid the same timer-induced hang after button taps. Made-with: Cursor
The native SDK could not reach the local mock server (http://127.0.0.1) because both Android and iOS block cleartext HTTP by default. Android: add usesCleartextTraffic and network_security_config.xml iOS: add NSAppTransportSecurity with localhost exception domains Also adds diagnostic logging in launchApp to trace SDK initialization. Made-with: Cursor
- Remove unnecessary flutter/foundation.dart import (fixes flutter analyze) - Override patrol_log to 0.4.0 to fix List.last crash in PatrolLogReader.readEntries that killed iOS shards after first failure - Make launchApp throw descriptive AssertionError on timeout instead of silent warning, so failure reason appears in CI test output Made-with: Cursor
pump(Duration) in LiveTestWidgetsFlutterBinding only calls Future.delayed — it does NOT process frames. Adding pump() (no duration) which triggers endOfFrame to explicitly request a frame. Diagnostics now report: - welcomeText (LoginPage Body rendered via byType) - scaffoldCount (1=MainPage only, 2=MainPage+LoginPage) - semanticsWidget (Semantics widget with label found by predicate) - fronteggState (init/auth/load/showLoader from FronteggProvider) This will reveal whether the issue is: a) StreamBuilder snapshot.hasData=false (no state reaching Flutter) b) State reaching Flutter but bySemanticsLabel not matching c) Frame not being processed after setState Made-with: Cursor
find.bySemanticsLabel relies on RenderObject.debugSemantics which requires the sendSemanticsUpdate frame phase to have completed. In LiveTestWidgetsFlutterBinding (integration tests), pump(Duration) only calls Future.delayed and does not explicitly process frames, so the semantics tree may never be compiled. The new _semFinder helper uses find.byWidgetPredicate to check the Semantics widget's properties.label directly, which works regardless of whether the semantics tree has been compiled. Confirmed via CI diagnostics: semanticsWidget=true (widget present), welcomeText=true (LoginPage rendered), scaffolds=2 (full tree built), but bySemanticsLabel returned empty because debugSemantics was null. Made-with: Cursor
The E2E test buttons are at the bottom of LoginPage's SingleChildScrollView and may be off-screen on CI simulators. Patrol's $.tap() fails with WaitUntilVisibleTimeoutException because the widget isn't hit-testable without scrolling. Use tester.ensureVisible() to scroll the widget into view before tapping with tester.tap() directly. Made-with: Cursor
…SemanticsLabel 1. tapWebButtonIfPresent: When the mock server's password login page has a prefilled password, the form auto-submits via JS before Patrol can find the "Sign in" button. The Swift reference demo silently succeeds in this case — match that behaviour by removing the AssertionError when the button is not found. 2. Replace two remaining find.bySemanticsLabel calls in the test file with _semFinder (byWidgetPredicate) to avoid the debugSemantics=null issue in integration tests. Made-with: Cursor
loginWithPassword now taps E2EEmbeddedPasswordButton and waits for UserPageRoot (auto-submit handles the webview flow), matching the Swift DemoEmbeddedUITestCase. The previous approach blocked the native Patrol server for 55s searching for a "Sign in" button that was already auto-submitted away, preventing the SDK from processing the redirect. patrol_log upgraded from 0.4.0 to 0.8.0 to fix the PatrolLogReader "Bad state: No element" crash that killed several test shards. Made-with: Cursor
pump(Duration) wraps through TestAsyncUtils.guard, which can deadlock when the native SDK presents a modal webview (WKWebView/WebView) after frontegg.login(). Using Future.delayed for timing + checking the widget tree directly works because LiveTestWidgetsFlutterBinding processes frames automatically via the real vsync signal. Made-with: Cursor
- onPush: use macos-latest + stable Flutter instead of macos-latest-large + pin (avoids jobs failing immediately when large-runner billing/quota is exceeded) - demo-e2e iOS: macos-15-xlarge -> macos-15 - demo-e2e Android: disable animations, longer emulator boot timeout; PATROL_MAX_RETRIES=2 - run_embedded_e2e_shard.sh: optional retries with adb restart between attempts - summary: wait for Android + iOS shards; tolerate missing artifacts on download Made-with: Cursor
CI showed launchApp never observing LoginPageRoot/UserPageRoot because delay-only polling never advanced the LiveTestWidgetsFlutterBinding frame pipeline. Use Future.delayed + pump() (no duration) so StreamBuilder updates apply; keep avoiding pump(Duration) for webview deadlock risk. Made-with: Cursor
Bare pump() in waitForSemantics/waitForText (and after tap) can block forever while the native SDK shows a modal webview, so deadlines never complete and GitHub cancels the whole job at 60m. Poll with Future.delayed only there; keep pump() for launchApp and immediately before tap only. Raise Demo Embedded E2E android/ios job timeouts to 90m for emulator boot, APK/xcodebuild, PATROL_MAX_RETRIES=2, and long per-test timeouts. Made-with: Cursor
22 scenarios × 4 tests/shard × PATROL_MAX_RETRIES=2 routinely exceeded 90m. Shard with MAX_TESTS_PER_SHARD=2 (11 matrix jobs) and PATROL_MAX_RETRIES=1 so each job finishes within the timeout budget. Made-with: Cursor
waitForSemantics had regained WidgetTester.pump(); pump blocks while the embedded WebView is modal so the poll loop never finishes and jobs hit the workflow timeout. - Drop pump() from waitForSemantics (delay + tree check only) - MAX_TESTS_PER_SHARD=1 (22 shards) so one slow scenario cannot block a pair - Job timeout 120m for long token/offline waits + build/emulator - Remove redundant waitForUserEmail after loginWithPassword in session test Made-with: Cursor
Skipping pump() in all semantics waits broke requestAuthorize, relaunch session checks, and token/offline scenarios that rely on MethodChannel-driven rebuilds. - waitForSemantics: optional pumpFrame (default true) - waitForUserEmail: awaitingUserPageAfterEmbeddedWebView skips pump only for UserPageRoot wait (embedded WebView deadlock); default false for requestAuthorize - loginWithPassword passes webview flag; SAML/OIDC/custom SSO/direct/Google social tests pass true - waitForText: pump each poll (runs after UserPageRoot is found) - waitForA11yTextContains: still no pump (Custom Tab); longer pre-wait + timeout - Poll up to 70s for scheduled token refresh; remove redundant waitForUserEmail after loginWithPassword Made-with: Cursor
…ndroid offline shard - Expose AuthenticatedOfflineModeEnabled and OfflineModeBadge in Flutter demo when E2E forces network-path offline (Swift/Android parity). - Widen OIDC/SAML and password-login timeouts; extend relaunch token test wait. - Android CI: omit testAuthenticatedOfflineModeWhenNetworkPathUnavailable via INPUT_EXCLUDE_METHODS and a dedicated matrix_android output until stable. Made-with: Cursor
…ction UI - waitForUserEmail: skip WidgetTester.pump while polling for user email when awaitingUserPageAfterEmbeddedWebView (same as UserPageRoot) to prevent CI hangs. - Plumb isOfflineMode from iOS native state; Android plugin sends false until SDK exposes it. - MainPage: show NoConnectionPage (NoConnectionPageRoot, RetryConnectionButton) when unauthenticated + offline; AuthenticatedOfflineRoot when authenticated without user. - Extend Android CI exclude list with testLogoutTerminateTransientNoConnectionThenCustomSSORecovers. - Longer wait for NoConnectionPageRoot in SSO recovery test. Made-with: Cursor
…oot for email - MainPage: mirror Swift MyApp — show global loader while initializing or isLoading so transient isOfflineMode does not replace Login with NoConnection (fixes missing LoginPageRoot / launchApp hangs). - waitForUserEmail: after UserPageRoot, delay briefly then waitForText with normal pump so user email widgets are built (avoids timeouts when pump was skipped). Made-with: Cursor
Use action-junit-report annotate_only so PR file annotations and job summary remain without creating extra Flutter E2E (shard N) check runs. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds embedded-demo Patrol end-to-end tests that mirror the native iOS/Android embedded E2E catalog: local HTTP mock auth server, shared scenario names, and sharded GitHub Actions jobs.
What changed
E2ETestModeMethodChannel (Android Kotlin + iOS Swift), E2E-only controls and Semantics labels on the embedded login/user flows; logout + JWTtoken_versionexposure for tests.integration_test/e2e/— mock server, harness,patrolTestsuite aligned withe2e/scenario-catalog.json.demo-e2e.ymlplus scripts to generate the matrix, run shards, and combine summaries.Verification
demo-e2eworkflow passes on this PR (Android shards + iOS job).Opened to validate remote CI; follow-up fixes expected if Patrol/emulator/Xcode need tuning.
Made with Cursor