Skip to content

Is playwright supported? Tests fail but show as passed #16

Open
@i-love-code

Description

@i-love-code

The sdk branch appears to use framework: playwright, and this is GitHub repository is linked to in the setup documentation for Playwright and BrowserStack.

In the sdk branch, when you set retries: 3 in the playwright configuration file for re-running test failures and then run npm run sample-test command, (with your own user/access key) it marks the tests in BrowserStack as passed, but throws these errors in the console and the SDK marks the tests as failed, leading to it retrying over and over.

Running 4 tests using 2 workers

1) [playwright-firefox@latest:Windows 11@browserstack] › tests\bstack_test_add_to_cart.js:3:1 › BStackDemo test add to cart

    Error: : expected object, got undefined

    Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

    Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

    Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
    Error: : expected object, got undefined

But in BrowserStack, it shows as passed sessions:
image

Activity

changed the title [-]Is playwright supported?[/-] [+]Is playwright supported? Tests fail but show as passed[/+] on Dec 13, 2023
i-love-code

i-love-code commented on Dec 13, 2023

@i-love-code
Author

It would appear that the browserstack-node-sdk that is wrapping the npx playwright test command does not correctly infer the outcome of the test, and marks all tests as having failed.

i-love-code

i-love-code commented on Dec 13, 2023

@i-love-code
Author

I have active BrowserStack support issues open for this, but I have not seen a helpful response/resolution. Trying all available paths for support.

rohit-y180

rohit-y180 commented on Dec 14, 2023

@rohit-y180

Hi @i-love-code , could you please let us know the support ticket ID that you have created on BrowserStack?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @i-love-code@rohit-y180

        Issue actions

          Is playwright supported? Tests fail but show as passed · Issue #16 · browserstack/node-js-playwright-browserstack