Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete Edge Dev results #34913

Open
foolip opened this issue Jul 20, 2022 · 2 comments
Open

Incomplete Edge Dev results #34913

foolip opened this issue Jul 20, 2022 · 2 comments

Comments

@foolip
Copy link
Member

foolip commented Jul 20, 2022

In web-platform-tests/interop#96 I found differences between Chrome and Edge in the form of missing results from Edge. Here are two runs from the same commit of wpt:

That's 2829 tests fewer. @mustjab have you seen such differences in the past, is it a known issue? Incomplete results will make it difficult to use the Edge results for spotting differences to Chrome, and other uses.

@foolip
Copy link
Member Author

foolip commented Jul 20, 2022

Taking another pair of runs, from the previous commit:

The Chrome number is the same but the Edge number is not. This suggests there's some flakiness involved.

@mustjab
Copy link
Contributor

mustjab commented Jul 20, 2022

We've seen differences between Chrome and Edge runs, a lot of these differences are due to diff between runs on Linux vs runs on Windows. Don't believe we ever investigated the reason behind these differences.

Are we able to run Chrome runs on Windows for comparison?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants