Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Be able to tweak the toHaveScreenshot config setting based on a failed test and some kind of slider/preview mechanism #33888

Open
mduivcap opened this issue Dec 6, 2024 · 0 comments

Comments

@mduivcap
Copy link

mduivcap commented Dec 6, 2024

🚀 Feature Request

In the playwright.config.ts you can add maxDiffPixelRatio, threshold, etc.
You can also tweak this per test.
If a test fails, you know it went over the boundairies.
But it's not very clear how much. So I notice that I quiet often spend some time tweaking this, by adjusting the settings, run the test again until I have it set to whatever I see fit for this test.

I would like to have a option, in UI mode for example, to see a test fail and than also see directly what the values should be to still pass. Like an advise maybe. And/Or it would also be very handy if you can just play with a slider and see what that does to the validation.

Example

We have a general setting as a baseline. And with images and text slightly different rendering will trigger it, so you might want to set these settings a bit more forgiving (since we have this running within our pipeline and it should pass and a slightly different rendered text is fine with us). However we do have screenshots that can contain elements that have chevron icons in front of some text. Now I've seen that this chevron became a square block, but because the settings where set to be slightly forgiving, this test passed while it should have failed (and the other way around)
Of course I could tweak the setting, run the test again, and with some trial and error I'll figure out the correct config for this test.
But it would be nice if you could "play" with the settings without touching the code just yet and when you figured out the correct setting, adjust the code and you should be good to go.
Image

Motivation

It will safe time to figure out the right balance for specific tests.
And you have a visual way to tweak your visual related configurations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants