Weird repeating anomaly with test results

Hi @asafm7,

This is odd. I did some manual testing and did not have the issue you describe.

Could you try to add a click to the consent management popup to see if this changes anything? For that, please add .cmplz-btn.cmplz-accept to the settings field Click an element before creating a snapshot.

If that does not help, we will continue to tweak the screenshotting service to get more consistant results. Additionally, in the next version of the plugin that we will publish soon, you have the ability to mark a comparison / test as false positive. So for this exact diff, you can prevent it from becoming an alert.

I will update you, once we have released the new version. Until then, I hope my other suggestion can solve this issue.

Thanks, @domtra.

I am actually already clicking the cookie banner in one of the tests, and this anomaly does happen there as well.

The new false-positive feature sounds helpful.

I’m curious about the logic, what will be used to ‘identify’ a false positive alert? I mean, how does the system know that a future alert is identical to one I already marked as a false-positive? and couldn’t it create false negatives? In this example, one day the email address may really disappear, but the alert will automatically be disregarded as false-positive.

Just some thoughts that I had. Don’t feel obligated to explain. I will play with the feature once released ??

Let me know if I can provide any more details for debugging.

Hi?@asafm7,

we have released the false positive feature in version 1.8.0 of the plugin. It is already available. Would be great if you could give us some feedback about how this works for you.

Not generating false negatives was really important for us. This is why we opted for a pretty strict definition of false positives. When you have an alert, you see the diff of the two images. A mostly transparent image with some red pixels in it. We compare these diffs to those diffs that you marked as false positives before. So only if the differences between two comparison are exactly the same, it will count as a false positive.

Hope you understand what I mean.

I am going to resolve this thread. Feel free to contact us with further questions or feedback.

Thank you, @domtra.

I’ve noticed the feature and have already utilized it. It’s very useful.

Thank you for a great plugin!