domtra

Hi @asafm7,

I looked at your screenshots in our database. The way we save the false positive is via a checksum of the diff image. That means not only that the differences need to be at exactly the same spot, but also that the rest of the image must not differ.

At least if you compare the screenshots from 05/29 and 06/16 to 06/19 and 06/19, you will see that at the bottom of the screenshot there is a small horizontal rule at the beginning of the footer.

For the other one, I cannot make out a visual difference but the checksum from the comparison is definitely different. I can also see in our DB that there are a few comparisons marked as false positives and some alert were filtered out. It just also seems that from time to time the checksum changes.

So from our side, the feature is working as expected. But we will keep your case in mind and see if we, in the future, make some changes to how a false positive is identified.

Hi @asafm7,

just to let you know, we have implemented a general 2 second delay for all screenshots to better deal with some of the loading issues. Currently there are no plans to add an option for additional delays.

Let us know if you have additional feedback or questions.

Hi?@asafm7,

we have released the false positive feature in version 1.8.0 of the plugin. It is already available. Would be great if you could give us some feedback about how this works for you.

Not generating false negatives was really important for us. This is why we opted for a pretty strict definition of false positives. When you have an alert, you see the diff of the two images. A mostly transparent image with some red pixels in it. We compare these diffs to those diffs that you marked as false positives before. So only if the differences between two comparison are exactly the same, it will count as a false positive.

Hope you understand what I mean.

I am going to resolve this thread. Feel free to contact us with further questions or feedback.

Hi @asafm7,

This is odd. I did some manual testing and did not have the issue you describe.

Could you try to add a click to the consent management popup to see if this changes anything? For that, please add .cmplz-btn.cmplz-accept to the settings field Click an element before creating a snapshot.

If that does not help, we will continue to tweak the screenshotting service to get more consistant results. Additionally, in the next version of the plugin that we will publish soon, you have the ability to mark a comparison / test as false positive. So for this exact diff, you can prevent it from becoming an alert.

I will update you, once we have released the new version. Until then, I hope my other suggestion can solve this issue.

@asafm7, it seems that previously there were some issue with timeout during the screenshot phase which were not resolved properly. This may result in some test being stuck.

I tried to get them unstuck for you and it should have worked at least for one of your tests. Can you confirm that?

If there is still an issue, can you try to disable the test and the undo via the link in the admin notice? After that, the test should be set up properly again.

Sorry for the inconvenience.

Hi @blackdranzer,

we have an update for you.

We do support multisites now!

For new installs it will just work out of the box. If you already have our plugin installed and run into issues, you need to delete the plugin and then manually delete some tables and options in your database.

Depending on your db prefix and the number of sites you have installed, the following queries need to be executed. Please be sure to make a database backup, before executing the queries.

DROP TABLE IF EXISTS wp_vrts_tests;
DROP TABLE IF EXISTS wp_1_vrts_tests;
DROP TABLE IF EXISTS wp_2_vrts_tests;
...

DROP TABLE IF EXISTS wp_vrts_alerts;
DROP TABLE IF EXISTS wp_1_vrts_alerts;
DROP TABLE IF EXISTS wp_2_vrts_alerts;
...

DELETE FROM wp_options WHERE option_name LIKE 'vrts_%;'
DELETE FROM wp_1_options WHERE option_name LIKE 'vrts_%;'
DELETE FROM wp_2_options WHERE option_name LIKE 'vrts_%;'
...

I hope this helps. Let me know if everything works as expected now.

Hi @asafm7, and thanks for the link.

The problem on the site you provided seems to come from Chrome itself. This is the browser we use to take screenshots. If you navigate in Chrome to your website and take a full page screenshot via the devtools, you will see, that although the iframes have been loaded, they will not show up on the screenshot. Chrome is somehow hiding them, when they are not in the viewport and fading them in, once they are.

Firefox, for example, behaves differently.

I have found a solution to this. But the technique might introduce issues in other situations. I will need to do some more tests and we nee to further evaluate if our current approach leads in general to better results, or the new one we came up with.

I will keep you posted about the progress. And sorry for your inconvenience.

Hi @asafm7,

thanks for your feedback.

There is currently no manual delay you can add while taking a screenshot. We have a lot of checks in our code that wait for the page to be finished load and the network being in idle.

There are some edge cases when this does not work.

If you can give me the URL of the website where you experience these issues, I might be able to have a look and find a solution to solve this.

Best, Dominik.