Sucuri != Fake Google Crawler

Did you whitelist it on the options page?

tim

@tim
Well, I guess I could, but isn’t it better if WordFence makes that distinction by itself? The sucuri checker is not a fake google crawler, and all users of WordFence might benefit if that is detected correctly. I know that the IPv4 sucuri address is whitelisted, so maybe that can be extended. Or maybe Sucuri might be persuaded to add something that WordFence can use to make the distinction.

Any further info about this?
I would not like to whitelist the Sucuri IP’s (don’t want to help spoofers), but it would be helpful if WordFence doesn’t recognize them as Fake Crawlers anymore. This has been reported before (with their IPv4 address), but apparently there now also is an IPv6 address. The previous issue was fixed by Matt.

@allm: Yes, we do add their new IPs to the plugin once we can verify them (usually Sucuri has reverse DNS entries for their servers that we can verify with a forward lookup, but this one currently doesn’t yet). It might take some time before it’s included in the next release, so whitelisting it is the best temporary solution, so your scan can finish correctly. Alternately, you can turn off the option to block fake Googlebots when running a scan.

The reason they get blocked is because their scanner identifies itself with a Googlebot “user agent” during part of their scan, which is the same thing that the fake Googlebots do when trying to crawl your site.

-Matt R

Hi Matt,

Thanks for the update. And thanks for the advice. I know why Sucuri gets flagged as a fake crawler, and I realize what (sub-optimal) solutions I can take.

I hope this new address gets fixed in the next release.

Thanks for looking in to it!

The service you are giving in this forum is really great.