Googlebot and Bing getting blocked

Ok, I’ve whitelisted all old urls and some spiders. Probably there is no other solution…

What causes are shown? What URL are shown on the Activity tab?

See also:

https://www.ads-software.com/support/topic/crawl-66-249-64-77-googlebot-com-probing-for-vulnerable-php-code/

https://www.ads-software.com/support/topic/locking-out-google-bot-services/

Probing for vulnerable PHP code Denied as usual, but most of them are valid previous URLs from the old site’s version. I whitelisted them, should be ok now. had to insert about 150 old URLs, as they are different from the new ones.

p.s. wordfence has option not to limit google bots at all, could be useful.

Unfortunately, there are plenty of fake googlebots out here. Secondly detecting domain name for each incoming request can drastically slow down a website. That’s why we consider this approach as not useful.

The right way is to whtitelist the extension of old pages (URLs) of the website. For instance if old pages had the asp extension:

{.+\.asp$}

Please read more: https://wpcerber.com/wordpress-probing-for-vulnerable-php-code/

Hi Gioni,

My visitors and Google get blocked too, but mine is to do with “Probing for vulnerable PHP code” all the time. I’ve tried to redirect as many pages as possible, but they are blocked after 1 “Probing for vulnerable PHP code” request.

Can you increase the “Probing for vulnerable PHP code” to be more than or equal to 2-3 times? Then if visitors come to the website through a VERY old link – they won’t be blocked immediately?

Thanks,
S.

Hi!

@ss88_uk
I agree the built-in firewall rules a bit restrictive for some cases. A new setting will be added soon and it enables you to set a less aggressive mode for the firewall. For now, you can add {.+\.php$} to the whitelist as described here: https://wpcerber.com/wordpress-probing-for-vulnerable-php-code/

Thanks Gioni! Really appreciate the new setting!

Yes, I used .shtml (very old –see!) for mine, thank you.