Blocking Google and MSN Crawler

Try to whitelist those IP′s ??

“Try to whitelist those IP′s”

That of course is what I did. But the fact that the plugin blocks the all important search engine crawlers is disturbing. Not sure about you, but I work hard to get them to my site(s) and then them being blocked is just not acceptable.

Sounds like this must be a common problem if people are just whitelisting. Best that I go back to old plugin:)

I do not have them whitelisted and they are not blocked so far ??
Did that started to happen since latest version of the plugin?

“Did that started to happen since latest version of the plugin?”

Did not update to latest until this morning and noticed the blocking at that time. So as image shows, was 1 and 2 days ago when blocked Google and MSN.

Disabled the plugin for now. Just can’t afford to have it blocking the two most important search engines on the net:)

thanks.

Hi!

The plugin doesn’t prevent search engines from crawling normal, public pages and indexing a website.

Please check those blocked URLs. Most likely your website was infected like this one: https://www.ads-software.com/support/topic/crawl-66-249-64-77-googlebot-com-probing-for-vulnerable-php-code/

“Most likely your website was infected like this ”

No sir, site is not infected. Standard WP install with no modifications.

I have removed the plugin and had google bot return and my webmaster panel at Google no longer shows crawl errors.

Was going to try and work with you and find real reason, but you have marked as “resolved” so guess in your eyes the problem is mine alone and does not really exist:)

Wishing you the best.

• This reply was modified 6 years, 9 months ago by Terry J.

If you’re sure that your site is not infected or altered, you can ignore all those messages because The plugin doesn’t prevent search engines from crawling normal, public pages and indexing a website.

Normally none of the search engines should crawl those suspicious/non-existing/fake URLs. Sometimes poorly designed plugins add lame links (URLs) to the content of your site and crawlers try to index them. Sometimes that might lead to sensitive data leakage. Anyway, all those requests may mean there is/was something wrong on your site. Also it’s possible that someone has put those links somewhere on the Internet because they want to compel you to disable protection.