Blocked real Googlebot instead of Fakebot!

  • Resolved lydiaflue

    (@lydiaflue)


    1 year, 1 month ago

    A client of our has installed the plugin for years. In the Tab “Firewall” and the sub-tab “Internet Bots”, they activated blocking fakebots. But starting on August 22nd, it suddenly started blocking not only fakebots, but the real Googlebot as well – leading to a deindexation of the entire platform! It took us a while to find the problem and we now de-activated blocking fakebots. But I wanted to make sure, everyone’s aware of this problem. And it would be nice to know if any changes will be made so that, in the future, only fakebots will be blocked, and not the real bot.

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Support hjogiupdraftplus

    (@hjogiupdraftplus)

    Hi @lydiaflue

    As you are saying issue started after the August. We have recent change related to Block fake Google bot is if the bot IP address do not resolve to a hostname it will consider it as fake bot.

    Here if it is correct google bot it will have Googlebot as user agent and the Server IP address host lookup will have “google”

    So Please cross check it do have correct visitor IP address detection under WP security > Settings > Advanced settings and the IP address the bot tries to fetch and its host lookup shows it is a google related host.

    Do you have the IP of that google bot ? So I can cross check it.

    Regards

    Thread Starter lydiaflue

    (@lydiaflue)

    Thank you. But under WP security > Settings > Advanced settings, I can only see which IP address is allocated to me. I cannot see any IP addresses from any real of fake Googlebots.

    Plugin Support hjogiupdraftplus

    (@hjogiupdraftplus)

    Hi @lydiaflue

    I am asking the IP address of the google bot which try crawl and blocked not allowed to access the site.

    WP security > settings > advanced setting is regarding IP detection method. If it is correct it will detect correct IP address of user / client / bot.

    you can cross check it with your IP address and https://whatismyipaddress.com/ detected correct one.

    Regards

    Thread Starter lydiaflue

    (@lydiaflue)

    We don’t know the IP address. We don’t really know which IP addresses Google uses to crawl the website – all we know is that suddenly, the Googlebot was excluded from crawling by a setting in your plugin. When we deactivated it, the Googlebot crawled again. So please check what you have changed. I cannot imagine we are the only ones with this problem.

    You don’t have to reply – I just wanted to make sure you’re aware of this problem.

    • This reply was modified 1 year ago by lydiaflue.
    Plugin Support hjogiupdraftplus

    (@hjogiupdraftplus)

    Hi @lydiaflue

    Block fake Google bots enabled, google can not crawl site issues reported only by you. I have kept an eye on one of my own sites and on 23 Sep it is showing as crawled. Also, I just tested using Google Search Console Live Test a URL to crawl and it worked.

    https://snipboard.io/PX9o8Q.jpg

    Regards

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Blocked real Googlebot instead of Fakebot!’ is closed to new replies.