• Since implementing the plugin, we noticed that all our product pages weren’t being indexed anymore due to 404 errors. We were finally able to track down the issue and something about the way this plugin hides the product is also preventing Googlebot from accessing the page. This happens to all products that have either “product available for all countries” or “product NOT available for all countries”. Normal web browser user agents will pick up the pages just fine but it blocks certain bots which can have negative impacts on search rankings. Imo, this is a pretty serious bug that needs to be addressed.

    You can test the issue with this: https://botsimulator.com/index.php

    I can provide more details via email.

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Author kuldipzorem

    (@kuldipzorem)

    Hi @dassels43,

    Could you please share screenshots and more details of the blocks caused by certain bots that might negatively impact search rankings? I didn’t quite understand the exact issue with our plugin in your store.

    Thanks,
    Kuldip

    Thread Starter dassels43

    (@dassels43)

    Hi, sorry I missed your reply. Unfortunately I don’t have any screenshots of merchant center at the time of the issue as I was able to modify the code to disable the restriction for GoogleBot specifically. This is the code in question:

    // woo-product-country-base-restrictions/include/products-restriction.php

    /*
    * Check restricted by the product id for simple product
    *
    * @since 1.0.0
    */
    public function is_restricted_by_id( $id ) {
    $restriction = get_post_meta( $id, '_fz_country_restriction_type', true );


    // If Googlebot we exit so it can track the pages in GSC
    if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot"))
    {
    return false;
    }

    if ( 'specific' == $restriction || 'excluded' == $restriction ) {
    $countries = get_post_meta( $id, '_restricted_countries', true );
    if ( empty( $countries ) || ! is_array( $countries ) ) {
    $countries = array();
    }

    $customercountry = $this->get_country();

    if ( 'specific' == $restriction && !in_array( $customercountry, $countries ) ) {
    return true;
    }

    if ( 'excluded' == $restriction && in_array( $customercountry, $countries ) ) {
    return true;
    }
    }

    return false;
    }

    This block of code somehow affects certain user agents, like Google and Bing (but not Yahoo for some reason). I’m guessing it must be the way the bot accesses the page, leading to a 404 error if the product is set to be restricted. Could also be related to which country the googlebot server is coming from, I’m not sure. I added the line:

    // If Googlebot we exit so it can track the pages in GSC

    if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot"))
    {
    return false;
    }

    To exit the function if the plugin encounters googlebot. One could add other bots if necessary depending on their situation.

    I think you can replicate the issue by creating some products in Woo and setting some restrictions on them and then testing with https://botsimulator.com/index.php

    It was not an easy issue to track down because when testing the links on Google Search Console, the live test would work but the normal inspection from the indexer would return 404.

Viewing 2 replies - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.