GoogleBot Rendering – but clear robots.txt

  • Hi there guys

    I seem to be having an issue with GoogleBot rendering my site correctly (juxtalegal.com). I attempted a mobile usability test and (although the site is optimised for responsiveness and works perfectly on a phone) I kept getting changing results – first saying a pass but with errors, then saying a fail, and also reporting that it was unable to fetch the URL.

    I’ve since delved deeper, and tried fetch and render in Webmasters.

    Having tested fetch and render for Desktop and Smartphone – it appears users see the site fine, but GoogleBot is not correctly rendering the page. I’m receiving a ‘Googlebot couldn’t get all resources for this page.’ error, with a large list of URL’s – all stating ‘Temporarily Unreachable’. I’ve also intermittently received a general ‘Temporarily Unreachable’ error when attempting fetch&render for smartphone devices – albeit this is intermittent and changes between that error and ‘Partial’. I’ve checked my robots.txt and I don’t have anything blocking GoogleBot at all.

    Having delved deeper I noticed my site is currently running 20 External CSS and 70 External JS. Could this possibly be the reason?

    It all seems very odd. The site was working perfectly just 2 weeks ago.

    Any assistance would be much appreciated.

    Thanks, Tom.

Viewing 7 replies - 1 through 7 (of 7 total)

  • When I look at your robots.txt file, I see:

    User-agent: *
Disallow:

    Have you tried deleting your robots.txt file entirely (or just renaming it) and re-running the Google mobile-friendly test?

    I’m not sure, but it sounds like Google is being blocked from some of your CSS and JS, and that weird entry in your robots.txt could be the cause.

    Thread Starter tomjuxta

    (@tomjuxta)

    Hi there linux, thanks for the response.

    Yes, tried unfortunately. No success. I don’t think GoogleBot is being blocked from reading my CSS/JS per se, more it seems unable to read them.

    Rather than a ‘blocked’ notification in Webmasters, I’m receiving a ‘Temporarily Unreachable’ for them all.

    Maybe it’s the external sites where your CSS and JS are hosted that are causing the problem? That would explain why it was working before and without any changes at your end, it stopped working.

    Thread Starter tomjuxta

    (@tomjuxta)

    It doesn’t seem to be this either unfortunately. All the URL’s listed are my domain URLs.

    The majority are:

    /wp-content
    /wp-includes

    Oh, when you said “external” I thought you meant you were loading CSS and JS from other domains.

    I’ve had trouble with sites I knew were responsive not passing Google’s mobile-friendly test, but it has always been due to errant restrictions in the robots.txt file or overly large images (> 300 px). If you find a solution, please post it. I’m curious to see what the cause is.

    Thread Starter tomjuxta

    (@tomjuxta)

    Sorry – my bad!

    I will do. I’ll keep plodding on until it’s fixed!

    Thanks, Tom.

    Hi,
    Website https://www.epnet.co.za

    I am having the same problem one of my files is as follows:

    htt?p:/?/ww?w.e?pne?t.c?o.z?a/w?p-c?ont?ent?/th?eme?s/c?loc?kwo?rk/?css?/fl?exs?lid?er.?css??ve?r=4?.1.?7
    Google returns: Style Sheet Temporarily unreachable

    The file is contained in the path and is not being restricted by anything I cannot understand it? Anyone know how to fix?

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘GoogleBot Rendering – but clear robots.txt’ is closed to new replies.