Bots not seeing files as concatenated?

  • Recently, many CSS and JS assets on our live site began appearing individually in our Google PageSpeed reports, when most of them were supposed to be concatenated and minified by W3TC (manual settings). Our PageSpeed score is halved and our GTmetrix load time has doubled.

    I checked a staging version of our site in PageSpeed and GTmetrix, and it’s fine: the files show up as concatenated and minified. I tried the live site in Chrome with Lighthouse, and it’s fine also. And I cross-checked this by inspecting the html head with Chrome developer tools: it requests the appropriate concatenated and minified files. But when I analyze the live site in PageSpeed and GTmetrix, they receive all the JS and CSS as individual files, not concatenated.

    We use W3TC with Cloudflare and CacheWall. Initially suspecting a problem with incorrect files cached, I tried clearing the cache in all three and turning on dev mode in Cloudflare and CacheWall to bypass the cache. No luck, and I can’t even imagine why the cache would send different files to different viewers anyway. There doesn’t seem to be a setting in W3TC to show non-concatenated files to bots (which is how I’m presuming it sees PageSpeed and GTmetrix).

    I’ve even let the site sit over the weekend in case something needed to propagate or work itself out. And Googling my issue yields nothing. I’m running out of debugging ideas and concerned about PageSpeed effects on our search rankings.

    Do you know why PageSpeed and GTmetrix could see the site differently than my browser, with the measures I’ve described in place?

    The page I need help with: [log in to see the link]

Viewing 13 replies - 1 through 13 (of 13 total)
  • Thread Starter Aaron

    (@aaronatfpc)

    Update: It appears that my browser only sees the live site files as concatenated when I’m logged into WordPress. I do have the setting turned on that says “Don’t cache pages for logged in users”.

    • This reply was modified 5 years, 9 months ago by Aaron.
    • This reply was modified 5 years, 9 months ago by Aaron.
    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @aaronatfpc,

    You need to revise your Manual minify settings. Manual minify can be tricky sometimes as it requires files to be added in proper order. Some files you just can not minfy. The part where it gets tricky is when you have some 3rd party files or some other plugin files.

    Thread Starter Aaron

    (@aaronatfpc)

    Thanks. Which setting would lead to site files only being concatenated for logged-in users?

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello,
    In Performance>Minify you can select Disable minify for logged in users box

    Thread Starter Aaron

    (@aaronatfpc)

    My “Disable minify for logged in users” is not checked. Yet The issue is that I get concatenated/minified files ONLY as a logged in user. I need my site visitors to receive them as well.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello,
    I can see that you are using Cloudflare. Are you using both Cloudflare and W3 Total Cache minify? If so, Please disable one of the minify solutions. It is not recommended to use both Cloudflare and W3 Total Cache minify.

    Thread Starter Aaron

    (@aaronatfpc)

    As noted in the opening post, we use W3TC with Cloudflare and CacheWall. I have already tried disabling the other two, and it did not fix the problem. I’ve also checked, and Cloudflare auto-minify was never on.

    • This reply was modified 5 years, 9 months ago by Aaron.
    • This reply was modified 5 years, 9 months ago by Aaron.
    Thread Starter Aaron

    (@aaronatfpc)

    After examining the site a more broadly, I’ve found something odd: the problem as I’ve described it only occurs on my home page. Again, I only get concatenated files when logged in on that page.

    But on other pages, I get them concatenated just fine when not logged in. Except WooCommerce pages, when I never get them regardless.

    I have manual minification set up for 404, page, page-full-width, and search templates, but not WooCommerce, so that may explain the WooCommerce part, but not the home page.

    What can you suggest I try next?

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello,
    I spoke with the team and this is a very interesting use case.
    Unfortunately, we cannot advise anything without debugging your server.
    In this case, I can offer you our professional support.

    Thread Starter Aaron

    (@aaronatfpc)

    Sorry for the delayed response; I had another problem to solve. I would love to speak with you further about the professional support.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello,

    You can go to Performance>Support select “Hosting Environment Troubleshooting”. With this solution, the developer will debug any problems with your web hosting setup clashing with the plugin.
    Please include the link to this topic in the message.

    Thread Starter Aaron

    (@aaronatfpc)

    Ah, and that would be a paid service?

    Thread Starter Aaron

    (@aaronatfpc)

    How about this: for now, while I work on a better solution, how would I go about preventing W3TC from touching the home page? At least with respect to minification/concatenation. This seems to be causing errors or cosmetic issues with that page. When I disable the W3TC plugin, those errors go away.

    I can see the “Never minify the following pages:” field. But it warns me to omit the leading “/”, so I cannot simply enter “/” for the home page.

Viewing 13 replies - 1 through 13 (of 13 total)
  • The topic ‘Bots not seeing files as concatenated?’ is closed to new replies.