Sitemap errors when in enhanced cache mode

  • Resolved ittone

    (@ittone)


    4 years, 1 month ago

    Hi Guys

    I changed my wordpress hosting to WPX Hosting last week for my main site which has been live for eight years now, having been on three different hosting platforms, all using Apache2 webserver and for most of that time utilising W3TC’s disk enhanced mode. All other functions of W3TC have always been disabled. Browser cache, minify etc.

    I love this plugin’s cache speed which is a good partner for my site.

    Now on WPX hosting which I believe is a Lightspeed server, I noticed on my Google console that my sitemap for https://www.backtothemovies.com was producing HTML header information errors instead of XML.

    After submitting my sitemap_index.xml over the last week several times, Google sees the xml file correctly the first time, but after approximately ten minutes, the next time sitemap_index.xml is accessed the html header error shows.
    As soon as I remove the code from my htaccess file, the sitemap is once again seen as an xml file.

    All tests sitemap tests have been carried out from https://www.xml-sitemaps.com

    Since I am using the Yoast SEO sitemap, I disabled the sitemap function and installed “Google XML Sitemaps” and this plugin also produced the same html error read behavior.

    When using W3TC cache program in the basic DISK mode there are NO problems with seeing the sitemap as an xml file at any time.
    When using WP-optimize cache program there are NO problems with seeing the sitemap as an xml file at any time.

    BUT when using W3TC in disk:enhanced mode, the errors occur.

    After raising a ticket for investigation at WPX on this problem and them saying I was the first to raise this problem then it is something to do with my plugins or .htaccess file that is causing the problem.

    The same plugins and htaccess file is the same as used on my previos host being Siteground….No issues in over four years.

    Now, I am using temporally a cache plugin that isn’t too bad (wp-optimize), but I still prefer W3TC in enhanced cache mode.

    So here is hoping you can look into my problem please and advise accordingly, as WPX hosting have now washed their hands of it.

    Unable to attach two w3tc config files for you to look at.

    Yoast sitemap_index.xml is being utilised.

    If you need me to do anything on the site, please let me know.

    Thank-you.

    The page I need help with: [log in to see the link]

Viewing 15 replies - 1 through 15 (of 18 total)
  • Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    Thank you for your inquiry and I am sorry about the issue you are experiencing.
    The interesting thing is that the issue started when you migrated to the other hosting service which always indicates the environmental issue.
    Please double-check if there is some kind of server-side caching and disable it to avoid conflict.
    You can also try excluding sitemap from caching:
    If you have “Do not process 404 errors for static objects with WordPress” enabled in Performance>Browser Cache, you should add the following:

    sitemap(_index)?\.xml(\.gz)?
[a-z0-9_\-]*sitemap[a-z0-9_\-]*\.(xml|xsl|html)(\.gz)?
([a-z0-9_\-]*?)sitemap([a-z0-9_\-]*)?\.xml

    to “404 error exception list:”
    In addition, in the “Never cache the following pages:” area in Performance>Page Cache.
    If Minify is enabled you should also add this to the “Never minify the following pages:” setting under Performance > Minify > Advanced (section).
    I hope this helps!

    Thread Starter ittone

    (@ittone)

    Hi Marko

    Thanks for the quick response.

    No server side caching but using WPX CDN service and not using w3tc browser cache only the cache module.

    If I exclude sitemap from caching would that not add extra processer utilisation etc?

    For the other recommendations would you also advise the same for basic disk caching, because that as I said works with no problem.

    Thanks

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    Thank you for the information.
    No, excluding sitemap won’t cause extra processer utilization.
    You can also use disk basic if that resolves your issue.
    Thanks!

    Thread Starter ittone

    (@ittone)

    Hello Marko,

    I have just checked another site of mine for sitemap errors and I am afraid to inform you that when using W3 total cache in disk enhanced mode on that site also produces the following on the test site:
    https://www.xml-sitemaps.com/validate-xml-sitemap.html

    XML Validation Results
    error Problem detected, please check details below
    Sitemap URL https://www.lifthillsandthrills.com/sitemap_index.xml
    Character Set
    Sitemap is valid No
    Warnings –
    Errors

    Incorrect http header content-type: “text/html; charset=UTF-8” (expected: “application/xml”)

    This is the section of W3TC that is causing the issue, and the exact area is on the second line.

    RewriteCond %{HTTP_COOKIE} !(comment_author|wp\-postpass|w3tc_logged_out|wordpress_logged_in|wptouch_switch_toggle) [NC]
    RewriteCond “%{DOCUMENT_ROOT}/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.xml%{ENV:W3TC_ENC}” -f
    RewriteRule .* “/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.xml%{ENV:W3TC_ENC}” [L]

    If the _index% is changed for test purposes to -index% all is OK and XML is passed from https://www.xml-sitemaps.com/validate-xml-sitemap.html.

    It appears that you may well have a ‘bug’ in your coding afterall.

    Thanks
    IT Tone

    Thread Starter ittone

    (@ittone)

    A little further information.

    When the sitemap_index.xml is tested from:
    https://freetools.webmasterworld.com/tools/site-validator/ I get the following output:

    XML Sitemap Validator
    <html><head><META HTTP-EQUIV=”Cache-control” CONTENT=”no-cache”><META HTTP-EQUIV=”refresh” CONTENT=”0;URL=/cgi-sys/defaultwebpage.cgi”></head><body></body></html> ———————————————————————————————————————————————–^ Fatal Error 76: Opening and ending tag mismatch: META line 1 and head on line: 1 column: 143 <html><head><META HTTP-EQUIV=”Cache-control” CONTENT=”no-cache”><META HTTP-EQUIV=”refresh” CONTENT=”0;URL=/cgi-sys/defaultwebpage.cgi”></head><body></body></html> ——————————————————————————————————————————————————————-^ Fatal Error 76: Opening and ending tag mismatch: META line 1 and html on line: 1 column: 163 -^ Fatal Error 77: Premature end of data in tag head line 1 on line: 2 column: 1 -^ Fatal Error 77: Premature end of data in tag html line 1 on line: 2 column: 1 <html><head><META HTTP-EQUIV=”Cache-control” CONTENT=”no-cache”><META HTTP-EQUIV=”refresh” CONTENT=”0;URL=/cgi-sys/defaultwebpage.cgi”></head><body></body></html> ———————————————————————————————————————————————–^ Fatal Error 76: Opening and ending tag mismatch: META line 1 and head on line: 1 column: 143 <html><head><META HTTP-EQUIV=”Cache-control” CONTENT=”no-cache”><META HTTP-EQUIV=”refresh” CONTENT=”0;URL=/cgi-sys/defaultwebpage.cgi”></head><body></body></html> ——————————————————————————————————————————————————————-^ Fatal Error 76: Opening and ending tag mismatch: META line 1 and html on line: 1 column: 163 -^ Fatal Error 77: Premature end of data in tag head line 1 on line: 2 column: 1 -^ Fatal Error 77: Premature end of data in tag html line 1 on line: 2 column: 1
    Validate sitemap from URL

    Test carried out on: https://www.lifthillsandthrills.com/sitemap_index.xml
    which is still currently in disk:enhanced mode

    Thanks
    IT Tone

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    Thank you for the information.
    However, I am unable to replicate the issue. I am using Yoast SEO sitemap and Disk: Enhanced. Tested the sitemap and it’s showing valid.
    I can confirm that I am seeing the same results when testing your sitemap. I’ve tried testing the sitemaps individually:
    https://www.lifthillsandthrills.com/post-sitemap.xml
    https://www.lifthillsandthrills.com/page-sitemap.xml
    https://www.lifthillsandthrills.com/category-sitemap.xml
    https://www.lifthillsandthrills.com/author-sitemap.xml
    And the results are showing that the test is passed and that the sitemap is valid.
    Again I am not able to replicate an issue and I am using Yoast also.
    Thanks!

    Thread Starter ittone

    (@ittone)

    Hi Marko

    How are you testing the sitemap?
    If you goto the URLs directly in a browser above they WILL show correctly.

    However, if you use an XML validation test site (below) as I have, you WILL get the errors. This is what Google have been seeing.

    https://www.xml-sitemaps.com/validate-xml-sitemap.html
    also
    https://freetools.webmasterworld.com/tools/site-validator/
    but this site appears to have a problem this morning. However the errors are showing as in my last post.

    I have just tried the xml-sitemaps.com site on the above urls you mentioned. They are all showing errors.

    Regards
    Sean

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    Here is what I am seeing when testing individual sitemap: https://imgur.com/a/tkML05X and https://imgur.com/sCxCMfY
    Have you tried excluding the sitemap from being cached with the instructions provided?
    THanks!

    Thread Starter ittone

    (@ittone)

    Hi Marko

    The sitemap reports as passed when the sitemap_index\.xml is added as per your advice. However, this is a workaround as the real issue appears to be how a PHP7.4
    module is interacting with the cached sitemap.

    As I said, when my .htaccess W3TC entries are removed then all is OK.

    I have just this minute cleared the cache from my Firefox, Edge and Chrome browsers…I have just cleared my server’s W3TC cache for lifthillsandthrills, tried the xml validation tests using the url:
    https://www.lifthillsandthrills.com/sitemap_index.xml

    and received this:

    XML Validation Results
    error Problem detected, please check details below
    Sitemap URL https://www.lifthillsandthrills.com/sitemap_index.xml
    Character Set
    Sitemap is valid No
    Warnings –
    Errors
    Incorrect http header content-type: “text/html; charset=UTF-8” (expected: “application/xml”)
    ————————————————————

    When the headers were examined, I recieved this:

    HTTP Headers Viewer
    Webpage URL
    https://www.lifthillsandthrills.com/sitemap_index.xml
    Response details
    Content size: 1,133 bytes
    Download time: 0.084s
    HTML Title: –
    ——————————————————-
    HTTP Headers found in response
    Response Code HTTP/1.1 200 OK
    Date Thu, 05 Nov 2020 14:31:37 GMT
    Server Apache
    X-Powered-By PHP/7.4.6
    Cache-Control max-age=0
    Expires Thu, 05 Nov 2020 14:31:37 GMT
    Vary Accept-Encoding
    Content-Length 1133
    Connection close
    Content-Type text/html; charset=UTF-8
    —————————————————–
    HTTP Headers Received
    HTTP/1.1 200 OK
    Date: Thu, 05 Nov 2020 14:31:37 GMT
    Server: Apache
    X-Powered-By: PHP/7.4.6
    Cache-Control: max-age=0
    Expires: Thu, 05 Nov 2020 14:31:37 GMT
    Vary: Accept-Encoding
    Content-Length: 1133
    Connection: close
    Content-Type: text/html; charset=UTF-8
    ————————————————————
    HTTP Request Sent
    GET /sitemap_index.xml HTTP/1.1
    Host: https://www.lifthillsandthrills.com
    User-Agent: Mozilla/5.0 (compatible; XML Sitemaps Generator; https://www.xml-sitemaps.com) Gecko XML-Sitemaps/1.0
    Accept: */*
    Connection: Close

    So how you are receiving a pass, I do not know.

    Can another reader of this thread please try the above sitemap.xml in the validator to confirm/not confirm my results please.

    IT Tone.

    Thread Starter ittone

    (@ittone)

    Marko

    In addition to my previous post
    Could you place this post at the top of the W3TC forum please so that we can get a third party test input.

    Thanks

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    Thank you for the info. Well now at the moment those sitemaps are no longer valid.
    Are you using Yoast sitemap? And one more time have you excluded the sitemap from the cache?
    Thanks!

    Thread Starter ittone

    (@ittone)

    Hi again Marko

    The site is still in disk:enhanced mode. (For your test purposes)
    The sitemap is STILL in cache.

    Once again, the sitemap reports as passed when the sitemap_index\.xml is added as per your advice. (ie: url to exclude), but this is a workaround, not a solution.

    Yes, I am using Yoast……But it does not matter, the html errors are also reported for Rank Math SEO sitemap.xml and also for Google XML Sitemaps sitemap.xml.

    Thanks

    Thread Starter ittone

    (@ittone)

    I forgot to add that the links you tested all passed, because you only tested each sitemap url once.

    By only accessing each one once places them into cache and they will report as passed. Any subsequent sitemap url will fail with html errors.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @ittone

    As I’ve stated before I was not able to replicate the issue. The fact is that the issue started to occur after you migrated to a new environment. This means that the issue is environmental, meaning specific to your hosting environment and your setup. The solution I provided, or the workaround as you call it, is helping out for your use-case.
    If the issue is with the W3 Total Cache specifically we would have hundreds of issues of this type already reported.
    So excluding the sitemap from caching is the only solution or using Disk: Basic.
    Thanks!

    Thread Starter ittone

    (@ittone)

    Hi Marco

    This original ticket was raised for my domain of backtothemovies.com which was migrated from siteground to WPX.
    Following this migration I noticed the sitemap error on google webmaster.

    The domain of lifthillsandthrills.com which we have been looking at since you told me to exclude the sitemap from backtothemovies.com (which no longer uses W3TC) is on my own LINUX server here at home, as stated above, has the sitemap error.

    This domain has the same issue and so does another domain hosted on another hosting company. (tonesworld.co.uk/sitemap_index.xml)

    You told me yesterday: “Thank you for the info. Well now at the moment those sitemaps are no longer valid.”

    I am now swapping all my five domains caching programs from W3TC which I have been using for numerous years.
    I know you only took over W3TC in 2019 but please remember that I highlighted this cache bug in W3TC enhanced cache mode during November 2020.

    Thanks for your assistance.

    IT Tone
    Linux Technical Support Manager.

Viewing 15 replies - 1 through 15 (of 18 total)
  • The topic ‘Sitemap errors when in enhanced cache mode’ is closed to new replies.