Some pages are no langer cached after a day, but no purges to be seen

  • Resolved webreturn10

    (@webreturn10)


    2 years, 3 months ago

    Dear developers of LiteSpeed Cache,

    We as a web design agency are happily using your plugin on over 100 websites, but there is one issue that is hard to grasp and debug: random pages appear not to be cached anymore after a day, even though the public cache TTL is set to a week. Here are some details:

    Website: https://www.webreturn.nl/
    Report number: NOMPNBLE
    Plugin version: 5.3

    We have used an external cache warmer tool to warm pages overnight, but it seems like some pages still indicate a cache miss instead of a cache hit. We’ve have read through the list of common problems and their solutions, and issue “Cache Purges Too Frequently” describes our issues perfectly, but there were no indications in the debug log that suggests that the public cache has been emptied between the 29th and 30th of November.

    There have been no changes to the website, not even admin logins from the 29th of November until today, the 30th of November. We are interested in what could cause this issue, since we have brainstormed about what could trigger a cache purge, only to find no proof of public cache purges.

    We have also taken a look online for potential causes, of which one suggested that it is possible that items could be removed from the cache if it has to make room for other items.

    Now we don’t think that that should be the case, since our clients have plenty of disk space, but is there a checklist that could be done by the hosting provider to check their LiteSpeed server configuration?

    We are willing to make use of a cache warmer or develop one for ourselves, since our hosting provider disabled the crawler of your plugin by default for clients that do not have a VPS.

    Do you have any tips on other cache crawlers that can recognize what pages are cached and warm those overnight, or is there perhaps an API in LiteSpeed Cache’s core that we could use to get all cached pages to prevent warming those while crawling an XML sitemap?

    Kind regards,
    Thijs Terluin

    The page I need help with: [log in to see the link]

Viewing 7 replies - 1 through 7 (of 7 total)

  • We have used an external cache warmer tool to warm pages overnight, but it seems like some pages still indicate a cache miss instead of a cache hit.

    External cache crawlers don’t work for this because they don’t support the LiteSpeed LScache cache varies.

    Do you have any tips on other cache crawlers that can recognize what pages are cached and warm those overnight, or is there perhaps an API in LiteSpeed Cache’s core that we could use to get all cached pages to prevent warming those while crawling an XML sitemap?

    Yes, there is such one, but this forum doesn’t allow to promote it. It’s a standalone crawler installed in parallel to WP installation that is especially made for WP, x-times faster and causes only half of the server load.

    Thread Starter webreturn10

    (@webreturn10)

    Thank you for your response. We have also tried a plugin (Automatic Cache Warmer) instead of an external cache crawler and it did publicly cache pages (tested while logged out).

    I don’t think the plugin has any code that ensures compatibility with the LiteSpeed Cache plugin in particular (since LiteSpeed Cache does not recognize the cache warmer’s mobile user agent as a mobile device).

    Should that mean that I could create a cache warmer plugin by myself without having to dig LiteSpeed cache’s code to see how to support LScache varies?

    Are LScache varies necessary to be supported if I only want to warm the public cache in order to serve cached pages for people who land on pages via Google for the first time?

    Why don’t external cache warmers work for warming the public cache, but plugins do work?

    Could you please also take a look at my questions regarding parts of the cache being emptied after a day? This is our #1 priority, since warming the cache would have little to no effect as parts of the cache are gone after a day.

    Thank you for your response. We have also tried a plugin (Automatic Cache Warmer) instead of an external cache crawler and it did publicly cache pages (tested while logged out).

    If you enabled Mobile View or webp replacement or Guestmode such external crawlers can’t warmup the cache because they don’t support cache varies, so only the basic cache profile (public) is warmed up with such crawlers.

    Should that mean that I could create a cache warmer plugin by myself without having to dig LiteSpeed cache’s code to see how to support LScache varies?

    Give it a try, but if you don’t know which headers to use, you’ll fail. You must also be able to make the crawler multithreading capable. Otherwise the crawling takes much too long and slower than the built-in crawler.

    Are LScache varies necessary to be supported if I only want to warm the public cache in order to serve cached pages for people who land on pages via Google for the first time?

    That depends on your settings. If you have Guestmode enabled you need cache vary support.

    Why don’t external cache warmers work for warming the public cache, but plugins do work?

    This hasn’t something to do if a crawler is external or a plugin. Only cache varies matter. No cache vary support no warmup, but depending on your settings.

    Could you please also take a look at my questions regarding parts of the cache being emptied after a day? This is our #1 priority, since warming the cache would have little to no effect as parts of the cache are gone after a day.

    There is no way to check which URL was purged, when and why. You can only check the current cache status.

    Thread Starter webreturn10

    (@webreturn10)

    Guest mode is enabled. If I understand correctly.. if I enable WebP for browsers, then a separate cache should be warmed for those browsers that support it. So should that be indicated by request header accept with the value containing image/webp?

    We don’t have WebP optimization enabled, but we do have a separate cache for mobile devices, so that should mean I would have to send request header user-agent with a value like Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1 Edg/107.0.0.0 in order to be able to warm that part of the cache?

    I’ll take a look into multithreading, thank you!

    Which of the following headers do I need to send besides accept and user-agent?

    1. accept-encoding
    2. accept-language
    3. cache-control
    4. sec-ch-ua
    5. sec-ch-ua-mobile
    6. sec-ch-ua-platform
    7. sec-fetch-dest
    8. sec-fetch-mode
    9. sec-fetch-site
    10. sec-fetch-user
    11. upgrade-insecure-requests

    Since there are no cookies yet for new visitors, that should mean I won’t have to send a cookie request header, right?

    I understand that I can not check what URLs are being purged, since they don’t appear in the debug purge log. However, I want to know whether there are any server related issues or configurations that could lead to parts of the cache being removed after a day.

    • This reply was modified 2 years, 3 months ago by webreturn10.

    Which of the following headers do I need to send besides accept and user-agent?

    All you need to know can be found in .htaccess.

    Since there are no cookies yet for new visitors, that should mean I won’t have to send a cookie request header, right?

    If no cookie is sent by the client (or crawler) and if you have Guestmode enabled you can only cache (warmup) the first view. Guestmode requests a URL twice if the needed cookie doesn’t exist and for each request a different cache copy is generated. You only have to look in .htaccess. There you can find all informations.

    I understand that I can not check what URLs are being purged, since they don’t appear in the debug purge log. However, I want to know whether there are any server related issues or configurations that could lead to parts of the cache being removed after a day.

    I’m not aware of any issue. If the cache of certain URLs was cleared before the TTL expired, you need to check the plugin settings. Nothing happens without a reason.

    Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1 Edg/107.0.0.0

    Too much information that causes too much load and takes too much time. For the crawler you need only “iphone Mobile” and version number if you use webp replacement, but the current defines in cache plugin are buggy…..

    • This reply was modified 2 years, 3 months ago by serpentdriver.
    Plugin Support qtwrk

    (@qtwrk)

    for warm cache with external service

    please take care of varies , like mobile-desktop version , webp and non-webp version , also guest mode and normal mode

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘Some pages are no langer cached after a day, but no purges to be seen’ is closed to new replies.