• Resolved futureyoon

    (@futureyoon)


    Hi

    I have enabled the preload with sitemap and I can see that a folder is getting renerated under /page_enhanced/mydomain/[xml_based_list]

    but one thing I noticed inside of the each folder

    For example,

    In the folder of /page_enhanced/mydomain/page1/

    I can see 2 files

    ‘_index_slash_ssl.html_br’

    ‘_index_slash_ssl.html_gzip’

    I guess it’s because enabled brotli and gzip.

    Interestingly, only when I manually visit the page1 on the frontend, i see another file generated ‘_index_slash_ssl.html’

    I believe the preload process should create the ‘_index_slash_ssl.html’ as well instead of being created by visiting the page manually, am I right?

    If so, why it is not generating the file? Did I mis-config something?

    Please confirm and guide me on this

    Thank you so much

Viewing 7 replies - 1 through 7 (of 7 total)
  • Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @futureyoon

    Thank you for reaching out.

    The Cache Preload in W3 Total Cache is a straightforward config, and once enabled and the sitemap URL is added to the Sitemap URL field, Cache Preload starts to work.

    I’ve tested this and I am getting _index_slash_ssl.html for the preloaded pages with _index_slash_ssl.html_gzip (Do not have br on this env)

    Can you please let me know for how long have you let the W3TC to preload the website and also, is the wp-cron enabled?
    What are the settings for the Update interval and pages per interval?

    THanks!

    Thread Starter futureyoon

    (@futureyoon)

    Oh now I see ‘_index_slash_ssl.html’ file after gzip and br created.

    I have a few further questions

    For testing purpose,

    I set Update interval : 300 seconds & Page per interval : 100

    Q1: Does this mean that this plugin will create folders and files under page_enhanced/mydomain/ (as part of preload process) for 100pages in every 300 seconds?

    Q2: What if it fails to generate the 100pages in 300secs (too short)? what will happen?

    Q3: After 300secs (after 1st run), the plugin will do preload for those pages not yet created from the 1st run (100pages from 1st run, 101th pages onwards from 2nd run)? or it will reset all from 0

    Q4: It’s related to Q3. What will happen to those preloaded pages after 300 secs? it will become obsolete?

    Q5: is there any cronjob involved with preload? if so, what is the name of the job?

    I am trying to understand how it behaves so that I can possibly optimize the settigns for my case.

    Thank you

    • This reply was modified 2 years, 1 month ago by futureyoon.
    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @futureyoon

    Thank you for your feedback.
    Lowering the Update interval and increasing pages per interval can impact the server performance. It is a lot to cache 100 pages at once every 5 minutes. This is why the default value is the best, it may preload the entire website slower, however, the impact on the server is minimal.

    1. Yes that is correct however you should avoid this as it consumes to much serevr resources.
    2. The remaining pages will be cached on the next run along with the old pages.
    3. It will not reset to 0. It will continue preloading the remaining pages accourding to your sitemap URL.
    4. Once the pages are preloaded, they are cached and remaining cached untill the next run. Once the pages are preloaded again the _old files are created which are then being temoved by the GC interval
    5. Yes, W3TC does use wp-cron for this. w3_pgcache_prime for priming the pages every x minutes(seconds) and w3_pgcache_cleanup for deleting the _old files.

    Thanks!

    Thread Starter futureyoon

    (@futureyoon)

    @vmarko Thank you so much for the details!

    I understand your concern on the server. it’s just for the testing purpose.

    Regarding Q4,

    the next run is determined by ‘Update interval (which i set as 300secs)’ or where is it being determined (which setting)?

    Q6: What will happen to the cronjob (w3_pgcache_prime) when all the pages are being preloaded? it will not auto stop, right? Similarly, how does system know that there are no more pages to be pre-loaded?

    Q8: Is there a way to skip some pages not to be preloaded? I am running bb press based site with a few thousands members. It goes like members/aria, members/arisha, members/bruce…
    Although my xml file doesnt include all these members, (only 1 entry for ‘members’), I see the system is trying to preload all those members, which will create lots of load on the server and members page is only for logged-in users so no point to doing the preloading. Can you guide me on this?

    Thank you so much for your detail info with expertise.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @futureyoon

    Thank you for your feedback.

    The Update interval is set to run for another batch of the pages per interval. So after 300s next 100 pages will be cached.

    When all the pages are cached, the preload will start from the beginning, so the preload never stops. It preloads the website over and over, making sure visitors are always getting the fresh cache, and w3_pgcache_cleanup is removing the expired cache.

    You can also use a custom sitemap and only add the pages you want to be preloaded, or remove the pages you do not wish to be cached.

    Alternatively, you can exclude pages from the cache in Performance>Page Cache>Advanced>”Never Cache following Pages” or add define(‘DONOTCACHEPAGE’, true); for specific templats you do not want to be cached.

    Thanks!

    Thread Starter futureyoon

    (@futureyoon)

    OK. Thank you for the info.

    What should I put in that section for all the child pages under members not to be preloaded?


    For example,

    members/aria

    members/arisha

    members/bruce

    can it accept the wild card like members/* ?

    Thank you

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @futureyoon

    Try adding /members/ to the ”Never Cache following Pages” as all sub pages will also be excluded from the cache.

    Thanks!

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘sitemap preload’ is closed to new replies.