sitemap preload

Hello @futureyoon

Thank you for reaching out.

The Cache Preload in W3 Total Cache is a straightforward config, and once enabled and the sitemap URL is added to the Sitemap URL field, Cache Preload starts to work.

I’ve tested this and I am getting _index_slash_ssl.html for the preloaded pages with _index_slash_ssl.html_gzip (Do not have br on this env)

Can you please let me know for how long have you let the W3TC to preload the website and also, is the wp-cron enabled?
What are the settings for the Update interval and pages per interval?

THanks!

Oh now I see ‘_index_slash_ssl.html’ file after gzip and br created.

I have a few further questions

For testing purpose,

I set Update interval : 300 seconds & Page per interval : 100

Q1: Does this mean that this plugin will create folders and files under page_enhanced/mydomain/ (as part of preload process) for 100pages in every 300 seconds?

Q2: What if it fails to generate the 100pages in 300secs (too short)? what will happen?

Q3: After 300secs (after 1st run), the plugin will do preload for those pages not yet created from the 1st run (100pages from 1st run, 101th pages onwards from 2nd run)? or it will reset all from 0

Q4: It’s related to Q3. What will happen to those preloaded pages after 300 secs? it will become obsolete?

Q5: is there any cronjob involved with preload? if so, what is the name of the job?

I am trying to understand how it behaves so that I can possibly optimize the settigns for my case.

Thank you

• This reply was modified 2 years, 1 month ago by futureyoon.

Hello @futureyoon

Thank you for your feedback.
Lowering the Update interval and increasing pages per interval can impact the server performance. It is a lot to cache 100 pages at once every 5 minutes. This is why the default value is the best, it may preload the entire website slower, however, the impact on the server is minimal.

1. Yes that is correct however you should avoid this as it consumes to much serevr resources.
2. The remaining pages will be cached on the next run along with the old pages.
3. It will not reset to 0. It will continue preloading the remaining pages accourding to your sitemap URL.
4. Once the pages are preloaded, they are cached and remaining cached untill the next run. Once the pages are preloaded again the _old files are created which are then being temoved by the GC interval
5. Yes, W3TC does use wp-cron for this. w3_pgcache_prime for priming the pages every x minutes(seconds) and w3_pgcache_cleanup for deleting the _old files.

Thanks!

@vmarko Thank you so much for the details!

I understand your concern on the server. it’s just for the testing purpose.

Regarding Q4,

the next run is determined by ‘Update interval (which i set as 300secs)’ or where is it being determined (which setting)?

Q6: What will happen to the cronjob (w3_pgcache_prime) when all the pages are being preloaded? it will not auto stop, right? Similarly, how does system know that there are no more pages to be pre-loaded?

Q8: Is there a way to skip some pages not to be preloaded? I am running bb press based site with a few thousands members. It goes like members/aria, members/arisha, members/bruce…
Although my xml file doesnt include all these members, (only 1 entry for ‘members’), I see the system is trying to preload all those members, which will create lots of load on the server and members page is only for logged-in users so no point to doing the preloading. Can you guide me on this?

Thank you so much for your detail info with expertise.

Hello @futureyoon

Thank you for your feedback.

The Update interval is set to run for another batch of the pages per interval. So after 300s next 100 pages will be cached.

When all the pages are cached, the preload will start from the beginning, so the preload never stops. It preloads the website over and over, making sure visitors are always getting the fresh cache, and w3_pgcache_cleanup is removing the expired cache.

You can also use a custom sitemap and only add the pages you want to be preloaded, or remove the pages you do not wish to be cached.

Alternatively, you can exclude pages from the cache in Performance>Page Cache>Advanced>”Never Cache following Pages” or add define(‘DONOTCACHEPAGE’, true); for specific templats you do not want to be cached.

Thanks!

OK. Thank you for the info.

What should I put in that section for all the child pages under members not to be preloaded?

For example,

members/aria

members/arisha

members/bruce

can it accept the wild card like members/* ?

Thank you

Hello @futureyoon

Try adding /members/ to the ”Never Cache following Pages” as all sub pages will also be excluded from the cache.

Thanks!