Ideal settings for keeping all pages, always cached

Hello @jdingman

Thank you for your inquiry and I am happy to assist you with this.
If the cron job is set to every minute and the interval is set to a higher value, the plugin will check if the interval specified in the configuration has passed since the last time.
So that means if the cron job is not set to every minute, the priming could be delayed, especially if you set it to 1 hour and priming to 1,5 hour
Best is to have the cron job interval smaller than the priming interval and the priming interval should be equal to or a multiplied value of the cronjob interval.
With this, you can use recommended settings in the Cache Preload. It would be a good idea to disable define(‘DISABLE_WP_CRON’, true); and Once this is done, create a cron job that will call wp-cron.php at a set interval which you obviously already did.
As for the cache lifetime, if for example, you have 2000 pages that you want to be preloaded and you are using the default settings, 10 pages per 900s which means that all of the pages will be preloaded in 50 hrs. Now you need to take that into consideration if you are using Browser Cache>HTML&XML “Set expires header” as in W3 Total Cache The TTL of page cache files is set via the “Expires header lifetime” field in the “HTML” section on Browser Cache Settings tab.
Also, make sure to adjust the “Garbage collection interval”. The settings basically depend on how big your website is and how frequently you wish to clear the garbage.
Also note that after the preload is complete, it will start again from the beginning and that means more cached pages in the cache folder and more space required.
I hope this helps!

That is helpful, thank you!

What I’m still wondering though, is what the preload and garbage collection and lifetime cache, should be set to

if I have 2000 Posts…

500 pages per Prime? every 300 seconds?

Ideally I want to make the servers work as little as possible, and have the longest cache possible. Please advise what you think is ideal for the settings to reduce server load, and keep the cache primed

Hello @jdingman

500 pages per Prime every 300 seconds might cause huge server load. The best settings are the default settings.
It also depends on your server configuration.
The best thing, of course, is to make tests and see how it goes!
Thank you!

I’m finding that pages are not always cached when I visit them in a fresh browser.

So I’m trying to pin point why and how to fix it.

Should I be increasing the cache lifetime?

Hello @jdingman

Yes, you might want to do that, again depending on a TTL.
Also, logging is available in wp-cli.
So, instead of calling wp cron via crontab – you should call it directly via
wp w3-total-cache pgcache_prime
and track the output which pages/sitemap positions are primed by script.
Thank you!

Wondering how you end up setting this.

I’m still testing different settings, no silver bullet yet.

Circling back on this.

@vmarko we have a distributed system for cron jobs, where we use Lambda to manage WordPress cron jobs, instead of the host machine.

Is there a way to call pgcache_prime via GET instead of wp-cli only?

@vmarko any suggestions?

Hello @jdingman

You can either call wp w3-total-cache pgcache_prime or create a custom cronjob on your server and of course, as mentioned add define('DISABLE_WP_CRON', true); first.
Thak you!

@vmarko thanks. I have a cronjob running every minute already, how would I create a custom cronjob that hooks into pgcache_prime?

I have a cluster of servers, so I don’t want the same cronjob running on all of them to run wp cli