Cloudflare APO & WPSpuer Cache Setup

Looks like a lot of caches are cleared when deleting posts. I assume this is the homepage but it also seems to check other pages. Sometimes we run an import program that can delete a thousand posts a day so do not need to clear these caches so often or keep checking pages. Could this be my issue?

Hey @willjames, when you say – a lot of caches are cleared when deleting posts, are they cleared in Super Cache?

If so, what are your cache settings?

There’s a setting in the Advanced tab:
> Clear all cache files when a post or page is published or updated.

If that option is checked, it’s going to purge all cache files when a post is updated.

When a post is updated, Super Cache is also going to purge any related taxonomies like tags and categories to that post, so that may be related to what you’re seeing as well.

Super Cache is watching for changes on your site, so if you’re updating a lot of posts frequently, it may be expected behavior.

Can you provide a couple specific examples of situations where caches are cleared but they shouldn’t be?

The main issue seems to be when I run an import using WP All Import. I will often have to delete hundreds or thousand or so posts (custom post type, not normal WP posts). Each on takes a minute to delete – so a 1,000 minute job that means that the various caches are cleared every time. Looking at the debug below there is no need for each WP post to be checked when I delete a post in a CPT of ‘Listings’

I not it also tries to delete the author pages – although not for the author of the post (and my pages are rewritten to /dealer/ or /seller/ in any case.

Is there any way of preventing all these checks on posts so it would speed the deletions up a little?

Also should I even bother if I have Cloudflare apo installed?

Any other ideas are welcome.

Thanks

Here is an extract from the debug:

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wp_cache_post_edit: Clearing cache for post 3332950 on delete_post

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 supercache dir: /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/trucks-for-sale/for-auction-2003-daf-lf45-170-beavertail-plant-trucks-for-auction-leeds-26th27th-28th-29th-oct-2022-0800am/140315864_08-jpg-2/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wp_cache_post_id_gc post_id: 3332950 https://www.truckpages.co.uk/trucks-for-sale/for-auction-2003-daf-lf45-170-beavertail-plant-trucks-for-auction-leeds-26th27th-28th-29th-oct-2022-0800am/140315864_08-jpg-2/ clearing cache in /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/trucks-for-sale/for-auction-2003-daf-lf45-170-beavertail-plant-trucks-for-auction-leeds-26th27th-28th-29th-oct-2022-0800am/140315864_08-jpg-2/.

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_get_realpath: directory does not exist - /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/trucks-for-sale/for-auction-2003-daf-lf45-170-beavertail-plant-trucks-for-auction-leeds-26th27th-28th-29th-oct-2022-0800am/140315864_08-jpg-2/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 prune_super_cache: exiting as file/directory does not exist : /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/trucks-for-sale/for-auction-2003-daf-lf45-170-beavertail-plant-trucks-for-auction-leeds-26th27th-28th-29th-oct-2022-0800am/140315864_08-jpg-2/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wp_cache_post_id_gc clearing cache in /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/page/.

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 Post change: supercache enabled: deleting cache files in /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: deleting /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: .

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: ..

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: trucks

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: trucks-for-sale

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: for-sale

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: dealer

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: iveco-to-restart-production-in-italy-and-spain

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: free-food-drink-from-scania

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: iveco-reveals-new-s-way-racing-trucks

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: calor-buys-first-electric-fuso-ecanter

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: truckeast-wins-scanias-annual-dealer-gong

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: scania-is-first-truck-operation-to-gain-iso-45001

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: volvo-trucks-defers-3-months-finance-fees-on-new-truck-sales

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: volvo-renault-trucks-furloughed-volunteers-making-ppe

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: mercedes-truck-dealer-orwell-s-and-b

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: job-losses-announced-at-renault-trucks

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: daf-updates-emergency-braking-system-across-truck-range

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: volvo-used-trucks-new-warranties-over-1-million-kms

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: new-boss-for-volvo-trucks-uk-ireland

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: reading files: thats-the-way-to-do-it

…This goes on for over 400 posts for every deletion. Sometimes we delete 1500 posts in one import…

Then it carries on here:

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 Post change: deleting page_on_front and page_for_posts pages.

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 Post change: page_on_front 639563

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 Post change: page_for_posts 0

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wp_cache_post_change: checking /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/meta/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: deleting /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk//author/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_get_realpath: directory does not exist - /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk//author/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_files: directory does not exist: /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk//author/

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_get_realpath: directory does not exist - /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk//author//page

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 prune_super_cache: exiting as file/directory does not exist : /var/www/vhosts/truckpages.co.uk/httpdocs/wp-content/cache/supercache/www.truckpages.co.uk//author//page

14:36:39 118106 /wp-admin/admin.php?page=pmxi-admin-import&action=process&id=70&failures=0&_wpnonce=cee0cede89 wpsc_delete_post_archives: deleting cache of author archive: https://www.truckpages.co.uk/author/

Is there any way of bypassing these checks?

I’m not sure how well Cloudflare works in this situation, but that’s something that you can experiment with. You can disable Super Cache dynamically and see what the load on the server is in certain areas of your site: https://github.com/Automattic/wp-super-cache/wiki/FAQ#how-to-exclude-a-page-from-the-cache.

I think this is something that’s going to rest on your shoulders to optimize.

One other solution I can think of is to remove the WP Super Cache delete listener and manually purge the cache when necessary, but keep in mind that your deleted posts will still be visible while the cache is present, so be careful.

https://github.com/Automattic/wp-super-cache/blob/trunk/wp-cache-phase2.php#L1516

OK, it seems that the slowing down of the deletions is largely down to the Cloudflare APO plugin that creates the KV worker database for them. Not really sure what to do about this as I assume they need to delete the record from their caches too…

However, when I delete without WPSC on as well, the deletions happen one per second.

So wpsc does not need to check all the posts of a different post type on deletion – this would certainly help. Also the homepage would be better refreshed only once the batch is completed rather than for every record?

Thanks

Yeah, that sounds like something you’ll need to customize using filters.

Regarding making the homepage wait until the task is complete, if any of the items in the list of pages to delete is linked to from the homepage, and the overall task takes 10 minutes – then the homepage would serve a link to a page that ends up in 404 for those 10 minutes, which would be pretty poor user experience.