W3 Total Cache, AWS Cloudfront & Invalidations

Hello @per4mance

I am sorry about the issue you are experiencing and I am happy to assist you with this.
Is “Only purge CDN manually” enabled in Performance>CDN? Also is the option “Export changed files automatically” enabled?
Thank you!

Hi @vmarko

Thank you for your reply.

I activated “Only purge CDN manually” under Performance > CDN now. Thanks.
Unfortunately I cannot find “Export changed files automatically”. Where is this located?

Regards ??

Hello @per4mance

W3 Total Cache can purge the CDN cache by sending invalidation requests or just telling it there are new files to be cached. The CDN will understand what to do with it. The “Only purge CDN manually” was built because of the problems with invalidation requests on CloudFront causing huge bills. If it’s enabled, you can click the purge CDN button in W3 Total Cache to send invalidation requests, otherwise, all changes will be detected automatically and an invalidation request will be sent immediately.
The option “Export changed files automatically” should be just below the “Only purge CDN manually” but only in case, you are using Origin Push.
If you are using Origin Pull that option is not there.
This being said now that you have enabled “Only purge CDN manually” you can manually purge the cache when you want to send the invalidations.
THank you.

@vmarko

Thank you for your detailed explanation. It seems I’m using “Origin Pull” only because I do not have any hint of “Export changed files automatically”.

When activated “Only purge CDN manually” and using W3 button in backend “Purge All Caches”, does this say I have to pay for Invalidations that time?

Amazon AWS has an interesting way to gain money.

Regards

Hello @per4mance

Here is the article on AWS which explains the invalidation feature.
I hope this helps you understanding how invalidations work.

Hi @vmarko

Thank you. That says it deletes one or more files in case you want to remove it earlier than the time stamp says, right?

If so, who sets the time stamp for a deletion? Does it come from the W3 entries?

But when I upload an image for a post I want to publish, why does your plugin send anything to AWS, what leads to an “Invalidation”? That is what I do not understand. There was no action from my side to do this.

Regards

Hello @per4mance

The side effect we want to prevent here is that the CDN may cache 404 response.
lets say
1. someone opens https://website.com/myimage.jpg – response is 404 and it’s cached by CDN
2. new image uploaded
3. we want to tell CDN to refresh its cache and don’t respond 404 anymore
This is the reason why W3 Total Cache is sending those invalidations.
Thank you!

Hello @vmarko

Thank you for answering but this makes no sense if I upload an image for a post the first time. So the case you describe won’t take effect. This makes sense if I want to delete an image or when updating an image. So it is the second step before the first.

For my opinion there should be a hint or an entry for this if a user wants to handle this that way. It seems tricky for clueless users to come into a situation like this when using Amazon AWS.

Please think about, if it is a good idea to treat customers/users this way. Thank you for your understanding.

Regards.

We don’t understand how in only one month we have received by surprise a bill of $1,053.10 in concept of “Amazon CloudFront Invalidations”… it’s due to the module W3 Total Cache???? Can someone explain to us how without knowing it we just lost 1,000 dollars? Amazon Cloudfront Invalidations – February bill

@instintnet

What I did to prevent AWS Cloudfront Invalidations is to check under “Performance > CDN > section CDN + CDN Enable”

I made this screenshot for you https://www.screencast.com/t/cYohS2awhS5k

I hope this helps!

Hello @instintnet @per4mance
In Performance>CDN there is an option “Only Purge CDN manually”

Cloudflare doesn’t charge for “request”, but for invalidation path.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#PayingForInvalidation
The first 1,000 invalidation paths that you submit per month are free; you pay for each invalidation path over 1,000 in a month. An invalidation path can be for a single file (such as /images/logo.jpg) or for multiple files (such as /images/*). A path that includes the * wildcard counts as one path even if it causes CloudFront to invalidate thousands of files.
so you can:
invalidate all your images per-case basis,
i.e.
https//my-host/wp-content/uploads/2019/01/my-jan-image1.jpg
https//my-host/wp-content/uploads/2019/01/my-jan-image2.jpg
https//my-host/wp-content/uploads/2019/02/my-feb-image1.jpg
that will be charged as 3 paths and that’s what w3tc does.
2. invalidate everything.
that’s what w3tc’s purge all button does. all objects will be removed from cache so your traffic will grow and performance temporarily decline. also, of course, you have to do it once after multiple posts updated, otherwise, if you purge after each change you’ll get the worse situation in the end (3 invalidate-all requests for 3 posts).
3. invalidate grouped based on knowing what you’ve modified. in the example below, you may invalidate one of:
3.1 https//my-host/wp-content/uploads/2019/* – one path, will invalidate all 2019 images only
3.2 https//my-host/wp-content/uploads/2019/01/*, https//my-host/wp-content/uploads/2019/02/* – 2 paths, but will invalidate only all 2019 jan and feb images.
I hope this helps!

Hi, @per4mance

First of all, thank you very much for your help. We are still digesting the news of the payment we must pay now.

On the other hand, we have proceeded to deactivate everything, both the Cloudfront distribution and the CDN functionality of the plugin until we understand if we can use it without having surprises of this type.

And referring to what you say, we had already activated what you show us in the screenshot …: ((

@instintnet

This is the right checkbox, sorry for the wrong part before.

https://www.screencast.com/t/IEayyt8NOl

ok, @per4mance and @vmarko. For another domain that we also have with Cloudfront we have already checked the “Only purge CDN manually” box.

And @vmarko, it is essential to indicate the invalidations that you have commented? Or having the box checked is enough? And if we have to indicate these rules, do we have to do it in the CDN > “Custom file list” section of the plugin? Or in the Cloudfront configuration?

I’m sorry, but we’re still lost and we don’t want to waste your time either. We are already investigating with the information you have provided.

Thank you very much to both of you

@instintnet

This is the only checkbox you have to check. It works for me for two months.