No expire headers in S3

Files uploaded on June 4 2019 are fine. The metadata has Cache-Control and Expires values.

Beyond that, June 24 2019 is the earliest upload and from there, the metadata is different (see OP).

These dates seem to coincide with the release of version 0.9.7.5. Which states in its changelog:

Updated AWS library

Suspicious. I tested the site without CDN and, although the files have correct Expire headers (1 year), the Cache-Control seems wrong:

cache-control: max-age=0, no-store, public

The cache control option and policy for these files are indeed ticked in the Browser cache section. Although, I looked at my .htaccess and W3TC doesn’t seem to be applying the policy. The relevant section:

<FilesMatch "\.(css|htc|less|js|js2|js3|js4|CSS|HTC|LESS|JS|JS2|JS3|JS4)$">
    FileETag MTime Size
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public"
         Header unset Set-Cookie
    </IfModule>
</FilesMatch>

No max-age.

I’m left wondering:

Who sets these weird x-amz-meta tags in S3?
And, why is W3TC not setting up my .htaccess for Cache-Control (max-age)?

Okay, so the max-age in Cache-Control should be set by mod_expires.c, which is indeed set by W3TC:

<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType text/css A31536000
    [...]
</IfModule>

The .htaccess seems fine. But I don’t get why Apache serves:

cache-control: max-age=0, no-store, public

I tried explicitly defining max-age in .htaccess by changing:

<IfModule mod_headers.c>
     Header set Pragma "public"
     Header append Cache-Control "public"
     Header unset Set-Cookie
</IfModule>

to:

<IfModule mod_headers.c>
     Header set Pragma "public"
     Header set Cache-Control "max-age=31536000, public"
     Header unset Set-Cookie
</IfModule>

Which does alter the live headers (no CDN). However, if I delete some files from S3 and reupload them thru W3TC, then I get the same weird/redundant metadata cited in OP.

Typical response looks like this:

last-modified: Sat, 10 Aug 2019 19:25:26 GMT
status: 200
date: Sat, 10 Aug 2019 22:32:37 GMT
x-amz-cf-pop: SEA19
x-amz-version-id: null
content-encoding: gzip
x-amz-meta-content-encoding: gzip
x-amz-meta-link: <https://my.site/usr/local/www/apache24/data/wp-includes/js/jquery/jquery-migrate.min.js>; rel="canonical"
x-amz-cf-id: aPZF_VMB8M9Wg3Wh6bfg_SKkbgBljTmpIp4HYJ-rHerU2v4LijNaxQ==
age: 365
x-amz-meta-access-control-allow-origin: *
x-amz-meta-content-type: application/x-javascript
content-type: application/x-javascript
server: AmazonS3
accept-ranges: bytes
content-length: 4014
x-amz-meta-vary: Accept-Encoding
etag: "a6c81e2f02bd04160d2de88c4e8f3559" 
via: 1.1 aa42484f82c16d99015c599631def20c.cloudfront.net (CloudFront)
x-amz-meta-last-modified: Sat, 10 Aug 2019 19:25:03 GMT
x-cache: Hit from cloudfront

Still no expires or cache control.

Hello @jpb10,

I am sorry for the issue you are having and I am happy to assist you with this.
Please check the link below as you may find the solution there.
https://www.ads-software.com/support/topic/browser-caching-issue-with-cdn-content/

I hope this helps.

Thanks for your response, but that doesn’t solve the issue.

The metadata in S3 is set incorrectly since June 2019.

I don’t expect users will manually append cache control and expires values to every file uploaded from this time forward. And that’s not addressing the unnecessary x-amz-meta-* values.

The problem seems to be… in this commit:

https://github.com/W3EDGE/w3-total-cache/commit/49dfa53324d1884796a910d27bf1647abd8227bf#diff-7971eb11690e9cb546b92abb2f8e6d63

Hello @jpb10,
Thank you for the information. We are going to inspect this and provide you with an answer as soon as we find the solution.

While looking for a workaround, I noticed that “S3 Compatible” behaves like the old W3TC version in setting the correct metadata.

Assuming you have a CNAME associated with your CloudFront distribution, setting that domain in the S3 compatible CDN settings should(?) work like a hypothetical “CloudFront over S3 Compatible” option.

What is the official use case for S3 Compatible? I couldn’t find any documentation on it.

• This reply was modified 5 years, 6 months ago by jpb10.

Hello @jpb10,

As you have opened issues for both topics in our GitHub public repo, please make sure to check those and post comments there. We are working on this and someone will get back to you asap.

Hello @jpb10,

This issue has been resolved. Please check our Github Public repo.

https://github.com/W3EDGE/w3-total-cache/issues/27