400 Bad Request from Cloudfront

I’ve asked the Blubrry dev team to look into this. I’ll send another message to you as soon as I get something back from them.

Thanks for your prompt reply. I’ve investigate in deep and I want share my notes with you.

I’ve executed an HTTP HEAD request using curl from my linux server. We can see a correct 302 response with a new url indicated in the Location field:

curl --head https://content.jwplatform.com/videos/d1PigmFW- cBZLvBBN.aac
HTTP/1.1 302 Moved Temporarily
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Connection: keep-alive
access-control-allow-origin: *
Cache-Control: max-age=600
Date: Fri, 27 Sep 2024 07:18:17 GMT
Location: https://videos-cloudfront.jwpsrv.com/66f660e9_7a7f590170ecee6f1fa1aa3a 228ebf9a84682af8/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a
Server: openresty
x-robots-tag: noindex, indexifembedded
X-Cache: Miss from cloudfront
Via: 1.1 8f8b2e327677c7bd81e94944dc0a4a60.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: MXP53-P2
X-Amz-Cf-Id: Eyvpki4xm5NPDqSN8hMw2Rv8bP_aiKkYHC-ZR-SKrNtWA7r3JwGcyA==



A second HTTP HEAD call using curl to this new link works correctly:



curl --head https://videos-cloudfront.jwpsrv.com/66f660e9_7a7f590170ecee6f1fa1aa3a228ebf9a84682af8/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a
HTTP/1.1 200 OK
Content-Type: audio/mp4
Content-Length: 53709152
Connection: keep-alive
x-amz-replication-status: COMPLETED
Last-Modified: Thu, 12 Sep 2024 07:40:37 GMT
x-amz-tagging-count: 1
x-amz-server-side-encryption: AES256
x-amz-version-id: _0NIKDtAXR_u1VhLzPT8pAOv0AgzPDGz
Server: AmazonS3
Access-Control-Allow-Headers: accept-encoding, cache-control, origin, dnt
X-Backend: 2bbpke26e9piHyfIxklTTy--F_S3
Via: 1.1 varnish, 1.1 varnish, 1.1 3db152c3c5c7475d90014f6ad36522cc.cloudfront.net (CloudFront)
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
X-Served-By: cache-iad-kiad7000098-IAD, cache-fra-eddf8230152-FRA
X-Cache-Hits: 0, 1
X-Timer: S1727355346.880939,VS0,VE1
Date: Fri, 27 Sep 2024 07:08:53 GMT
ETag: "9df7062627c0466ef8181ba96eef3a5d-7"
Vary: Accept-Encoding
X-Cache: Hit from cloudfront
X-Amz-Cf-Pop: MXP63-P2
X-Amz-Cf-Id: 6qYzjSzI-yeTka5TBus_jpFgrxoHJo9ce5f2HIAfS5g0v88IsDYHRQ==
Age: 151303

If I repeat the first call to aac url, after some minutes, I can notice that the Location url changes, They are similar but the url is different:

curl –head https://content.jwplatform.com/videos/d1PigmFW-cBZLvBBN.aac
HTTP/1.1 302 Moved Temporarily
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Connection: keep-alive
access-control-allow-origin: *
Cache-Control: max-age=600
Date: Fri, 27 Sep 2024 07:37:29 GMT
Location: https://videos-cloudfront.jwpsrv.com/66f66569_1fef9650fb2b2dfe5c6153d80d9a74750f15a324/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a
Server: openresty
x-robots-tag: noindex, indexifembedded
X-Cache: Miss from cloudfront
Via: 1.1 e4ff8e5d95961f2029ed707a2c4209ea.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: MXP53-P2
X-Amz-Cf-Id: TnvuHzfl-GzPagKFflqMm3wu1n3kJHSwMwgqhdtJ0AxzVRz5L1jExA==

Has you can see the Location url has changed. This means that those have expiry attached to them, meaning they will only be valid for a given amount of time. They aren’t a permanent resource and by definition begin to throw errors at a certain point.

So, I suspect that the verification url process inside the Blubrry PowerPress using an expired url and get this 400 Bad Request error.
This is an example of the error:

curl --head https://videos-cloudfront.jwpsrv.com/66f558da_944ac5e673d7d70f28b689b8f81a43ce2f8b377b/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a

HTTP/1.1 400 Bad Request

Content-Type: audio/mp4

Content-Length: 430

Connection: keep-alive

Server: Varnish

Retry-After: 0

Accept-Ranges: bytes

Via: 1.1 varnish, 1.1 varnish, 1.1 892b64cb4f7d422e3a1221397ea1a546.cloudfront.net (CloudFront)

Access-Control-Allow-Headers: accept-encoding, cache-control, origin, dnt

Date: Thu, 26 Sep 2024 12:35:45 GMT

Access-Control-Allow-Origin: *

X-Served-By: cache-iad-kiad7000098-IAD, cache-lin1730021-LIN

X-Cache-Hits: 0, 0

X-Timer: S1727354146.578074,VS0,VE97

X-Cache: Error from cloudfront

X-Amz-Cf-Pop: MXP63-P2

X-Amz-Cf-Id: 1WViFbyYF9Pi-1joKQc5bb15etisgVE7L6QZCnj4bqpLJiuObQddog==

Can I ask you how Blubrry PowerPress calls the url? Is it use php curl?
Can be a cache issue inside php curl?

It seems that the verification step doesn’t read the updated Location.

Thanks in advance for your cooperation.

Here’s what the dev team had to say:

“unfortunately i think this issue is out of our control. the media URL redirecting to a different url every time it loads up will be hard to deal with. most likely their server is caching the response from the first request made, which would still have the Location from the initial request, not the updated one. I’m guessing they either need to figure out how to get their server to stop caching data from their CDN, or change the way they have things set up in Cloudfront”

Hi experts.
I did some more tests and I think we have figured out the problem but we need your help, please.
There is no DNS cache or other request cache issue.

Seem that AWS Cloudfront cache varys based on the http accept-encoding header.
In my curl example, by default, there aren’t any accept-encoding.

I repeted the test with –compressed flag and I start to get 200 OK also in the previous 400 Bad Request example, see these logs:

curl --head https://videos-cloudfront.jwpsrv.com/66fbf792_aec54e5b7e0576b18e4b319b6567b013b3c06946/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a

HTTP/1.1 400 Bad Request
Content-Type: audio/mp4
Content-Length: 431
Connection: keep-alive
Server: Varnish
Retry-After: 0
Accept-Ranges: bytes
Via: 1.1 varnish, 1.1 varnish, 1.1 9d4ff65dfbb2eb35f945c6fa4d05cf3a.cloudfront.net (CloudFront)
Access-Control-Allow-Headers: accept-encoding, cache-control, origin, dnt
Date: Tue, 01 Oct 2024 13:04:30 GMT
Access-Control-Allow-Origin: *
X-Served-By: cache-iad-kiad7000098-IAD, cache-mxp6928-MXP
X-Cache-Hits: 0, 0
X-Timer: S1727787870.185893,VS0,VE96
X-Cache: Error from cloudfront
X-Amz-Cf-Pop: MXP63-P2
X-Amz-Cf-Id: KxdZ3g4a4Q_bXmSyVJKE5Oy7dLB5LgVTLPNNn5_9OGIY8_a81NxcsQ==

NOW SAME URL WITH --compressed flag:

curl --head --compressed https://videos-cloudfront.jwpsrv.com/66fbf792_aec54e5b7e0576b18e4b319b6567b013b3c06946/content/conversions/lVlQc2wi/videos/bpXpoDWa-34656231.m4a

HTTP/1.1 200 OK
Content-Type: audio/mp4
Content-Length: 53709152
Connection: keep-alive
x-amz-replication-status: COMPLETED
Last-Modified: Thu, 12 Sep 2024 07:40:37 GMT
ETag: "9df7062627c0466ef8181ba96eef3a5d-7"
x-amz-tagging-count: 1
x-amz-server-side-encryption: AES256
x-amz-version-id: _0NIKDtAXR_u1VhLzPT8pAOv0AgzPDGz
Server: AmazonS3
Access-Control-Allow-Headers: accept-encoding, cache-control, origin, dnt
X-Backend: 2bbpke26e9piHyfIxklTTy--F_S3
Via: 1.1 varnish, 1.1 varnish, 1.1 66a9d30cb1014679858f80448b50159c.cloudfront.net (CloudFront)
Accept-Ranges: bytes
Date: Tue, 01 Oct 2024 12:59:41 GMT
Access-Control-Allow-Origin: *
X-Served-By: cache-iad-kiad7000098-IAD, cache-fra-etou8220067-FRA
X-Cache-Hits: 1, 0
X-Timer: S1727787581.222736,VS0,VE13
X-Cache: Hit from cloudfront
X-Amz-Cf-Pop: MXP63-P2
X-Amz-Cf-Id: cYb0y-TGDBLMvWle3vT0zbklsBkK9F3ocPYnirDMKEabgakRQCsz1Q==
Age: 1767

So, I want kindly ask you if the verification step in Blubrry PowerPress uses any Accept-Encoding header?
If not, could you implement it for aac extensions file? Or for Cloudfront CDN?
I really hope in a positive response.

Thanks in advance.

I’ve tested curl –compressed option and sometimes worked, somtetimes not.
The real problem seems that Cloudfront doesn’t accept HEAD method.

To fix the issue I’ve modified the file \plugins\powerpress\mp3info.class.php
and modified the code at line 360 adding a condition to use only GET method for Cloufront CDN.

original code:
$curl = curl_init();
			// First, get the content-length...
			curl_setopt($curl, CURLOPT_USERAGENT, $this->m_UserAgent );
			curl_setopt($curl, CURLOPT_URL, $url);
			curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
			if( defined('MEPR_PLUGIN_NAME') ) {
				curl_setopt($curl, CURLOPT_COOKIEFILE, ""); // For MemberPress
			}
			curl_setopt($curl, CURLOPT_HEADER, true); // header will be at output
			curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'HEAD'); // HTTP request 
			curl_setopt($curl, CURLOPT_NOBODY, true );
			curl_setopt($curl, CURLOPT_FAILONERROR, true);
			if( preg_match('/^https:\/\//', $url) !== false )
			{
				curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 2 );
				curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, true );
				if( defined('ABSPATH') && defined('WPINC') )
					curl_setopt($curl, CURLOPT_CAINFO, ABSPATH . WPINC . '/certificates/ca-bundle.crt');
			}

new code:
$curl = curl_init();
			// First, get the content-length...
			curl_setopt($curl, CURLOPT_USERAGENT, $this->m_UserAgent );
			curl_setopt($curl, CURLOPT_URL, $url);
			curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
			if( defined('MEPR_PLUGIN_NAME') ) {
				curl_setopt($curl, CURLOPT_COOKIEFILE, ""); // For MemberPress
			}
			
			//add custom code to handle Cloudfront JWP CDN that doesn't suppoer HEAD method
			if(strpos($url, 'jwp') !== false){
				//it's a jwp url, use ONLY GET
				curl_setopt($curl, CURLOPT_HEADER, false); // header will be at output
				curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'GET'); // HTTP request 
				curl_setopt($curl, CURLOPT_NOBODY, false );
			}
			else {
				//use HEAD method
				curl_setopt($curl, CURLOPT_HEADER, true); // header will be at output
				curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'HEAD'); // HTTP request 
				curl_setopt($curl, CURLOPT_NOBODY, true );
			}
			
			curl_setopt($curl, CURLOPT_FAILONERROR, true);
			if( preg_match('/^https:\/\//', $url) !== false )
			{
				curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 2 );
				curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, true );
				if( defined('ABSPATH') && defined('WPINC') )
					curl_setopt($curl, CURLOPT_CAINFO, ABSPATH . WPINC . '/certificates/ca-bundle.crt');
			}