Amazon S3 and Beijing Bucket not working

Hi,

I will ask my colleague to check on this.

Thanks,
Bryle

Hi,

Apologies for the inconvenience.

There is not currently a method to manually set the region for the Amazon S3 remote storage.

If possible, please could you run a test database backup to Amazon S3, without your code addition? This backup will fail (and can be stopped after the first attempt), but the log may show more information as to why the auto detection is not working.
The contents will be too long to post here directly, but you can use an online service such as Pastebin, and post the link here.

In the meantime, you can instead use the S3 generic remote storage. This allows an endpoint to be explicitly set (which in this case should be s3.cn-north-1.amazonaws.com.cn)

Hi @dnutbourne ,

Please see the following pastebin: https://pastebin.com/Wjzin7fv – the same error as when doing connection test occurs, without more information.

Adding an error_log( print_r( $storage, true ) ); on line 321 of methods/s3.php shows the endpoint attribute is set to s3.amazonaws.com and the region:UpdraftPlus_S3_Compat attribute is set to us-east-1, seemingly indicating the region and endpoint are not detected automatically from the credentials.

——

Using S3 generic remote storage fails with the following alert:

S3 settings test result: Failure: We successfully accessed the bucket, but the attempt to create a file in it failed. Please check your access credentials. (UpdraftPlus_S3::putObject(): [InvalidRequest] The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.)

I imagine this is because of protected $use_v4 = false; on line 12 of methods/s3generic.php.

Please advise.

• This reply was modified 4 years, 1 month ago by Alexandre Froger.
• This reply was modified 4 years, 1 month ago by t-p.

Have you enabled Beijing on your AWS account? Note that the Beijing region is different to others – use of the Beijing region is disabled on AWS accounts at Amazon’s end until you request for it to be enabled.

N.B. S3’s history means that there is a concept of a “default” region, when there’s no region specified; if the credentials supplied don’t work the first time, then US East 1 is then tried.

You can also try using the “S3 Generic” storage and entering the endpoint in there.

Hi @davidanderson! Thanks for the follow up.

Have you enabled Beijing on your AWS account?
Have you got any other tool in which access to this bucket/region has been tested and confirmed to work?

Not sure what you mean by “enable Beijing on your AWS account”, but it seems that the answer is yes to both questions – the same website is using a China S3 Beijing region bucket with Media Cloud plugin without issue.

You can also try using the “S3 Generic” storage and entering the endpoint in there.

I did and it doesn’t work. Please see the previous answer.

• This reply was modified 4 years, 1 month ago by Alexandre Froger.
• This reply was modified 4 years, 1 month ago by Alexandre Froger.

Hi,

Please could you change line 12 of methods/s3generic.php from:

protected $use_v4 = false;

to

protected $use_v4 = true;

Then, please attempt a connection test using the generic S3 method.
Does this allow the generic S3 to connect to your bucket?

• protected $use_v4 = true; on line 12 of methods/s3generic.php
• Key and Secret set to confirmed to work credentials
• Location set to existing bucket name updraft (Access point is present in Amazon S3 bucket, and access is set to “Objects can be public” on both the bucket level and the Access point level)
• Endpoint set to s3.cn-north-1.amazonaws.com.cn

Using S3 generic remote storage fails with the following alert:

S3 settings test result: Failure: We successfully accessed the bucket, but the attempt to create a file in it failed. Please check your access credentials. (UpdraftPlus_S3::putObject(): [AuthorizationHeaderMalformed] The authorization header is malformed; the region ‘us-east-1’ is wrong; expecting ‘cn-north-1’)

UpdraftPlus_BackupModule_s3generic class is a child class of UpdraftPlus_BackupModule_s3 and inherits of the same issue.

———

With the following code on line 1203 of of the file methods/s3.php to force the region:

$storage->setRegion( 'cn-north-1' );

Using S3 generic remote storage succeeds with the following alert:

S3 settings test result: Success: We accessed the bucket, and were able to create files within it. The communication with s3.cn-north-1.amazonaws.com.cn was encrypted.

———

Furthermore, I tested adding the following code on line 222 of of the file methods/s3.php to force the region during actual backup:

$storage->setRegion( 'cn-north-1' );

In this case, during backup, the archive is successfully created in the bucket. This seems to indicate it is an issue when initializing the SDK.

Digging further, I noticed the class UpdraftPlus_S3_Compat is used when calling public function getS3(). In its constructor, the region is only set to default us-east-1, value of the $region property:

	public function __construct($access_key = null, $secret_key = null, $use_ssl = true, $ssl_ca_cert = true, $endpoint = null, $session_token = null, $region = null) {// phpcs:ignore Generic.CodeAnalysis.UnusedFunctionParameter.Found
		if (null !== $access_key && null !== $secret_key)
			$this->setAuth($access_key, $secret_key, $session_token);

		$this->use_ssl = $use_ssl;
		$this->ssl_ca_cert = $ssl_ca_cert;

		$opts = array(
			'key' => $access_key,
			'secret' => $secret_key,
			'scheme' => ($use_ssl) ? 'https' : 'http',
			// Using signature v4 requires a region (but see the note below)
			// 'signature' => 'v4',
			// 'region' => $this->region
			// 'endpoint' => 'somethingorother.s3.amazonaws.com'
		);

		if ($endpoint) {
			// Can't specify signature v4, as that requires stating the region - which we don't necessarily yet know.
			// Later comment: however, it looks to me like in current UD (Sep 2017), $endpoint is never used for Amazon S3/Vault, and there may be cases (e.g. DigitalOcean Spaces) where we might prefer v4 (DO support v2 too, currently) without knowing a region.
			$this->endpoint = $endpoint;
			$opts['endpoint'] = $endpoint;
		} else {
			// Using signature v4 requires a region. Also, some regions (EU Central 1, China) require signature v4 - and all support it, so we may as well use it if we can.
			$opts['signature'] = 'v4';
			$opts['region'] = $this->region;
		}

		if ($session_token) {
		  $opts['token'] = $session_token;
		}
	
		if ($use_ssl) $opts['ssl.certificate_authority'] = $ssl_ca_cert;

		$this->client = Aws\S3\S3Client::factory($opts);
	}

The setRegion method doesn’t seem to be called anywhere else after the UpdraftPlus_S3_Compat object is instantiated, resulting in the failure described in previous messages.

Please advise.

[ Don’t bump. ]

• This reply was modified 4 years, 1 month ago by Alexandre Froger.