MySQL Bugs: #73616: mysqlbackup should handle 307 redirects for s3

Bug #73616	mysqlbackup should handle 307 redirects for s3
Submitted:	17 Aug 2014 18:33	Modified:	9 Sep 2014 16:48
Reporter:	Daniël van Eeden (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Enterprise Backup	Severity:	S3 (Non-critical)
Version:	3.10.2	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	aws, Backup, cloud, s3

Description:
Amazon uses redirects for various reasons. Currently mysqlbackup doesn't handle this well.

http://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html

How to repeat:
 mysqlbackup: INFO: Checking the object key.
 mysqlbackup: INFO: Initializing upload.
 mysqlbackup: ERROR: Received error 307. Retrieving error info.
 mysqlbackup: INFO: 
 Response headers. 
 HTTP/1.1 307 Temporary Redirect
 x-amz-request-id: xxxxxxxxxxxxxxxxxx
 x-amz-id-2: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 Location: http://xxxxxxxxxxxx.s3-eu-west-1.amazonaws.com/xxxxx?uploads
 Content-Type: application/xml
 Transfer-Encoding: chunked
 Date: Sun, 17 Aug 2014 18:31:54 GMT
 Connection: close
 Server: AmazonS3
 Response buffer size 439 bytes.
 
<?xml version="1.0" encoding="UTF-8"?>

<Error>
<Code>
TemporaryRedirect
</Code>
<Message>
Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.
</Message>
<RequestId>
xxxxxxxxxxxxxxxxxxxxxxx
</RequestId>
<Bucket>
xxxxxxxxxxxx
</Bucket>
<HostId>
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
</HostId>
<Endpoint>
xxxxxxxxx.s3-eu-west-1.amazonaws.com
</Endpoint>
</Error>
 mysqlbackup: ERROR: Upload initialization failed.
 mysqlbackup: INFO: Cloud operation complete.

mysqlbackup failed with errors!

Thank you for the report.

Is there any way to force AWS to do redirect or I simply should make enough tries until I hit the redirect?

This is what I understand from the linked aws page:

- If you create a new bucket in a non-US region (e.g. an europe region) and then imidiately try to use it you will get a redirect. This is because DNS hasn't yet propagated.
- If you submit a request to the wrong endpoint then you will get a redirect.

This is how I encountered it:
- Created a new EU bucket
- Tried to use it imidiately with mysqlbackup
- Result: unhandled redirect

This is how I think this could be tested:
- Use /etc/hosts to redirect requests to the wrong endpoint
- This should result in redirects.

Thank you for the feedback.

Verified as described. To repeat choose "Ireland" as region, then try to backup to it. Workaround: choose some US region.