Support

Akeeba Backup for Joomla!

#21399 Amazon S3 new authentication scheme

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Saturday, 15 November 2014 05:20 CST

user61484
Description of my issue: I'm trying to upload backups in new Amazon S3 region (Frankfurt), but it seems it uses a newer authentication scheme that's not supported by Akeeba Backup.

AEUtilAmazons3::putObject(): [InvalidRequest] The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

nicholas
Akeeba Staff
Manager
This is correct. Please take a look at http://stackoverflow.com/questions/26533245/the-authorization-mechanism-you-have-provided-is-not-supported-please-use-aws4

Unfortunately we can't use the V4 authentication API because it requires the entire content body to be signed. This would require the entire backup archive to be loaded to memory and processed. With part sizes over 512Kb (not a typo: five hundred and twelve KILOBYTES) this will always cause a memory or timeout issue.

At this time we don't have any solution. The solution is not up to us. It is a limit of what Amazon wants and what your servers can do. These two areas don't intersect. We can't do anything about it, sorry.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user61484
Mkay. Thanks for reply and sharing the link. As Frankfurt people we would love to use Frankfurt as region. So we take Ireland for our backups now.

nicholas
Akeeba Staff
Manager
The past several days we have been refactoring the backup engine. While we were at this, I decided to try integrating the new Amazon SDK for PHP in it, which supports the AWS4 API with multipart uploads. Well, the Amazon SDK was buggy as far as multipart uploads are concerned, I patched it and now it does work. The next version of Akeeba Backup -which is currently in final testing- will have a new Amazon S3 post-processing engine which supports the new authentication API used in the Frankfurt and other newer S3 Regions.

The important thing to keep in mind is that using the AWS4 authentication scheme requires MUCH more memory than the old scheme. As a result you MUST have a PHP memory limit of at least 64Mb and you MUST either enable multipart uploads (which performs the upload in 5Mb pieces) or use a part size for split archives less than 5Mb. Just to give you an idea of peak memory consumption by the backup engine itself:

- Using the old AWS2 authentication scheme, single part 10Mb upload - 7.6Mb
- Using the new AWS4 authentication scheme, multipart 10Mb upload - 22Mb

The 12+ Mb difference using AWS4 can be attributed to the need to load each 5Mb upload chunk in memory twice, once to get an MD5 sum and once to upload to Amazon S3.

For what is worth, the new post-processing engine still supports using the old AWS2 authentication method for buckets created in old regions, such as Ireland, in case you have limited PHP memory available and don't wish to use a small part size for split archives.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!