Support

Akeeba Backup for Joomla!

#9038 Getting error when transferring to Amazon s3

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Friday, 09 September 2011 02:44 CDT

user35166
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes
Have I searched the forum before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: (1.5.23)
PHP version: (5.3)
MySQL version: (unknown)
Host: (optional, but it helps us help you)
Akeeba Backup version: (3.3.3)

EXTREMELY IMPORTANT: Please attach your Akeeba Backup log file in order for us to help you with any backup or restoration issue.

Description of my issue:

When performing backup, I am getting following error on post-installation step. While sending to Amazon S3

AEUtilAmazons3::uploadMultipart(): [RequestTimeout] Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
Failed to process file /home/chaseweb/public_html/administrator/components/com_akeeba/backup/site-chasewebsolutions.com-20110908-101654.jpa
Post-processing interrupted -- no more files will be transferred

My Log is too big to attach (13mb)

nicholas
Akeeba Staff
Manager
First, let's try making sure that the SSL setting doesn't cause the problem. First go to Akeeba Backup's Configuration page and find the "Data Processing Engine" drop down. Click the Configure button next to it. A new pane opens below. Find the "Use SSL" checkbox and make sure it is not checked.

The second thing you have to check is that your host's firewall allows access to Amazon S3. Ask them if they have a firewall which blocks outgoing connections. In this case, please tell them to allow TCP/IP connections to ports 80 and 443 of s3.amazonaws.com. If they request an IP, please tell them that this domain name is a multicast one and they have to run "host s3.amazonaws.com" from their server to obtain the IP. It doesn't matter if this sounds like Chinese to you, your host's support technicians will understand (or should, at the very least).

Finally, some hosts do not play very well with Amazon S3's multi-part upload feature which allows us to upload a very big archive file in 5Mb chunks. In this case you will have to follow Plan B which is to have Akeeba Backup split the archive file in small chunks, one file per chunk, and then upload each of those chunks in one go. This is a two-legged solution.

For the first leg of the solution, please go to Akeeba Backup's Configuration page and find the "Data Processing Engine" drop down. Click the Configure button next to it. A new pane opens below. Check the "Disable multipart uploads" option and make sure that the "Process each part immediately" option is not checked.

Now, for the second leg, we have to do some trial and error. Still in Akeeba Backup's Configuration page, find the Archiver Engine drop-down and click on the Configure button next to it. A new pane opens below. Find the "Part size for split archives" option and select the 49.99 option. Try a new backup. If it crashes while uploading files to Amazon S3, go back to this option and try smaller values, i.e. 20, 10, 5, 2 or even 1, trying a new backup after setting each one of those values.

Please let me know how it went!

Side note: You said that your log file is 15Mb long. It sounds like you have a gazillion files in there! If there are files or directories which need not be backed up (e.g. additional sites on subdomains which should be backed up on their own right) please use the Files and Directories Exclusion feature of Akeeba Backup to exclude them from the backup.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user35166
Hi,

Thanks for the reply - what I have done so far

1. SSL is not enabled
2. I zipped my backup file for review (I only saw 3 WARNINGS where it failed to upload Part 9 of the Multipart upload

Some other notes before I switchup approach of backup. This was working before and started failing only recently. I updated to the latest version, so I don't believe there is an issue with my host provider.

I will now try the other approach and let you know.

Corey

user35166
Hi

So yes when I did the second approach set at 20.00mb that worked, but now I have 6 files 1 (jpa) and j0X 1-6 files on the S3.

So this means that the one file approach that was working is no longer working for me?

Corey

nicholas
Akeeba Staff
Manager
Yes, that's correct. The problem seems to be with some buggy versions of curl. Some versions of curl work just fine, some routinely crash when dealing with large files. Most likely your host did a minor server upgrade which replaced curl and caused this issue to manifest itself. Apart from the multi-file approach, I can see no other alternative :(

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user35166
Ok that's not an issue, I am more concerned about reliability since I have this going on cron.

I only noticed it because the size of my site was very huge.

Thanks alot for you help.

Corey

nicholas
Akeeba Staff
Manager
If you're using backup.php to run your CRON jobs, it will automatically print out any warnings. If there are warnings, it will print out a very prominent block which notifies you that the backup completed with warnings. The CRON daemon takes the output and emails it to you (if you haven't set up the email notifications on your host, do it now). So, the reliability issue is now a matter of simply creating email inbox rules to archive the email notifications of successful backups and mark as important the emails of backups which completed with warnings. Easy-peasy :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!