Support

Hi Paul,

I can see that in this case you are using the multi-part upload and that's causing it to fail. The thing is, the multi-part upload doesn't seem to be the stablest thing in the world, as it tends to halt pretty frequently on Amazon's end. Let's try the following:
1. Upgrade to Akeeba Backup 3.3.6
2. Try disabling the multi-part upload for Profile #2 (warning: do not edit Profile #1, it won't work ;) ). Double check that you disabled multi-part uploads for profile #2!
3. If this doesn't work, try lowering the part size to something like 500Mb, 250Mb or 100Mb.

If all else fails, please ZIP and attach you new log file.

I think you are editing the wrong profile. Your backup which doesn't work is using profile #2. On that profile, multi-part upload is enabled (the checkbox is NOT checked).

Go to your site's back-end, Components, Akeeba Backup. In the drop-down above the buttons, select your second profile. The page reloads. It should now say that the active profile is such and such and next to its name #2 should be printed. Now click on the Configuration button and take a look at the "Disable multi-part upload" checkbox. Is it checked? If not, check it and click on Save, then retry the backup.

Paul,

You are either editing the configuration of the wrong backup profile or you are overriding the configuration in your CRON command line. Can you please send me a PM with Super Administrator access to your site so that I can take a look and see if I can rule out the first suspicion?

Let's see what the log has to say:

DEBUG |111114 04:12:43|S3 -- Legacy (single part) upload of site-XXXXXX-20111114-040001.j01

WARNING |111114 04:20:17|Failed to process file /administrator/components/com_akeeba/backup/site-XXXXXX.j01

WARNING |111114 04:20:17|Error received from the post-processing engine:

WARNING |111114 04:20:17|AEUtilAmazons3::putObject(): [56] Failure when receiving data from the peer

I honestly believe that the part size you are currently using is right at the verge of causing an Amazon S3 timeout. As I wrote 4 days ago regarding the part size:

If this doesn't work, try lowering the part size to something like 500Mb, 250Mb or 100Mb

Please do that.

I'm looking at the timestamps in your log:

DEBUG |111117 04:04:28|S3 -- Legacy (single part) upload of site-qms.completepicture.co.uk-20111117-040002.jpa

WARNING |111117 04:09:00|Failed to process file /administrator/components/com_akeeba/backup/site-qms.completepicture.co.uk-20111117-040002.jpa

It looks like that the connection speed to S3 has dropped since the last time you tried. The last time it was about 150 Mb/minute, now it certainly under 100. Just try going a little backwards. Start with a part size of 20 Mb and then progress to 50, 100, 200, 300 Mb until you find which one works reliably.

This is a coincidence. Your logs tell me that all previous backups failed when they were trying to upload the backup archives to S3. This code runs BEFORE the quota code. The quota code (which only does a few non-data-intensive DELETEs) was succeeding. If the two problems were somehow linked, then it should be the quotas failing (as they come AFTER the upload), not the uploads. The effect can never precede the cause ;)

Hello Paul,

The Administer Backup Files page doesn't know if all or just a few of the files have been uploaded. If no error was reported during the upload, it assumes all parts were uploaded. The workaround woud be to query Amazon S3 every time you loaded the Administer Backup Files page, wasting your money and risking timeout errors.

Regarding the uploads, I still can't see how enabling the remote file quotas has anything to do with the upload. After the backup is complete, the Finalize domain runs and the following actions are executed, in this exact order:
- Remove the temporary files which are stored in the output directory
- The stats table is updated with the execution time
- The total filesize of the backup is update in the stats table
- The post-processing (upload to cloud storage) is performed
- The local quotas are applied. Note: this only removes files on YOUR server
- The remote quotas are applied. Note: this only removes files on the REMOTE server, i.e. Amazon S3
- An email is set to notify about the backup completion, if this feature is enabled

The order of these tasks is defined in administrator/components/com_akeeba/akeeba/core/domain/finalization.php lines 49 to 57. As you can see on the same file, lines 270-417, the post-processing (upload to S3) doesn't care about quotas. The first time the quotas are being evaluated is lines 495-699 (the apply_quotas() method) which runs AFTER the upload of files.

Therefore any quota settings CAN NOT interfere with the upload, because these settings ARE NOT read AT ALL until AFTER the upload code has run. It is IMPOSSIBLE that a single integer, stored in the database and not used ANYWHERE in a block of code can have any effect at all in that block of code. Computers are the dumbest things in the Universe. They will only do what you tell them to do. They don't have a mind of their own. We are not telling your server to use the quota value until AFTER the download, therefore it can't just think of its own and decide to use it nonetheless! That would be absurd. As a result, the quota setting and the upload to S3 can not be linked.

If you don't believe me, please do have another developer take a look at the code. I just pointed out the exact file and line numbers where the code runs.

Yes, please do! This thing is very strange. I know it can't be the quota setting, but I can't make heads or tails of your results. Better leave it running for a week and let's see what's going on.

You have a completely different problem. Please start a new thread. If we start discussing two unrelated issues on the same thread, I can promise you that I'll get confused and reply the wrong thing to each one of you and we'll all be very unhappy.