Akeeba Backup for Joomla!

Wednesday, 26 November 2014 15:37 CST

refreshpc

Hi Support.

The product works great, however I have an issue with the dropbox post processing (backup completes, all uploads ok etc).

The issue is that for some reason, after the backup is completed the connection to dropbox/amazon isnt terminated.

My server went through 240GB in one day of outgoing traffic to amazon. No backups were running at this point in time, at all.

223.252.38.225 50.19.88.223 11353 17024310 44079 443 6 eth1
223.252.38.225 50.19.88.223 14391 21568154 44089 443 6 eth1
223.252.38.225 23.21.152.35 14403 21587600 44261 443 6 eth1
223.252.38.225 23.21.152.35 14404 21589102 44257 443 6 eth1
223.252.38.225 23.21.152.35 14409 21596602 44259 443 6 eth1
223.252.38.225 23.21.152.35 14413 21602600 44263 443 6 eth1
223.252.38.225 23.21.152.35 14416 21607102 44255 443 6 eth1
223.252.38.225 23.21.152.35 14422 21616100 44245 443 6 eth1
223.252.38.225 50.19.88.223 14428 21621229 44091 443 6 eth1
223.252.38.225 23.21.152.35 14429 21626600 44235 443 6 eth1
223.252.38.225 50.19.88.223 14436 21637100 44083 443 6 eth1
223.252.38.225 23.21.152.35 14438 21640100 44241 443 6 eth1

Can you please shed some light onto why this connection isnt terminated?

Regards,
Darren.

Wednesday, 26 November 2014 16:12 CST

dlb

There is a new feature in 4.0.5 that will retry a failed backup. The default settings will retry three times before it gives up. So the default settings would not do what you're describing. If you changed the settings, it is possible that it could retry many times or even an infinite number of times.

The retry is used when the upload times out, for example. It is not something that you would want to retry forever.

You should see a "Pending" backup under Manage Backups if you have a backup in progress.

Dale L. Brackin
Support Specialist

English: native

Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Wednesday, 26 November 2014 16:24 CST

refreshpc

There is no pending backups and the retry options are set to default, no more then 3 retries, with 10 minute breaks.

No site failed to transfer to dropbox to trigger the retry. This has been confirmed by manual runs via the admin backend and also via cron.

Any other thoughts?

Thursday, 27 November 2014 01:47 CST

nicholas

It is TECHNICALLY INFEASIBLE that PHP will send data to Dropbox or Amazon S3 forever for the following simple reasons:
* Time limit. PHP is bound by a time limit. If you exceed it, the process is killed (and with it all data transfer)
* Server time limit. Apache is also bound by a time limit. Same rules apply.
* Connection time limits. Dropbox/Amazon will close the connection if it takes too long.
* Upload limits. Dropbox has a limit of 150Mb per upload attempt. Amazon S3 has a limit of 5Gb. One more byte and your connection is terminated. That's 234.75 and 235Gb short of your alleged 240Gb uploads.
* Any data uploaded would have appeared in your Dropbox, even if you did something as pointless as running a new backup every single minute due to a badly written CRON job. Unless, of course, you are using the same filename for all backups.

Most likely you're doing something wrong with your CRON jobs or, most likely, you are misdiagnosing the issue. It's simple to find out. Please uninstall our software completely. If you still get uploads to Amazon/Dropbox/whatever you are using because it's not clear in your message then you know you've misdiagnosed this issue.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 27 November 2014 02:08 CST

refreshpc

Certainly not alleged in the amount of data transfered.
Cron jobs were configured to run once a day, no more.
---
Upload limits. Dropbox has a limit of 150Mb per upload attempt. Amazon S3 has a limit of 5Gb. One more byte and your connection is terminated. That's 234.75 and 235Gb short of your alleged 240Gb uploads.
---

So, this may be something however, and where it gets interesting.
Above you say a limit of 150MB per upload attempted.

The full jpa is 350MB, uploaded successfully to dropbox, no failures. Subsequent runs were incremental, one via cron.

I have 2 more sites above the 150MB limit that transfer successfully.

If you wanted to look into this any further Im happy to assist, Maybe take it out of public though.

Thursday, 27 November 2014 05:22 CST

nicholas

Files larger than 150Mb are transferred in 20Mb increments. After all of these increments are uploaded we need to issue a command for Dropbox to glue them together. If we upload more or less increments than they are exactly required to build the file, the file will not appear in your Dropbox. If we do not send the command to glue them together, the file will not appear in your Dropbox. If we send the wrong list of increments to Dropbox, the file will not appear in your Dropbox.

Also, your numbers don't add up. The transfer of the 350.96 Mb to Dropbox takes about 21 minutes (see the difference between backups 3 and 5). This means 1Gb per hour or 24Gb per day even if we accept that for some reason which defies what is technically possible there was a Holywood-inspired continuous upload of phantom data to Dropbox. That's 90% less than what you claim was transferred. Even if we take for granted that the 6.97Mb uploads in 1 minute, that's just 9.8Gb per day, so we're still missing 230-odd Gb.

All the evidence points to you having misdiagnosed your problem.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support