Support

Akeeba Backup for Joomla!

#13292 Failing when transfering to S3 and/or Transfering but saying

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Saturday, 18 August 2012 11:12 CDT

user52353
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes
Have I searched the tickets before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: Joomla! 2.5.6 Stable [ Ember ] 19-June-2012 14:00 GMT
PHP version: 5.3.13
MySQL version: 5.1.63-cll
Host: Rackspace Cloud
Akeeba Backup version: Akeeba Backup Professional 3.6.1 (2012-08-01)

EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue:

If you look at my other tickets you will know that this has been going on for some time. For a very long time my backup admin screen would show that the backups failed even though I would then find them in Amazon S3. In my last ticket you told me to change the mysql timeout time on the server. I got busy with other things and didn't worry too much about it since the backups were transferring but just not showing properly in Akeeba Admin. Then I upgraded to your latest version the other day and afterward the failed backups really were failing. The backups were completely failing and no longer making it to S3. So once again I went through your whole troubleshooting checklist. I made sure that the maximum execution time shown in the Akeeba configuration screen was less than the server setting. The result was that Akeeba would still say failed backup on the initial screen after running, but then on the main Akeeba screen it would say OK for the last backup but when I went into Akeeba Admin it would say failed, but it would show up in S3. So then I started trying to correct the server mysql timeout to try to address the failures. Nothing works.

The backup process always seems to go fine, then when it begins post processing it sits for a long time and shows no server response for seconds climbing until it gets to about 595 seconds then goes to the ajax and timeout error. If I try to go anywhere from that screen Joomla Admin just freezes up. I have to close my browser and go back in, and even that doesn't always work. The Admin screen then either shows OK but the backup is local and not transferred or Obsolete and it is transferred.

Why does it take so long to transfer to S3? Is there anything that can be changed in the Akeeba settings to make it transfer to S3 more smoothly? At this point I have both mssql.connect_timeout and mssql.timeout set to no limit on the server and it not only has not helped but has made the backup process slower. And I am not comfortable leaving it with no limit. max_execution_time is now 60 on the server (it was 30) and your configuration wizard set it to 30 in Akeeba configuration.

Please help. I want to be able to run backups without freezing Joomla Administrator, get the backup to go to S3, and be able to administer it from Akeeba Backup Admin as it's intended. I don't want to leave server settings unlimited in cases that might be risky.

Attached are my PHP Configuration Editor settings from the server (before the latest increases mentioned above), and the Akeeba Configuration settings. Please tell me what changes are required to get this working. The last log just says aborted at the request of the user. I did not request that it abort.

Thank you.

user52353

user52353

user52353
I tried to backup again. It got through the backup processes fine, then started the post processing and just counted the seconds from the last server response up to almost 600 seconds, then ajax error and failed message. Could not get out of the screen, closed the browser, re-opened, the Akeeba main page said the last backup was ok. Went into the Administer Backup Files screen and it says the backup is "OK" but it is local and not transferred to A3.

Attached is the log.

user52353
I just checked again, literally minutes later, and now it says "failed" instead of "ok" and the file name is no longer shown. This is truely maddening. I have spend days on this over the last months and get nowhere.

nicholas
Akeeba Staff
Manager
You don't have to write War & Peace, just a short description of the problem and the log file suffice. That said, you are confusing unrelated things in your ticket and ultimately ended up in reading only the parts of the documentation and troubleshooting information which do not solve your problem. I'll try answering to all of them and lead you to the correct workaround.

For a very long time my backup admin screen would show that the backups failed even though I would then find them in Amazon S3.

That was because you filed a ticket back in April IIRC and I told you it was a bug to be fixed in the next release. You didn't install the next release for a very long time. Between the time you reported the issue and the publication of the fix it was just one week.

The result was that Akeeba would still say failed backup on the initial screen after running, but then on the main Akeeba screen it would say OK for the last backup but when I went into Akeeba Admin it would say failed, but it would show up in S3.

I have an idea what's going on. When a backup is running you should never, ever, visit another page besides the Backup Now page. If this is not the case, then the switch from OK to Failed happens after three minutes. What's going on is that as soon as the backup itself is complete (before the transfer to S3 is initiated) the backup status is set to OK. This is the desirable behaviour, as the backup is complete and the files on your server (that's what the "OK" status means). However, since the process is still running there is a file stored in your output directory with the backup status. Akeeba Backup waits for this file to get at least 3 minutes old. If it does and it's not automatically removed, Akeeba Backup will assume that your last backup attempt is stuck, remove that file, remove the backup archives and set the backup record's status to Failed. This is consistent with a timeout error when transferring files to an off-site storage like Amazon S3. I need the log file to check that case.

The backup process always seems to go fine, then when it begins post processing it sits for a long time and shows no server response for seconds climbing until it gets to about 595 seconds then goes to the ajax and timeout error. [...] Why does it take so long to transfer to S3?

It's already documented. However, I am not sure if this is the case here as I don't know the backup file's size which is shown in the backup log file.

At this point I have both mssql.connect_timeout and mssql.timeout set to no limit on the server and it not only has not helped but has made the backup process slower.

These are settings for the legacy Microsoft SQL Server driver bundled with PHP. They have nothing to do with your site. They're irrelevant. Probably you wanted to take a look at the mysql and mysqli sets of parameters in your php.ini?

max_execution_time is now 60 on the server (it was 30) and your configuration wizard set it to 30 in Akeeba configuration.

Which means that as per the documentation you should set a Part Size for Split Archives which will result in files smaller than what can be transferred in 60 seconds. How much is that? No idea. The troubleshooting you read tells you to try values of 50, 20, 10, 5 or even 2Mb. It all depends on your server's total bandwidth and its available bandwidth to Amazon S3. It's a try and error approach.

Please help. I want to be able to run backups without freezing Joomla Administrator, get the backup to go to S3, and be able to administer it from Akeeba Backup Admin as it's intended. I don't want to leave server settings unlimited in cases that might be risky.

This is a good clue in absence of a log file. It means that the backup completes so your only problem is transferring to Amazon S3. And I believe the Part Size for Split Archives is what you need to change.

I tried to backup again. It got through the backup processes fine, then started the post processing and just counted the seconds from the last server response up to almost 600 seconds, then ajax error and failed message. Could not get out of the screen, closed the browser, re-opened, the Akeeba main page said the last backup was ok. Went into the Administer Backup Files screen and it says the backup is "OK" but it is local and not transferred to A3.

This further verifies my comment above. The backup itself completes (hence the OK status) but the transfer to S3 is not. SInce you go all the way to 600 seconds (the hardcoded maximum timeout limit in Akeeba Backup's interface) I am 99.999999999% sure that you simply need to change your Part Size for Split Archives.

I just checked again, literally minutes later, and now it says "failed" instead of "ok" and the file name is no longer shown. This is truely maddening. I have spend days on this over the last months and get nowhere.

Further evidence that you have a timeout error during the transfer to S3.

And the log file confirms what I just said:
DEBUG |120818 02:07:51|Total size of backup archive (in bytes): 1516671215

Your backup archive is 1,516,671,215 bytes long, or about 1.3Gb. It's a single part archive. Transferring a file that big will take significantly more than 60 seconds, your PHP timeout limit.

So it all comes back to reading the documentation and troubleshooting document. Follow the last paragraph of the troubleshooter:
Now, for the second leg, we have to do some trial and error. Still in Akeeba Backup's Configuration page, find the Archiver Engine drop-down and click on the Configure button next to it. A new pane opens below. Find the Part size for split archives option and select the 49.99 option. Try a new backup. If it crashes while uploading files to Amazon S3, go back to this option and try smaller values, i.e. 20, 10, 5, 2 or even 1, trying a new backup after setting each one of those values.

If the 50Mb option works you can try increasing it until it fails. My guess is that a value of 40-60Mb will work with your server.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user52353
First of all, my last ticket was created June 1st, #12533, was brief, and your response was effectively, it's not my software, contact your host and ask them how to change the mysql timeout limit. So I did. You did not mention a bug fix. And I HAD updated prior to the June 1st ticket, and most certainly since April.

Secondly, I'm sorry that you find my attempts (after many, many months of this, and many many hours and days testing and changing and getting nowhere) to thoroughly tell you what I've tried before opening this ticket, to be "writing War & Peace". I simply wish to be able to use the backup software that I paid for.

Thirdly, I did NOT visit another page, AND this is irrelevant when discussing command line backups that are processed by cron. During testing I sit staring at the screen for 20 minutes while it counts up the seconds to no server response since . . . . then goes to fail. You can see the latest fail by command line attached to this response.

I don't know why you don't know the backup file size, since I DID provide a log file last night, but another is attached to this response. The backup file is generally 1.4 G.

I'm sorry that you find mssql.connect_timeout and mssql.timeout to be irrelevant. Again, in my last ticket #12533 you told me to change the mysql timout settings. I contacted cpanel support an this is where they told me to do so. I provided you with copies of all the mysql parameters in php.ini in this ticket in case you had any further advice since you brought this issue up in the first place.

Once again, not sure why you have no clue as to my settings and keep saying absence of a log file. I provided you with copies of both last night. A new log file has also been provided in this response.

Now you do recognize the log file?

Ok, I will play around with the part size and php timeout limits and let you know how it goes.

Thank you.







user52353
FYI - "I will play around with the part size and php timeout limits and let you know how it goes". Isn't this what the configuration wizard is supposed to do?

user52353
Also, the troubleshooter that you link to in the last paragraph above, is not the same troubleshooter that the error message links me to in Akeeba when it fails (https://www.akeebabackup.com/documentation/troubleshooter/abbackup.html?utm_source=akeeba_backup&utm_campaign=backuperrorbutton), hence the reason I went through the wrong troubleshooting steps.

user52353
Well, that solved it. 50 mb works.

I have changed my mysql timeout limits back to 60.

I wish I had been told to try this long ago, instead of being referred to the same troubleshooter over and over, https://www.akeebabackup.com/documentation/troubleshooter/abbackup.html?utm_source=akeeba_backup&utm_campaign=backuperrorbutton, and being told to change the mysql timeout settings on the server. I did everything in the troubleshooter I was referred to many times. I followed the links a the end for if you still have problems. I don't know how I ever would have found https://www.akeebabackup.com/documentation/troubleshooter/abamazons3.html. I just tried searching your documentation for Amazon S3 and it did not come up.

When I first setup Akeeba on my old server, and then again on the latest, it worked fine, because I DID go through the instructions and increase part size. It isn't until after errors, and making all the adjustments instructed in the troubleshooter, including being told to use the wizard, that it gets decreased back down.

Thank you. Now I know not to use the Configuration Wizard, even when instructed to do so by the troubleshooter, and instead to immediately increase split part size if I have issues.

nicholas
Akeeba Staff
Manager
First of all, my last ticket was created June 1st, #12533, was brief, and your response was effectively, it's not my software, contact your host and ask them how to change the mysql timeout limit. So I did. You did not mention a bug fix. And I HAD updated prior to the June 1st ticket, and most certainly since April.

Hm, I can see your first ticket on February 15th, 2012. The next one was opened February 28th, 2012 and last updated April 6th, 2012: https://www.akeebabackup.com/support/11294#p69341 (it's a private ticket). I practically begged you to give me access to your site to help you, but you ignored my reply back then. That's the ticket I was remembering. I was suspecting you were using an outdated version and the newer version had already fixed your issue. I didn't tell you that you need to update. You're right about that. That part was just in my manager notes, not the posts you are able to see as a user. Your last ticket before this one was, indeed, June 1st and it was a MySQL error like we had discussed.

Secondly, I'm sorry that you find my attempts (after many, many months of this, and many many hours and days testing and changing and getting nowhere) to thoroughly tell you what I've tried before opening this ticket, to be "writing War & Peace". I simply wish to be able to use the backup software that I paid for.

I didn't mean to offend you. It was a very lengthy description and confused me at many points.

Look, I understand that you are paying for the software and I want to help you use it! In order to do so I need to have up to the point information and feedback from you. Most of the times something doesn't work I have to stick a finger in the wind and make an educated guess of what could be wrong. Usually this comes down to 2-10 different causes. I'll have to walk you through potential fixes so that I can narrow it down to the one cause which is the real root cause of the issue. If you can keep your replies up to the point and coming regularly (like you're doing with this ticket now) I can certainly help and I'll be very happy to :)

Once again, not sure why you have no clue as to my settings and keep saying absence of a log file. I provided you with copies of both last night. A new log file has also been provided in this response.

Now, about my original reply. Yes, it was not very helpful except the last two paragraphs. When I started replying, I hadn't seen your next posts. I was typing my reply as I was following along the thread. I saw the log file after I got to the bottom of your first, very lengthy, post. As I wrote:
And the log file confirms what I just said

which should mean that at this point I did see your log and, yes, it did confirm what I wrote above. I did not delete the rest of my reply as it shows you the way I deduced what is going wrong with the backup. I did so in hope that it will help you with your future troubleshooting. I understand that reading log files is for developers. Troubleshooting based on observations, though, is something everyone can do. Probably I should have gone back and edited my reply above. Being bombarded with a constant stream of emails before you had some coffee on a Saturday morning doesn't help my editing skills, unfortunately :(

FYI - "I will play around with the part size and php timeout limits and let you know how it goes". Isn't this what the configuration wizard is supposed to do?

Um, yes and no. The Configuration Wizard will modify the part size only if it finds out that there is a host limitation for the maximum size of a file that PHP can create. It cannot test the transfer to S3 or anything else. As a matter of fact, when you use the Configuration Wizard it sets the post processing to "None" (no transfer to S3 or any other service). That's why the documentation explains why need to modify it manually and how to do it.

Also, the troubleshooter that you link to in the last paragraph above, is not the same troubleshooter that the error message links me to in Akeeba when it fails

Yes, I know it's not the same. I'd wish I could point you to the correct page for every possible issue, believe me! At the time a backup fails Akeeba Backup has crashed (that's why the backup failed after all) and it can't know why. If I was able to deduce that information I wouldn't present you with just a link to the troubleshooter page, I'd have code to automatically fix the cause. I've tried doing that in the past, but not very successfully. So the best I can do is show you the generic backup troubleshooting page. However you had already deduced that the problem is not the backup part, it's the transfer to S3. I thought you had therefore read the Amazon S3 troubleshooting page. I wasn't sure, though, that's why I provided the link.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nicholas
Akeeba Staff
Manager
Seems like our posts crossed paths :)

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!