Support

Akeeba Backup for Joomla!

#18545 Site Transfer Wizard Hangs

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Wednesday, 18 December 2013 01:50 CST

topwebs
EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue: Trying to use the Site Transfer Wizard to transfer a site between two accounts on my own server. The process hangs at random files and then times out (90 seconds). It always stops on trying to upload different files. Normal ftp operations on the server have no issues. I have repeated this over 6 times and it always hangs at a different file. I have deleted files in the destination directory and it still hangs at a different file every time.

nicholas
Akeeba Staff
Manager
This is a limitation of how most FTP servers work and how PHP works.

Problem 1: Blocked by FTP server. In order to avoid server timeouts, the site transfer (essentially: a backup) has to be performed in multiple small steps. On each step we have to connect to the remote FTP server, log in, transfer a few files, log out. This roughly means that we have to do this log in, transfer, log out routine 3-4 times per minute over several minutes (or hours). This tends to cause most FTP servers to consider this behaviour as an attack and block logins from this IP for a while. The next time we try to connect to the FTP server we are blocked and the site transfer fails.

Problem 2: Large files. The definition of "large" mostly depends on the transfer speed between your original and your target servers. If a file takes too long to transfer it is possible that the FTP server closes its connection. It is also possible that, before that, PHP runs into a timeout and halts. In both cases it cases site transfer failure.

When this happens we recommend an easier, alternative method to transferring your site. Create a regular backup profile and use the "Upload to Remote FTP Server" or "Upload to Remote SFTP Server" to transfer the backup archive to the new host. As a precaution, you are advised to set the
Part size for split archives in the Archiver Engine configuration pane to 20Mb or smaller. After the backup is complete the backup file(s) will be transferred to the new server. Just upload Kickstart into it and run Kickstart. This method uses far fewer FTP connection requests than the Site Transfer Wizard, it's faster (usually: five to ten times faster!) and only requires you to upload 1 file manually.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

topwebs
As we have a WiredTree server which can be customized in every way, how can we tweak the standard FTP server to allow the STW? Quite frankly, if the STW does not work on most FTP servers and you can't give us the requirements for it to work, it really shouldn't even be published.

As a developer, I would like a way to go to Akeeba in my source site and transfer the site to another account without leaving the source site backend (this assumes a database exists on the other account). As the STW does not seem usable, see my initial comments, I would like to recommend that you change the STW or add an option to utilize the Upload to Remote FTP Server (which does work), include the transfer of kickstart and create a link to open kickstart on the target site. And yes, I know I can go into my FTP program and upload kickstart and do the same thing, but I would like to do it all from the source site where I can set up a variety of profiles for different targets. This shouldn't be difficult. What do you think?

nicholas
Akeeba Staff
Manager
I already told you what the possible problems could be. Let's start with the easiest issue, #2. Make sure that your origin server has a large PHP timeout (90-120 seconds at least) and that Apache is configured to have a timeout larger than PHP's.

If that doesn't work, you'll have to check that your target FTP server doesn't block connections if too many connect attempts originate from the same IP address. I can't help you with that, I am not your host.

> Quite frankly, if the STW does not work on most FTP servers and you can't give us the requirements for it to work, it really shouldn't even be published.

I disagree. It is a great way to transfer sites within the same server. Moreover, if you are strictly speaking, all web software doesn't work on MOST servers. Even Joomla!. It needs specific web software, PHP versions etc. Does this mean that no web software should ever be published because it can't work on MOST servers in the world?

> I would like to recommend that you change the STW or add an option to utilize the Upload to Remote FTP Server (which does work), include the transfer of kickstart and create a link to open kickstart on the target site.

It is being already considered. However, uploading Kickstart would potentially expose a security hole on your site. As you know, you can only upload files which are present on the source site. This means that we'd have to put kickstart.php on your source site, in a rather predictable location. If for any reason that folder is not properly secured (which could happen on many servers for various reasons) you would have a gaping security hole on your site.

I understand that you prefer convenience, but we prefer security. Unless we can guarantee that no security hole will be opened we won't offer an convenient, but insecure, feature.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

topwebs
The accounts are on the same server. Wiredtree support indicates that the firewall should not be blocking internal FTP transfers. The timeout is not the issue at all. Files are transferred quickly and then the transfer process just stops and nothing happens until the PHP timeout. PHP timeout = 90 seconds. Apache timeout = 300 seconds. It seems to me that STW should work easily on the same server, but it's not. Any other ideas?

nicholas
Akeeba Staff
Manager
When you get this message it means that this command:
$res = @ftp_put($this->_ftphandle, $realname, $sourceName, FTP_BINARY);

has failed. When the error message is produced we have established that $sourceName is a readable file, so there's no question whether that files exists. We not, however, told why the upload failed, this is a limitation of PHP. I do however see this time discrepancy:
DEBUG   |131216 20:15:10|  Adding installation/angie/language/et-EE.ini; Next offset:164960
ERROR   |131216 20:16:40|Uploading installation/angie/language/et-EE.ini has failed.

PHP is trying to upload a file of 10Kb for 90 seconds. As you are on the same server there is no way it can possibly take that long. It's also impossible that you have a networking issue (the network isn't used on loopback connections). This leaves us with the only possible reason: the FTP server, for some reason, doesn't respond.

If that file was uploaded on your server, your FTP server didn't send a 2xx successful response back to PHP. As a result PHP doesn't know if the upload succeeded, which is why the transfer wizard fails.

If that file wasn't uploaded on your server, your FTP server didn't send a failure response back to PHP. As a result PHP doesn't know if the upload failed, which is why the transfer wizard fails.

Both of these conditions occur inside the remote FTP server software. There's nothing I can do in my code to prevent, detect or work around it.

Nicholas K. Dionysopoulos

Lead Developer and Director

πŸ‡¬πŸ‡·Greek: native πŸ‡¬πŸ‡§English: excellent πŸ‡«πŸ‡·French: basic β€’ πŸ• My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!