Akeeba Backup for Joomla!

Thursday, 27 October 2011 23:09 CDT

user9599

Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes, I read AkeebaBackup.com Troubleshooter including troubleshooting instructions for Akeeba Kickstart and all related documentation.
Have I searched the forum before posting? Yes
Have I read the documentation before posting (which pages?)? Yes I read the Akeeba Backup documentation and the Kickstart documentation.
Joomla! version: Joomla! 1.5.23 Stable
PHP version: 5.3.8
MySQL version: mysql Ver 14.14 Distrib 5.1.59
Host: Rackspace cloud server
Akeeba Backup version: Akeeba Backup Professional 3.3.4

EXTREMELY IMPORTANT: Please attach your Akeeba Backup log file in order for us to help you with any backup or restoration issue.

Description of my issue:

When I attempt to restore a (7GB) jps archive with kickstart, the restoration goes extremely slow and the I/O of my browser from the target site is 75KB/s incoming and 75KB/s outgoing for a total of 150KB/s. I've tried this with Google Chrome and with Firefox. In contrast, restoring a jpa only takes a few minutes and uses almost no browser bandwidth. The jps takes so long I have yet to complete a test restore. I wonder, is the entire 7GB archive being piped through my browser? Is this the expected behavior for restoring a jps archive with Kickstart? Are there any reliability issues with this behavior?

In order to complete secure remote backups with compressed archives, jps over ftp seems the only option in Akeeba backup. SFTP is not supported for post processing, and archiving is not available for DirectSFTP. No problem if jps is a little more trouble to use. Most of our backups are to local media, the remote backups are only for redundancy. But I want to make sure jps is as reliable as jpa before I depend on it. Is it safer to stick with jpa which I note is the recommended format?

Thanks,

Kirk

Friday, 28 October 2011 08:27 CDT

nicholas

Hi Kirk,

JPS archives are encrypted with AES-128. This means that they have to be unencrypted and uncompressed. This is too slow. Moreover, it needs a lot of page reloads which is what is driving your bandwidth usage crazy.

On the other hand, JPA and ZIP archives are not encrypted. Moreover, larger files are not even compressed. This makes their extraction much faster and requires less page loads.

That's the difference you are experiencing :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 28 October 2011 21:35 CDT

user9599

Thanks Nicholas. It would be nice if Kickstart worked with Linux text browsers like elinks or lynx. Then we could run Kickstart with no network activity needed. The bottleneck in the jps extraction process looks like the page loads, because the server shows no load from the extraction itself. If only the "start" button in Kickart could be activated from a text browser, it looks like it might work.

If we ever need to actually use our remote backups, it won't make much difference how long the extraction takes, as long as it works. But I do not like to depend on backups without periodically testing them. Simply to make it easy to test the remote backup archives, it would be great if there were some way to do the extraction process from the server itself. I'm thinking it could be simpler to make Kickstart work in a text browser than create a separate command line utility, but perhaps neither is practical.

Thanks,

Kirk

Saturday, 29 October 2011 06:23 CDT

nicholas

Text browsers do not support Javascript which is required in order to split the extraction process in multiple steps. If you do not split the extraction process its steps, this 90-1000 second operation would have to complete within a single page load. Let's say that PHP time limit (usually 10-30 seconds) was not an issue. Memory limit would be an issue. Also, server-imposed CPU usage restrictions would be an issue. In other words: it would not work.

Making a command-line script is actually very easy. But except you and me, I don't think anyone else would like to use it :D Well, if you are interested, tell me so and I will look into that next week.

Alternatively, you can extract the JPS archive on your local machine using the cross-platform Akeeba eXtract Wizard software we give away for free, then upload the extracted files to your host by FTP. Just access the installation script as http://www.example.com/installation/index.php and you can go on with the restoration. If you are restoring to a local testing server, this is a very convenient alternative, actually! I use myself :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Saturday, 29 October 2011 17:05 CDT

user9599

I guess extracting jps with Kickstart is the best way to test the archives either way. I let it run all night and it completed without any problem. With your explanation of how this works I feel confident it should be as reliable as jpa.

As a matter of fact, the first jps archive test I did was with the Windows Akeeba eXtract Wizard, but I get 'invalid archive format'. This is with Akeeba eXtract Wizard 3.3 and the same jps that works fine with Kickstart, produced with Akeeba Backup Pro 3.3.4. I don't personally need this to work any time soon, but for what it's worth.

Thanks,

Kirk

Saturday, 29 October 2011 23:56 CDT

dlb

Kirk,

How did you download the archive file to your local computer? FTP in binary mode is the recommended way. ASCII mode can damage the archive, which would account for the invalid format.

Dale L. Brackin
Support Specialist

English: native

Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

Sunday, 30 October 2011 19:56 CDT

user9599

Downloaded in binary mode and verified the md5 checksums. The 'invalid archive format' error is produced instantly, before the actual files could be checked I think.

Monday, 31 October 2011 02:43 CDT

nicholas

That was a problem with Akeeba eXtract Wizard 3.2, but version 3.3 -released on August 21st- solved that. Which version do you have?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 31 October 2011 13:28 CDT

user9599

I'm using Akeeba eXtract Wizard 3.3. I saw that this was an issue with 3.2 and even questioned my own eyes by downloading and installing 3.3 again from a fresh copy. Same error.

Remember that I personally do not need this to work any time soon. If you think this could be some mix up on my system alone you can simply wait and see if anyone else has a problem as far as I'm concerned.

Thanks,

Kirk

Monday, 31 October 2011 16:19 CDT

nicholas

Hm, I don't know what's going on. I tried producing JPS archives on different servers with Akeeba Backup 3.3.4 and tried them with eXtract Wizard on three machines (Windows, Linux and Mac OS X) and they worked. Apart from a corrupt archive, I can't think of anything else :(

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 31 October 2011 18:15 CDT

user9599

Can't be a corrupt archive because the same archive works with kickstart on a server, and the md5 checksums between the two copies match. I would not doubt it is a an issue with my particular Windows version or installation. I'll try it in another Windows setup and see what I get.

Thanks,

Kirk

Tuesday, 01 November 2011 02:58 CDT

user9599

Hi Nicholas. Further testing makes me wonder if this could be a PHP version and/or a mcrypt PHP extension version issue. JPS archives made on the server with PHP 5.3.8, and mcrypt built for PHP 5.3, work with Kickstart running on the same server, but these archives do not work with eXtract in Windows 7, Windows XP, or OSX. JPS archives built on a system with PHP 5.2.13 work fine for me in eXtract. eXtract is version 3.3 in all cases, and all archives were created with Akeeba Backup Pro 3.3.4.

Now that I think about, I really would like this to work in OSX. Am I right that eXtract uses it's own internal encryption rather than the libraries on the system, while Kickstart uses the system libraries? I ask because this means there is no way for me to resolve the issue myself in the system running eXtract.

What do you think? Does eXtract need to be updated to support newer PHP and/or mycrypt libraries? Is the mycrypt setup on our sever at fault? Attached is a backup log with the same password I sent by pm for another post yesterday.

Thanks,

Kirk

Tuesday, 01 November 2011 03:45 CDT

nicholas

Hi Kirk,

Kickstart is written in PHP, uses the PHP mcrypt extension which, in turn, uses your system's mcrypt implementation. eXtract Wizard has its own AES-128 library. However, note that the standard is one and ANY implementation of AES-128 is able to decrypt any other AES-128 implementation's output! It's the basic premise of encryption and exactly what allows, say, a Windows 32-bit machine to access over SSL a site hosted on a 64-bit Linux server.

That said, I did a lot of testing (about 6 hours worth of it!). I tried creating JPS archives with: Win 7 32-bit PHP 5.3.5, Win 7 32-bit PHP 5.2.11, Linux 64-bit PHP 5.3.8, Mac OS X 64-bit PHP 5.3.8 and restore them on four machines running Windows XP 32-bit, Windows 7 Home Premium 64-bit, Ubuntu Linux 11.10 64-bit and Mac OS X 10.7.2 64-bit. In all cases the result was the same:
- If I used the WRONG password (or none at all), I would get an immediate message about the archive being corrupt, which is the expected outcome.
- If I used the CORRECT password, everything worked just fine.

I tend to believe that the problem is in the password you used. If it contains anything else except A-Z, a-z, 0-9 and the standard special characters of a US keyboard (!@#$%^&*(),.<>/?\|'";:]}[{-_=+) it is possible that you get decryption errors. More specifically, I have confirmed that if you use non-ASCII characters, the encryption fails. This has to do with the internal encoding conversion performed on each system. As long as you stick to ASCII characters I found no issue when decrypting JPS archives with Akeeba eXtract Wizard.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 01 November 2011 04:17 CDT

user9599

The password conforms to the specification. I used the same password for the tests with the system that did work with eXtract. BUT if I intentionally enter the wrong password with working archives, I do get the same 'invalid archive format' error. I am going to test with a reset password in the profile configuration. I won't be back to see the results till this afternoon, but I am hopeful.

Thanks,

Kirk

Tuesday, 01 November 2011 04:24 CDT

user9599

Shoot, I now realize a bad password in the config can not explain this because the same archives work in Kickstart with the intended password :-/

Tuesday, 01 November 2011 04:41 CDT

nicholas

I'd recommend following the "stupid troubleshooter" method (that's what I always do!). First, go to the configuration and reset the password to something short, just for testing, e.g. test1234. When clicking on Backup Now, do note that there is a password field. Normally, it contains the intended password. Browsers have this strange tendency to overwrite the field with your Joomla! login password (don't you love auto-complete?). Type the password in there again and take the backup. Download the backup using FTP in binary transfer mode. Run Akeeba eXtract Wizard, select the archive and type in the password. This should work. I've done it dozens of times yesterday during testing :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 02 November 2011 02:34 CDT

user9599

Same result: 'invalid archive format'.

My other backups were made with the native cron script on the shell.

Wednesday, 02 November 2011 02:36 CDT

nicholas

I can't replicate it over here. There seems to be some kind of incompatibility with your server, but I can't understand what is it :(

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Wednesday, 02 November 2011 02:45 CDT

user9599

Good because that's what I think too :-) I'm thinking this must be an issue that has shown up under some other application. I'll try and search for info and that when I get time. I'm guessing this will more than likely be something to change or fix on our server.

Thanks,

Kirk

Wednesday, 02 November 2011 02:48 CDT

nicholas

OK. Please let me know if you find something!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 03 November 2011 22:55 CDT

user9599

Hi Nicholas. Could you send me the output of phpinfo for the Linux 64-bit PHP 5.3.8 system you tested with? Then I can compare with our system, possibly find something missing. So far I can not find anything wrong with our zlib or mcrypt.

I even did more "stupid troubleshooter" tests, same result, can't be any mistaking this issue as bizarre as it sounds.

Thanks,

Kirk

Friday, 04 November 2011 02:06 CDT

user9599

You want to hear something really crazy? Or maybe it will make sense to you. I discovered this by accident. If some of the multipart archives are absent, eXtract will check the existing parts and then error with 'A decompression error occurred while processing the archive'. No idea which parts need to be absent to reproduce this. I can remove some parts and still get 'Invalid archive format' without any parts being checked that I can see, instant error message. Remove other parts, I get 'A decompression error occured while processing the archive'.

Friday, 04 November 2011 03:07 CDT

nicholas

It's not crazy at all. It's absolutely expected. Remember when the documentation was screaming at you to put all archive parts on the same folder? That's why ;) Both Kickstart and eXtract Wizard treat a multi-part archive as a single big data stream. If a part is missing, they can not read from it any more and the extraction fails.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 04 November 2011 15:23 CDT

user9599

It doesn't strike me as odd that the extraction would fail if all the parts are not in the same folder. Of course I would expect that. What strikes me as odd, eXtract will initially accept my archives as valid (no instant 'Invalid archive format' error before the parts are processed), if certain parts and only certain parts are missing from the archive. With all the parts intact my archives are never accepted by eXtract as valid, not for a split second. I can remove many parts from the archive and the archive is still not accepted as valid. But remove certain parts and the archive is accepted as valid. It is as if there is something wrong with certain parts and only certain parts of the archive. Again the archives work with Kickstart on the server and the md5 checksums in the copies I download are identical to what is on the server, so all the parts appear to be fine as far as Kickstart is concerned. It seems there is something in the contents of certain parts that eXtract will not accept. If that does not strike you as significant, then it is irrelevant, just trying to explain exactly what it is that I thought was crazy about the result.

Thanks,

Kirk

Friday, 04 November 2011 16:29 CDT

nicholas

Hi Kirk,

Here's how it works. First, it tries to read the header from the first part of the archive (the .jpa). It checks the number of parts and tries to see if the last part is there. If you remove the last part, it will fail immediately.

Then, it will start extracting the archive, reading data from the first part. When the first part is over, it tries to find the next part and so on. If at some point the expected backup archive part is missing, it will halt with an error; the archive data is missing and will throw an error regarding this problem.

There is a catch, though. If the start of a part coincides with the start of a file, you can remove it, as long as the next part available does begin with the start of a file as well.

Normally, if you have a full backup set, with all archive parts, the archive should be read normally.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 04 November 2011 17:04 CDT

user9599

Thanks Nicholas. That makes sense. And if you don't think this is significant as far as the behavior of eXtract I don't need to understand as far as that goes, but I am looking for clues about what could be wrong with our server setup to produce jps archives that are incompatible with eXtract. To that end I hope you will forgive me questioning such odd things.

The thing that I still can not see is this: Why does removing a part cause eXtract to see the jps as valid, when the jps is seen as invalid when all the parts are present and intact?

If it is a missing part that contains the start of a file that makes the difference in response, which does match the behavior perfectly, why would this cause eXtract to disregard the idea that the jps is not valid when otherwise it instantly errors with 'Invalid archive format'. If I understood that, it might help me understand what library or component on our server might be responsible for producing an incompatible archive.

Thanks,

Kirk

Saturday, 05 November 2011 03:47 CDT

nicholas

Hi Kirk,

Um, this sounds like a bug in eXtract Wizard. Does it immediately say that the JPS is invalid, or after a while, that is after it has extracted some files?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Saturday, 05 November 2011 04:13 CDT

user9599

eXtract immediately says the jps is invalid, no delay at all.

Support