Support

Akeeba Backup for Joomla!

#8506 Incremental backup

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Wednesday, 12 October 2011 04:37 CDT

suelasrotas
Hi all,

I was wondering how to create a profile in order to setup a proper incremental backup, as the site I'm managing is quite large (400MB) and needs to be backed-up everyday.

In the documentation says the Pro version is able to do it but unfortunately there's a lack of information there about how to do it.

Thanks in advance.

dlb
Akeeba Pro doesn't do an incremental backup. This feature has been requested, but has too many technical limitations to be implemented.

Akeeba (both versions) can do a database only backup. Depending on how often you get new files on your site, the database only backup may work as a substitute for your incremental backup. That is where the most frequent changes are.

One of the features that is only available in the Pro version is the ability to trigger a backup with a cron job. That may be where the confusion arose.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

suelasrotas
Thank you Mr. Brackin for your post.

In that case there's a mistake in the documentation where you can read 'incremental backups' on page number 7. I based my decision on that to buy the Pro version. But, because I've been using Joomla! Backup for years, I'm more than happy to contribute to keep this project alive. You have saved my 'life' before many times so it's fair enough.

Although I recommend to change that line in the manual.

Thans again and keep going!

dlb
You're right, that's what it says. It is on page 2 of the current edition. My definition of "incremental backup" is that it only backs up changed files. So to restore, you have to start with a full "base" backup and restore all your increments up to the most current. Akeeba doesn't do that. (Unless he slipped it in there and I missed it. :( Wouldn't be the first time that happened.) I'll flag this for Nicholas to review.


Dale L. Brackin
Support Specialist


us.gifEnglish: native


Please keep in mind my timezone and cultural differences when reading my replies. Thank you!


????
My time zone is EST (UTC -5) (click here to see my current time in Philadelphia, PA)

nicholas
Akeeba Staff
Manager
Yes, this shouldn't be in the description, because it is a manual process.

The idea is that you can have files-only and database-only backups which you can extract on top of the other to effectively simulate incremental backups. Since this process isn't automated, I'm removing this from the documentation and I'll also update our site in the next beta release.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

suelasrotas
Hi Mr. Dionysopoulos,

As long as I can tell by your posts, Akeeba Backup could be able to do a sort of incremental backup creating different profiles, am I right?

So, what will be your strategy if you were to backup a 400MB web site?

Thanks a lot in advance,

nicholas
Akeeba Staff
Manager
Just call me Nicholas, I know that my surname is impossible to pronounce or remember (unless you're Greek) :)

The "incremental" backup strategy is not a truly incremental one, i.e. you won't be able to backup only the updated files. However, you can apply some smart rules to reduce the volume of backups. Most of the times, the bulk of your site's content exists in rarely modified and quite large media files. In this case, I would create two profiles:
1. The default profile which backs everything up. This would serve as a starting point for all incremental backups and I'd run it at least once per week, let's say every Sunday. It's good to have a "last known good" state to start over, in case everything goes awry.
2. "Small" full site backup. In this profile I would create a RegEx files filters with rules to avoid backing up media files. Let's say I have a lot of MP3s, MP4s and AVI files I would like to skip. I'd create the following three rules:
#\.mp3$#i
#\.mp4$#i
#\.avi$#i
This profile would run daily.

Let's assume that a site crash occurs on a Thursday. In order to roll back to Wednesday's backup I would have to extract last Sunday's backup from Profile #1 and extract on top of it Wednesday's "small" backup from Profile #2. The resulting set of files would be equivalent to a full site backup on Wednesday.

As I said, this is a manual and long process, far from perfect. I am now considering of creating an optional filter which will help in incremental backups. When activated, it will only back up files which have changed since the time the last backup taken using the same profile. This will be a truly incremental backup, however the restoration process will still require you to extract multiple archives one on top of the other. This filter will be present in the 3.1 release.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

suelasrotas
Thank you so much, Nicholas! I think I'll setup those strategies and wait for the filter in 3.1, which it'll be just fantastic for me.

BTW, your surname is easy to remember as my mother tongue is Spanish, but I'll rather go for Nicholas then.

10 out of 10 support team!

nicholas
Akeeba Staff
Manager
Thank you for your kind words :) I'll be working on this feature today and, probably, a bit of the rest of the week.

Yes, if you come from a Mediterranean country (Greece, Italy, Spain or Portugal) my surname should look very pronounceable :D

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user31869
Sorry for bringing such an old thread back to life, but I feel it is pertinent to ask where things are in terms of incremental backups.

In a particular portal we are building, the production environment is completely sealed off to us. Therefore, as we add more functionality to the dev environment, the tougher it will be to upstream the changes to production.

If Akeeba Backup Pro provided the ability to backup and merge only the changes to certain files and tables between dates X nd Y, it would be brilliant.

I also have a practical question. When using a second profile which only backs up certain designated files / directories and DB tables, will it merge with the installation it is restored upon or will it replace everything?

nicholas
Akeeba Staff
Manager
Hi!

Incremental backup is already implemented, albeit it works only for files, not for database content. It's technically infeasible to perform a table merge the way you describe it. What you need is called "Site synchronisation" and I had written a blog post last November explaining why that can not work. It's a long read, as it is an in-depth analysis of the technical challenges involved. I wrote it because people kept asking me to create something like that ;)

Regarding your practical question, restoring that backup will only overwrite the files and folders contained in that backup archive. Anything not in the archive will not be touched. This is true with the restoration of any and all Akeeba Backup backup archives.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

toomanylogins
can I just clarify how the incremental backup is working. I have as follows

Profile 1 is a full site backup scheduled every day using the Plesk scheduler and excluding all of our image files. This works out at 30mb

Profile 2 so site files backup of the image files only which is 660mb.

I would like to create a profile 3 which is incremental of the image files. However in the documentation it states "The "last successful backup" refers to the last backup made using this backup Profile"

This means the incremental backup has to use profile 2 how can I schedule this as profile 2 is already defined.

Therefore it seems the only way to do an input mental backup is to keep editing the profile ?

Am I misunderstanding something.

Thanks
Paul

toomanylogins
Can you add an override variable to the command line to switch from "site of files only" to "files only incremental"

If so can someone please provide the syntax.
Thanks

nicholas
Akeeba Staff
Manager
Hi Paul,

Well, that's a scenario that will not work. The incremental backup works by examining the modification date of each file and comparing it to the last backup's date and time. If you backed up your site on the 1s of the month and you're backing it up again on the 5th, it will backup all files modified between the 1st and the 5th.

In your case, Profile #2 runs daily. The incremental backup would, therefore, not include all the modified files if you used the command line switch. You can, however, do something different.

Assuming that you want profile #2 to become an incremental backup profile, all you have to do is to set its type to "files only, incremental" and remember to DISABLE all quota settings (you wouldn't want to delete your old backups which are required to restore your images, would you?)

Alternatively, copy profile #2 and set the new profile to incremental. Run profile #2 every week and the incremental backup every day. Set the count quota of profile #2 to 1 and the incremental profile's Count quota to 7. This way all you need to restore your images directory is to restore the latest backup from profile #2 and all of the archives taken with the incremental profile (up to 7 of them, depending on the day of week). If you turn on the remote quotas setting, you have set yourself up with an amazingly simple and powerful solution to efficiently backup your very big images directory without paying a lot of money to Amazon :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

toomanylogins
Thank you for the reply. Your last paragraph describes the situation perfectly. I will implement this as described the only thing I don't understand is how the copy of profile#2 which will then be profile 3 works with the statement "The "last successful backup" refers to the last backup made using this backup Profile"

What I think you are you saying is that the first time you run profile 3 it will in fact include all 660mb thereafter it is incremental. And when the first run of profile 3 is deleted via the quota we have the situation with a weekly 660mb from profile 2 and the daily incremental from profile 3.

regards
Paul

nicholas
Akeeba Staff
Manager
Copying the profile is very easy! Go to the Profiles page, select the checkbox on the left of profile 2 and click on the Copy toolbar icon. The profile -including all of its settings- is duplicated. Click on the copied profile to change its name. Back in the Control Panel page, select the new profile from the drop down. When the page reloads, click on Configure. That's all :)

And yes, everything else is as you understood them.

BTW, that was a use case I hadn't thought of when designing this feature. It's good to know that such unforeseen uses not only exist but are perfectly possible :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

toomanylogins
hello Nicolas,

I have incremental and full backups working on a schedule including remote quotas on Amazon S3. Very effective. The only anomaly I have noticed is the log. As we using the command line the log file is overwritten for each scheduled backup.If we have a problem with a backup ( four example failed to upload to Amazon as it did this morning ) and the next scheduled backup runs we lose the log of the previous backup. Is there any way for log file per profile?

Regards
Paul

nicholas
Akeeba Staff
Manager
Unfortunately, no. The backup log filename follows the conventions akeeba dot origin dot log. This naming convention is hardcoded and is mandatory to allow two essential things:
- Auto clean-up of older log files
- The ability for the View Log page to work without frustrating the user

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!