Akeeba Backup for Joomla!

Friday, 23 August 2013 07:43 CDT

dkdb

Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes
Have I searched the tickets before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: 2.5.14
PHP version: 5.3.25
MySQL version: 5.5.32-cll
Host: blanye.dk
Akeeba Backup version: 3.7.10 Pro

EXTREMELY IMPORTANT: Please attach a ZIP file containing your Akeeba Backup log file in order for us to help you with any backup or restoration issue. If the file is over 2Mb, please upload it on your server and post a link to it.

Description of my issue:
Hi Guys
I'm still seeing the problem with old backups not being cleared out from Amazon, I have to manually delete them.
It works once or twice it seems, but then it fails.

Best regards Kenneth

Saturday, 24 August 2013 01:36 CDT

nicholas

The log file you sent us is unusable. Please set the log level to "All information and debug" and take a new backup. Remember to send us the log file from the correct origin. If you have not taken a backup from the back-end of your site please do not send us the back-end log file as it contains no usable information whatsoever.

Regarding the S3 quotas do remember that it is two different things you have to do for each backup profile in its Configuration page, under the Quota management header:

- Check the box next to "Enable remote files quotas". As the description reads "When enabled, the quota settings below will be applied to files stored on remote storage, such as Amazon S3 or remote FTP servers." This is what you want, apply quotas to files stored on remote storage (Amazon S3).

- Set up the quotas using the rest of the options. The tooltips will tell you what each option does. For more information on each option please consult our documentation.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 27 August 2013 07:11 CDT

dkdb

Hmm, I find it very annoying that I can't update tickets outside business hours, and it really slows things down, as I'm very busy at work at that time, and I received updates during off hours, that require further information, and then I have to make a reminder to get this done at a later date, not so good.
Oh well, I have no idea why the log was the wrong one, it was downloaded via the webinterface.
I've done this again, and attached it below, I hope it goes better now...
Regarding quotas, I'm pretty sure we discussed that in depth in the previous ticket, that we closed as I had deleted too much, so we couldn't provoke the problem.

Best regards Kenneth

Edited by dkdb on 2013-08-27 07:16 CDT

Tuesday, 27 August 2013 11:29 CDT

nicholas

I will leave alone the first few paragraphs of your reply and only tell you this: I had replied to 168 posts that day (16 bloody hours), with a backlog of 30. If you don't want to respect that that's fine, but don't make an issue about it. I could always do what other developers do: let you file tickets whenever and get back to you weeks or months later. Would that sound better to you?

Regarding your support request, this line contains the answer you want:

DEBUG |130821 02:37:31|No remote files to apply quotas to were found

There are no remote files stored on this backup profile to apply remote quotas on. Remember that as per the documentation quotas are applied per profile. This means that Akeeba Backup will only look at backups taken with profile #5. Please do read the documentation which I kindly linked you to in my previous reply before sending a reply. I believe that you still do not understand quota management. If you find it difficult to grasp please consult our video tutorials where it is explained in great detail.

Be advised that my next reply will be in one week at the earliest. I am taking a few days of vacation for the first time in two years.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 27 August 2013 12:40 CDT

dkdb

First of, have a nice vacation, secondly, yes, it would be nice to have the tickets open at all hours, and as far as I can see, the support hasn't specified a SLA (Service Level Agreement) with short response, so no need to worry about that, but you need somebody to do triage for you.

Ok, to the issue at hand. I've attached the quota settings from profile 5 here.

On the Amazon I have 13 backups at the moment, but looking at the 'Manage Backups' I only see two old 'remote' (profile 5) backups.

Best regards Kenneth

Wednesday, 28 August 2013 01:37 CDT

nicholas

Oh, believe me, I'm looking for a viable support person for months (eleven months and two weeks, but who's counting)! They are either scared off by the volume/level of the support requests even before they begin or they are just not at the level I am looking for. I could easily find personnel if I lowered the support quality drastically, but I vehemently deny doing that. So, yep, I'm still looking. In the meantime I have to resort to methods which limit the number of support requests. I am tempted to have a "rolling window" for replies: You get two four-hour slots to file a new ticket or reply per day, each day having different hours. No matter your timezone you'd get at least three days that this window is within normal waking hours. I reserve this as my Plan B.

OK, back to your issue. Akeeba Backup works perfectly fine. There is absolutely no way* for Akeeba Backup to know which files on S3 are backups of your site, so it never reads the contents of your S3 bucket. This leaves us with a question: how on Earth can we apply quotas? So we are storing the backup archive location, size etc in the database, in the #__ak_stats table. Therefore what we are doing is look in the database records for backups taken with this specific profile. If they satisfy the quota criteria we queue them up for deletion. In your case there were no files detected satisfying the quota criteria (up to 3 backups). Therefore nothing was there to be deleted.

* No the filename is not a good way to find this out for two reasons. Firstly, it would be dead slow and cause timeout issues. Secondly, the naming of the files is up to you and you can even have the same conflicting name across multiple sites or profiles, e.g. site-[DATE].

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 29 August 2013 02:21 CDT

dkdb

Hi again

But why aren't there anything? This backup job has been running for a long time (as you can see from the previous ticket), and it ran again yesterday (it now shows three i manage backups).
I do not prune this manually, it is only done via cron jobs.
So the question is why does it not clean up, even from looking in the database? Do I need to set the quota to 3.1? To have a sharp line from 3 backups to 4?
Well, ponder on that while you are on vacation, and I'll keep monitoring it here...

Best regards Kenneth

Thursday, 29 August 2013 15:54 CDT

nicholas

Try removing all backup archives on S3 and make sure that no other site and/or backup profile creates files in that S3 bucket and directory. You will then see that at most 3 files will be present - remember that the current backup doesn't count towards the quota. If you have other sites or backup profiles backing up at the same S3 location then of course their files are completely ignored.

Moreover please do check the obsolete count quota setting. This explains why obsolete records (the backup archive has been deleted from your local server) disappear from the database.

Akeeba Backup only does what you tell it to do. It cannot do what you would like it to have done unless you have configured it to do that and not something else.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 02 September 2013 05:21 CDT

dkdb

Ok, I'll try and do the cleanout again, I did this the last ticket as well, without any change.
The obsolete record is set to 50, but I don't see any 'remote' obosoletes in the 'manage backup' list, despite that setting, I've set it to 10 now instead.
Not sure what the last sentence refers to? As far as I can understand the docs and the quota settings, it doesn't do what I've set it to. There are no remote obsoletes, and it doesn't clean out old backups.
But as I said, I will now clean out ALL the remote backups, both in the internal akeeba list, and on the S3, and then monitor it.

Best regards Kenneth

Monday, 02 September 2013 09:45 CDT

nicholas

Yes, please do that. It's the only way to understand what is really going on.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 03 September 2013 00:38 CDT

dkdb

Ok, I've set it to run every day to S3, instead of twice a week, but I got a 'failed' last night. I attached the log below.

Best regards Kenneth

Tuesday, 03 September 2013 00:58 CDT

nicholas

According to your log file the backup has run successfully.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 03 September 2013 10:03 CDT

dkdb

Ok, I now see 4 backups, one obsolete, and S3 has four backups as well.
This was four backups taken on the same day, and the 'local' profile has been disabled in the cron.
I'll now let it continue nightly for now, and keep monitoring it, and if that still works, I'll enable the local profile as well, and set the remote to only run twice a week again.

Best regards Kenneth

Tuesday, 03 September 2013 10:28 CDT

nicholas

OK. Sounds like a plan.

Please note that as far as quota management (deleting old backups) is concerned the backups taken with each of the two backup profiles do not count towards the same quota. Simply put: backups from the local profile are from Venus and backups from the remote profile are from Mars. Is that part clear or would you like me to explain further?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 03 September 2013 13:36 CDT

dkdb

No need, it's crystal clear, I just didn't want anything to disturb the testing, and this way will give faster results if something fails.

Best regards Kenneth

Tuesday, 03 September 2013 14:55 CDT

nicholas

OK and sorry for asking. Sometimes the smallest misunderstanding can lead to hours and hours of mutual frustration :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 09 September 2013 07:19 CDT

dkdb

It has now run for a while with only remote backups, and I see 8 obsolete backups, and 4 'active', on the S3 I see four backups as well.
So far it looks fine.
I'm going to activate the 'local' backup as well now, and let the S3 backup run twice a week as before.
Here's hoping that it is working properly now...

Best regards Kenneth

Monday, 09 September 2013 07:26 CDT

nicholas

Please remember that you will be using two different backup profiles, one for locally stored backups and one for backups stored in Amazon S3. The quotas on each profile run independently. I believe that this is what was confusing you in the first place.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 10 September 2013 00:38 CDT

dkdb

Oh no, I'm well aware that they are supposed to be seperate 'tracks', and their quota should not influence each other.
But when testing, as many influences should be removed as possible, to make sure that you are not testing several parameters at the same time.

Best regards Kenneth

Tuesday, 10 September 2013 01:07 CDT

nicholas

I was just making sure you're not going to get confused during testing :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 23 September 2013 16:46 CDT

dkdb

It's still working as planned, so I guess we can safely close this ticket.
Not sure what the issue was, but I suspect a previous problem wasn't solved until I did a complete cleanout of S3 as well as old backup jobs in Akeeba.

Best regards Kenneth

Support