Support

Akeeba Backup for Joomla!

#34266 Predefined variables in the message

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Akeeba Backup version
n/a

Latest post by nicholas on Monday, 28 December 2020 11:26 CST

OlegKor

During the creation of Automatic backups, the system can send a message about this to e-mail.

We can use predefined variables in the subject and body of the letter:

[DATE]
[HOST]
[PROFILENUMBER]
[PROFILENAME]
[PARTCOUNT]
[FILELIST]

This is all great, but the variables are clearly missing:

[TIME] (or [DATETIME])
and
[FILESIZE] (or [FILELISTSIZE])


Or do they exist, we just do not know about them?

Are there any other predefined variables besides those described in the tooltips?

 

nicholas
Akeeba Staff
Manager

You do not need a time variable, but it's still there. You can use all of the archive naming variables in the subject and body. Moreover, the body can additionally use the [PROFILENUMBER], [PROFILENAME], [PARTCOUNT], [FILELIST] and [REMOTESTATUS] variables which are specific to the email message body.

There's a good reason there's no size variable. If you are using a remote storage engine and have enabled both the option to delete the files after uploading and the immediate upload as soon as the archive part is ready the size recorded by Akeeba Backup at the end of the backup is the size of the first and the last part only. If your backup has more than two parts this is incorrect. Doing otherwise would require either removing the immediate upload feature or the upload retry feature in some remote storage integrations (OneDrive, Dropbox, Google Drive etc). Weighing all our options, not reporting the backup size in emails is actually the best course of actions with minimal disruption for our clients.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

OlegKor

This is so, but when the System reports the name of the file to (FILELIST) it would be logical to indicate the size of THIS file.
And if they are in the list, they have their own unique names and apparently they have their own unique size.
It is so?

And what happened to them later (it was deleted, loaded, moved or disappeared in space) is already a conversation

The developer is always right - if you think the job number is useful and the file size is rubbish. Then everything is OK.

 

nicholas
Akeeba Staff
Manager

We do not store the name or size of each part anywhere.

The part name is easy to deduce. The naming pattern for JPA files is .j01. .j02, ..., .jpa, for JPS files it is .j01, .j02, ..., .jps and for ZIP files it is .z01, .z02, ...., .zip. Knowing the number of parts (which is stored in the database) we can produce an accurate list of the names of the part files.

The sizes, however, are another kettle of fish. You might think that the full parts (.j01, .j02, ...) have a file size that's equal to the part size for archive splitting. While this is typically the case it's not necessarily the case. They might be a few bytes smaller when after putting a full file in there the part size minus the current file size is smaller than the size of the header of a new file record. This is rare but it does happen. Therefore this size could be a few bytes off.

Even if that was the case we couldn't know what is the size of the last part (.jpa, .jps, or .zip) reliably. For single part archives this is easy: it's the recorded backup size in the database. When using remote storage and you DO NOT have enabled both options to delete the files after uploading and the immediate upload as soon as the archive part is ready we could assume that it's the total size of the backup minus n-1 times the part size where n is the number of parts of the backup archive. However, this is contingent to each full part being exactly the part size which, as I said, is not necessarily the case. Therefore this size could also be a few bytes off.

Moreover, when using remote storage and you DO have enabled both options to delete the files after uploading and the immediate upload as soon as the archive part is ready we can't deduce individual part sizes, period.

Storing the individual part sizes would be the better option but it's impractical. We could implement it two ways. One is using a second database table which records the backup ID and part number along with the part size in bytes. This would make the Manage Backups page very slow to load because we would need to join two very large tables which is abysmal for performance. The other option would be to store this list as a JSON encoded field in the #__ak_stats table. However, if your backup has hundreds of parts (believe me, it's far more common than you may think!) we end up with a supermassive query which might exceed MySQL's packet size and cause the backup to fail with a database error. Even if we decided to drop support for any MySQL version older than 5.7 and used MySQL's built-in JSON handling features to append to this JSON-encoded field we'd run into the same MySQL maximum packet size issue when restoring the backup of the #__ak_stats table, i.e. your restored sites would not know about older backups.

There are other more subtle issues which have to do with the implementation of archive import, backup management and the JSON API among other things but they are secondary and relatively easier to handle. The main problem is that we would either have potentially inaccurate sizes reported or we would get into a situation where we have backup and restoration issues just to report backup file sizes in the email.

As I said, weighing all our options, not reporting the backup size in emails is actually the best course of actions with minimal disruption for our clients.

To make it abundantly clear: you are not the first one to think about reporting file sizes. I did think of that already, a long time ago – about ten years ago, actually. I already considered what it would take and how it would affect the component. In the end of the day reporting individual file sizes is not a critical feature but trying to implement it can and will cause issues in critical features (you can't really publish backup software if you know there are preventable issues during backup and restoration that you could solve by removing the non-critical feature that's causing them!).

So, I have to insist that this is a feature that I will not implement not because I don't care or don't know how but exactly because I do care, do know how and fully understand the second and third order effects of its possible implementations. It's easy to say yes to all features, it's really hard to know when to say no. That's the difference between bad and good software. If you've ever wondered why Windows, macOS, Android, iOS, GNOME and KDE seem to drop features, that's why. Delivering good quality, consistent software frequently requires removing or refusing features which cause more trouble than the marginal benefits they deliver.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

OlegKor

Okay, it doesn't matter.
If everything is so easy to get in the database, compare and analyze ... Why send Newsletters at all? There is no functionality for sending messages to the mail - no problem. Why did it all build?

I just didn’t know that THIS was impossible to implement as a VARIABLE ... I didn’t know that you can get the file name into a variable and its size is extremely difficult.

Let it be as it is - everything is simply amazing. No problems or questions.

As far as I understand, there is no point in asking more serious questions.

If there are such problems with a variable (10 paragraphs of explanations why you don't need to do it), then it's better not to ask about the possibility of specifying two or more mailing addresses to send such a message? Probably there will be three pages of text about why several addresses cannot be specified separated by semicolons, etc.

Why so much text at all. Everything is much simpler.
You send a regular message and no one demands from you that this information does not change during the further use of the System. This message simply informs that such a file or 10 files were created at such and such time.

Excuse for troubling.
Let it remain as it is, otherwise, God forbid, something will fall off because of the variable. I understand that the value of a variable sent by mail is a CRITICAL SITUATION ...

There are no more questions and I think that even if there are difficulties, there will be no more questions. I will write queries to the database.

All the best

nicholas
Akeeba Staff
Manager

Regarding multiple email addresses this is a support request and I can answer it. You can add as many email addresses as you want, separated by commas. For example:

[email protected], [email protected], [email protected]

This has not always been the case. Someone asked us for this feature, we implemented it. If you feel there could be an improvement in the presentation of this field and / or its documentation we would appreciate your feedback in a new ticket.

Regarding everything else I will start by addressing your uncalled for allegation that you cannot expect technical support from us.

When you submit a technical support request we will of course reply and help you. If it's something already documented we will point you to the documentation. If we feel that your question does not have an obvious reply in our documentation we will try to explain it to the best of our abilities. If you are facing an issue we will do our best to identify the root cause of your problem, contingent on your cooperation regarding any necessary additional information or access to an affected server is reproduction outside of it seems to not be feasible. If the problem lies on our code we will make changes to it to address your issue. If the problem lies on a server configuration setting we will let you know what it is so you or your host can address it (we obviously cannot reconfigure your server ourselves since we have not set it up and do not have root access to it).

When you file a feature request, like you did here, there are two ways a software development company can go about it. Most companies will respond with something to the tune of "Thank you for your feedback, we will forward your request to our developers". We think this is nonsense, both because it's just a polite dismissal and because it wouldn't make sense to say that when you are already talking to the developers directly – we do all of the support for our software ourselves.

When you make a request for a feature in any software, not just ours, there is no guarantee that the feature will be implemented. That's why it's called a feature request and not a custom development contract. All feature requests are evaluated by developers and product managers on whether they make sense at all, whether they are technically feasible and whether implementing them would break something else. The priority of any software development company is to not break core functionality and existing features of their software by implementing a feature request. I explained why what you requested is not immediately possible (due to an existing issue regarding the accuracy of the total backup size under certain conditions) and why implementing your alternative would break core functionality for a number of our existing users, both paying and free. It doesn't make business sense to break core functionality to add a feature that amounts to little beyond window dressing. Moreover, you never presented a coherent use case for the requested feature which makes us think that you are not entirely sure yourself why you need this feature in the first place.

As to reporting the backup size. I already told you in my first reply that the only reason we are not giving you a variable for the total size of the backup is that currently this may not be accurate under a certain combination of configuration options. We already have a roadmap item to address this issue. Because it's a rather rare issue which doesn't affect the core functionality it was deprioritized over far more important, time sensitive concerns such as supporting Joomla 4 and PHP 8 where the core functionality was broken. Once we will have addressed the total backup size accuracy issue we will of course be able to implement a [TOTALSIZE] variable in backup completion emails as well, even though this is something that literally nobody else has requested in the past 10+ years. The reason I can not commit within any degree of certainty that the [TOTALSIZE] variable will be implemented is that the fix for the size accuracy issue it depends on is work not yet done and I cannot promise features I do not know for a fact we can deliver.

As for why I wrote a long second reply, it is because you asked me a technical question: why we report individual file names but not file sizes. In retrospect I should have given you a reply to the tune of "file names are known based on the backup file's base name recorded before the backup start and the total number of parts since only the file extension differs, following a predictable pattern, whereas the individual file size is not and cannot be recorded", before ignoring any further enquiries i.e. what every other company would have done in this case. It's just not how I'm wired to reply to technical questions.

Regarding your demeanour, I would hope that your sarcastic and derisive reply was an emotional overreaction to what is, objectively, a dry technical matter. I understand that you are frustrated but I'd like to make two points. First, I do not appreciate unprofessional replies and uncalled for allegations, especially when they have to do with my professional integrity in providing the services you paid for. Second, I do consider accidentally breaking core functionality of the software to be a critical issue, even when it only affects a few dozen existing clients. At the very least, not breaking core functionality is a far more important business goal than adding a feature that's essentially window dressing and which has only ever been requested by a single person.

I am now closing this ticket and will not accept further enquiries from you regarding reporting file sizes in backup completion emails. Please do consider this your first warning to please keep our conversations civil. Have a great day and a happy new year.

 

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!