Akeeba Backup for WordPress 8 is a major rewrite over version 7. This will let us maintain the software more efficiently in the future. However, we hit some issues with WordPress itself which caused a lot of issues with lost backup profiles and errors during plugin updates.
This was rewritten on October 4th, 2023.
Lost backup profiles
What happened: Your backup profiles appear to have reset to defaults and/or you are asked to run the Configuration Wizard.
Why it happened: We have moved the backup engine from wp-content/plugins/akeebabackupwp/app/Solo/engine to wp-content/plugins/akeebabackupwp/app/vendor/akeeba/engine. The profile settings are encrypted in the database with a key which was stored in wp-content/plugins/akeebabackupwp/app/Solo/engine/secretkey.php. This file needs to be now moved to wp-content/plugins/akeebabackupwp/app/Solo/secretkey.php, one directory up. We had already written post-upgrade code which does that. Because of a WordPress issue, this code did not run if you used WordPress' plugins updater. As a result, a new file with a new key was created. Therefore, your backup profiles cannot be decrypted and they are reset to factory defaults. This only happens if the update took place using WordPress' plugins updater (not the updater in the plugin itself), you did NOT visit the plugin's main page after the update, and you tried to take a backup through a CLI script, WP-CLI, the remote JSON API, or the remote backup URL, or WordPress looked for plugin updates. If you visited the plugin's main page after installing the update, or if you used the update method provided inside the plugin itself this was not a problem: the post-upgrade code ran and everything was working fine.
What to do: If it's an option, restore your site to a backup taken before installing version 8.0.0, 8.0.0.1, or 8.0.0.2. That's the easiest way and why you have backups, to restore your site to working order if something breaks.
If this is not an option, restore the last backup taken before installing version 8.0.0, 8.0.0.1, or 8.0.0.2 (a backup taken before Friday, September 29th will do) on a temporary subdomain. Then do the following:
- Delete the
wp-content/plugins/akeebabackupwp/app/Solofolder from your live site. - Copy the
wp-content/plugins/akeebabackupwpfolder from the temporary site to the live site. - Copy the
wp_ak_profilesdatabase table from the temporary site's database to the live site's database (note that thewp_prefix may be different on your site).
Error while installing updates (of any plugin)
What happens: You would get a PHP Error “ InvalidArgumentCount”.
Why it happens: This is a WordPress bug, introduced in WordPress 6.3 (Update October 14th, 2023: it is now fixed in WordPress 6.3.2). The changed the way they call the upgrader_process_complete hook so now it is passed only one argument. Before it would take two arguments. At the time of this writing, WordPress' official documentation is still wrong, showing that the hook handler should expect two arguments. All plugins written as per the WordPress documentation and which were working perfectly fine in previous versions of WordPress are affected, but only on sites running on PHP 8. On PHP 7, passing one argument to a hook handler which expects two arguments is merely a PHP Warning — and the post-upgrade code does not run. On PHP 8 it's an ErrorException (PHP stops executing).
What to do: Go to Akeeba Backup, System Configuration, Update tab. Set “Show Akeeba Backup for WordPress update in the Plugins page” to No.
Why was this not caught before release
We do test our software before release. In this case, we were continuously testing it for 3 weeks prior to the release. However, we can only test for things that we know, reasonably expect, or just run into. We cannot test for what we don't know, cannot reasonably suspect might be a problem, and have not run into. That's exactly what happened here: this was a problem that occurred only when a specific sequence of events took place which were not in our test plan.
Namely, the issue only occurred if you used WordPress' plugins updater to update the plugin (instead of the updater included in the plugin itself), or installed the new version using the Plugins, Add New page. Moreover, after the update you had to NOT visit the plugin's main page and one of the following take place:
- An automatic update tried to ran through WP-CRON, WP-CLI, our CLI scripts, the remote JSON API, or the legacy backup URL.
- WordPress tried to look for plugin updates and the “Show Akeeba Backup for WordPress update in the Plugins page” option was set to Yes.
While we do test updates methods, there were two problems in our methodology:
- After installing an update, we would visit (in both manual and automated tests) the plugin's main page to ensure that the plugin actually loads its interface. This had the unintended consequence of masking WordPress' failure to execute the post-upgrade code.
- The update installation did not use the WordPress' plugins updater because it is very convoluted to get it to reliably look for updates. WordPress uses a number of serialised data entries in the
wp_optionstable which cannot be manipulated outside of WordPress (therefore making automated testing fail most times), and lacks a method to forcibly refresh the updates. Therefore, we tested the plugins updater only once (which did fail, but we could not reproduce this issue until after we made a public release), with subsequent tests taking place only by using our own updater, and a direct installation of the plugin. The bad assumption we made is that all update methods are equivalent since we had already put in place theupgrader_process_completehook handler, therefore WordPress should surely run the post-upgrade code. This was not the case.
This testing methodology had not failed us in the past, but now we see why it did fail.
The first problem is that WordPress does not really have a way to run post-upgrade code. It relies on the upgrader_process_complete hook handler. Unfortunately, that's the same hook handler which has an uncharacteristically backwards incompatible and undocumented change in WordPress 6.3, causing it to crash, therefore not run the post-upgrade code.
The second problem is that by having our tests visit the plugin immediately after the update we were masking this problem. Our plugin checks if the post-upgrade code has run and, if not, tries to run when you visit its main page.
In retrospect, it's very easy to say that had our test process not create an accidental happy path we would have caught the problem. However, we had no reason to suspect the testing method was wrong in the past, nor had we ever had any reason to believe that WordPress would make such a major backwards incompatible change without announcing it, and without giving a way to developers to address it.
Why we did not send an email about it
Simply put, it's illegal.
When you subscribe we do not collect your explicit consent to be contacted over email. In fact, the consent you give is limited to transactional emails (you did something on the site which has to send you an email as a result, e.g. reset your password, expect a ticket reply, etc), matters concerning your subscription, and as a direct result of you contacting us. As per the EU GDPR it would be illegal for us to email you about the problems we found out in Akeeba Backup for WordPress 8.0.0.
Why we do not have realtime social media channels such as Discord, Slack, etc.
Because it's only two of us.
Between software development, support, and running the business there is no time to manage social media channels. We have a Facebook page. I did not have time to update it. These past four days I've put in 6 hours of sleep in total.
Why we don't have beta versions
Because they did not work for us for a decade.
We did have beta versions between 2010 and 2020. That's why there's still the “Minimum stability” option in the System Configuration, Update tab of Akeeba Backup for WordPress.
When we did have beta versions very few people installed them. Out of those people, we never got any feedback during the beta period. We only got feedback after releasing a stable version to the tune of “Hey, I had seen such and such bug during the beta. I thought you'd fix it before the stable, but it is still there”. This was supremely unhelpful. That's no different than just skipping the beta stage altogether and get releases out faster. So, beta versions were discontinued.
If you want to argue, do remember that there are developer releases regularly published on our site, and they do serve the same purpose. Nobody stops anyone from using them, testing them, and submitting feedback.
Why didn't we tell you in advance about the risks (sort of)
We do publish release notes for each release. They are linked from the main downloads page, right under the button to download each version. The release notes for 8.0.0 did say that it's a major refactor.
As for why we did not tell you that the profiles would be lost: because we had written post-upgrade code to make sure this does not happen.
Don't be mistaken into believing this is the first and only time we've done a major refactor of any of our software, or Akeeba Backup for WordPress specifically. There is an average of one major change in each one of our software every 14 months. Just download Akeeba Backup for WordPress 1.0.0 and compare it with 7.9.2. See how much has changed. There is a different backup engine, ALICE checks structure, view names, template files technology, CSS framework, JavaScript; it's like the Ship of Theseus. You just never knew any of that happened. The post-upgrade code made sure of that. You only know about this one case because the post-upgrade code did not run when using WordPress' plugins updater. Do not think that what you didn't notice has not happened.
Why didn't we pull the release (we actually did)
When we received the first reports about the issues with 8.0.0 on Sunday morning we pulled it.
We did write a quick workaround which allowed sites migrating directly from 7.9.2 to 8.0.0.1 to work without losing the backup profiles. We even released a version 8.0.0.2 to address the hook issue.
The problem is that at this point a lot of sites had already upgraded over the weekend automatically, either with WordPress' automatic updates or through a third party service, and people had not noticed (because it was over the weekend). At this point, their backup profiles were already reset. So, naturally, installing 8.0.0.1 or 8.0.0.2 made no difference at best.
Even worse, today we found out that if they had already reconfigured the backup profiles after installing 8.0.0, updating to 8.0.0.1 or 8.0.0.2 (which moved the old encryption key to the correct location, overwriting a previously regenerated key) had the effect of corrupting the reconfigured backup profiles. This does make sense, we just had gotten the impression from the feedback we got that people had not reconfigured their backup profiles just yet. So, there goes that idea for a fixed version…
Therefore, we decided to completely remove 8.0.0.1 and 8.0.0.2 and update this page with all the information so far and address all your questions.
What we will be doing
First of all, any inquiries about Akeeba Backup for WordPress version 8 will be redirected here with no further comment. We don't do that because we don't want to talk to you, but because we actually need to spend our time implementing the next steps instead of going through the same information which is already on this page.
From a development point of view, we start by discontinuing support for WordPress' plugins updater. You will only be able to update Akeeba Backup through its own updater, the one we can control and test. The fact that WordPress' post-upgrade capability hinges on a plugin hook which has to be present in the old version of the plugin is problematic because it essentially asks developers to be fortune tellers. For example, For example, migrating from Akeeba Backup for WordPress 7.0 – 7.7 to Akeeba Backup 8 would not run the post-upgrade code because those versions did not have a post-upgrade hook; they didn't need one. It was only added in version 7.8.0. And yet, there is no way to prevent them from doing that, or even warn them. That's… not good!
Moreover, the fact that they broke it without warning or even documentation update means that it cannot be trusted for mission-critical migrations, like the one we are dealing with here. If it cannot be trusted, it will not be used, as simple as that.
As a bonus point, on hosts with PHP's OPcache enabled the update would also cause the plugin to fail to load because WordPress does NOT reset the OPcache for the newly written files. This is an issue that we had also discovered in Joomla. It's fixed in Joomla, it's not fixed in WordPress. It is also addressed, you guessed it, in our updater. This is yet another reason to decide to only go through our updater.
We still have the problem that installing the new version from the Plugins, Add New page does not run the post-upgrade code anyway. We are already storing the last version of Akeeba Backup where we ran the post-upgrade code in the database. We will use that to try and run the post-upgrade code. If that fails, we will prevent any further automated use of Akeeba Backup until this post-upgrade code does run, which means either using our own updater, or visiting the main page of the plugin.
To do that, we need to refactor the code even deeper than we did for version 8.0.0. WordPress has no less than NINE (9) different ways it can initialise and call our plugin. This leads to a lot of discrepancies on what is taking place. We are refactoring the code to tidy up the initialisation into a single, unified block of code which will execute no matter how WordPress decides to initialise and execute our plugin. This will let us implement the check described in the previous paragraph consistently.
There is the problem with the sites which had updated to 8.0.0, 8.0.0.1, and 8.0.0.2, or which put Akeeba Backup 7.9.2 on top of an 8.0.0, 8.0.0.1, or 8.0.0.2 installation. These sites may have two secretkey.php files in two different places: app/Solo and app/Solo/engine under the plugin's folder. Which one is the right file to use? The best we can do is try to decode the default backup profile's settings (#1) with each of these key files separately. The key which does not result in an error will be kept. If both keys result in an error, we will keep the new file (app/Solo/secretkey.php). If no key file is present, or the encryption is disabled, we will do no key migration obviously.
From a testing perspective, we have updated our testing methodology to now try and take an automatic backup right after update, before visiting the plugin's main page. This should always lead to the update code running as part of the automatic backup and we have ways to check that this is the case (remember what I mentioned above about putting a version marker in the database?).
We will also publish a beta for the upcoming version 8.1.0. Given the number of people who were shouting “where was the beta for 8.0.0” we expect to have a massive response to it. Right? I guess we'll find out.