Dom,
Yes, it is probably a cache problem. The following explanation should help you to clear it.
Hello and thank you for contacting us about your update issue.
Before we begin please understand that extension updates are discovered, downloaded and installed by Joomla itself, using code that's under the absolute and total control of the Joomla project. Akeeba Ltd can not and does not have any influence on that code. In simple terms, your issue is a Joomla bug which is NOT influenced by anything we have done, currently do or can ever do. We can only offer a workaround and an explanation.
Workaround
IMPORTANT! Please follow all of these steps in the exact order provided below, without skipping any one of them even if you think it is unnecessary. Each step and the order we tell you to follow is the result of dozens of hours poring over Joomla's code, understanding how it works and figuring out the shortest workaround to many weird things Joomla does.
For each one of your sites:
- Go to the backend of your site
- Go to our software's main page e.g. click on Components and then on Akeeba Backup or Admin Tools or Akeeba Ticket System Professional depending on which of our software you have.
- Click on Options in the top right corner. This opens Joomla's com_config component showing the settings for our software.
- Clear your Download ID.
- Click on Save & Close.
- Click on Options in the top right corner again.
- Enter your Download ID.
- Click on Save & Close.
- If you have more than one Professional version of our software installed repeat the process from the beginning until here for each one of: Akeeba Backup Professional, Admin Tools Professional, Akeeba Ticket System Professional.
- Go to Extensions, Manage, Update.
- Click on Clear Cache.
- Go to System, Clear Cache
- From the dropdown select Administrator
- Click on Clear All even if the list is empty. Joomla has a hidden query cache which screws up updates if you don't clear it.
- Go to Extensions, Manage, Update.
- Click on Clear Cache again.
- Click on Find Updates.
- You should now be able to perform the updates to our software
Explanation
Joomla handles the extensions updates on its own, using code inside Joomla itself. This feature was originally written to only support free extensions. Back in Joomla! 3.2 support for paid extensions was added. The following is a technical explanation of how Joomla finds updates and how it determines the download URL when you choose to update a paid extension.
How updates work
Joomla cannot read the Download ID from the component's configuration. Whenever you click on Save & Close in our software's Options page you come back to the main page of our software. There, we check if the Download ID in the Options has changed. If it has, we find the row in the #__update_sites table for our software and add
dlid=THE_DOWNLOAD_ID_YOU_ENTERED
in the extra_query column.
When Joomla tries to find updates it reads the contents of the URL in the location column of the #__update_sites table. This is a static XML file listing all published versions of an extension. Joomla finds the latest version compatible with your Joomla version. Then it checks the latest version with the version currently installed on your site. If the version is different (not necessarily newer, mind you!) it decides there is an update available. At this point it creates a new row in the #__updates table and copies the extra_query column from the #__update_sites table's row to the new row in the #__updates table. At this point Joomla HAS NOT YET determined the download URL for the new version.
When you select an extension and click on the Update button Joomla reads the corresponding row of the #__updates table. It goes ahead and reads the contents of the detailsurl of that row. This is the same XML file it read to determine if there's an update available. Now it reads the downloadurl information from the XML file which contains the download URL for the new version without the Download ID. At this point Joomla combines this download URL with extra_query column of the #__updates table's row to create the real download URL. Moreover, it calls the XXX event to let any plugins in the "installer" group to modify the download URL. It then tries to download the new version's ZIP file from this modified download URL.
Why you get a 403
As you can understand from the explanation above, if the extra_query column in the #__update_sites table is empty or contains an invalid Download ID when Joomla tries to find updates then the download URL will contain no or an invalid Download ID. As a result our site will forbid downloading the update with a 403 Forbidden error message. Joomla suppresses all information except the 403 error code.
The only way to correct this issue is to replace the extra_query in the #__update_sites table, clear all updates, have Joomla find all updates again (so the #__updates rows have the updated extra_query) and then retry to apply the updates.
Why do you need the complicated workaround
This brings us back to Joomla doing some really weird things with updates.
For starters, it does not read the Download ID from the component's Options, it uses the extra_query column in the #__updates table. The only way to work around that and have our component replace the extra_query column in the #__updates table is to clear and re-enter the Download ID which triggers our code that applies the extra_query.
The second weird thing is that the extra_query column appears twice, once in the #__update_sites and once in the #__updates table. Therefore we need to replace the #__updates table records. The only way to do that is through Joomla's extension updater's Clear Cache which empties the #__updates_table.
However, Joomla does something which makes no sense. It caches the database queries! Even though we replaced the extra_query column in #__update_sites and cleared the #__updates table (i.e. the updates cache) Joomla will STILL see the old, non-working updates unless we go to Clear Cache and clear the administrator cache. Even funnier, the database query cache which is part of the "system" cache may not even show up in the Clean Cache page, totally misleading the user! That's why we ask you to blindly click on Clear All on that page.
Now that the queries cache is cleared we might still see updates because Joomla did not, in fact, clear the updates cache (the #__updates table) when we told it to because of the database query cache. That's another thing that makes no sense. Therefore we need to click on Clear Cache in the Update page again to have Joomla actually clear the update cache.
At this point we can now tell Joomla to Find Updates which will trigger the entire process we detailed above, finding the updates and using the correct extra_query. At this point applying updates works fine since we've worked around all the weird things Joomla does when trying to find and apply updates. Lucklily, this is only required once.
Why did that happen
The problem is that the extra_query column in the #__updates table went missing. There are two reasons why this could happen:
1. You clicked on the Refresh Update Sites button in the Updates page. This feature is VERY BUGGY. It silently removes the extra_query column from ALL RECORDS of the #__update_sites table. You get no warning whatsoever.
2. A third party extension triggered the Refresh Update Sites feature or stupidly deleted the extra_query column of the #__updates_sites table.
Neither of these two root causes are under our control, nor can we do anything to stop that from happening.
Can we do anything about it?
There is only one thing we can do: create an "installer" plugin which detects the lack of a Download ID in the download URL of our software, looks into the component's options and applies it just before Joomla tries to install an update.
There were three reasons we originally chose to use the extra_query method instead of an "installer" plugin. For starters, the extra_query method existed before the "installer" plugin method by nearly two years. By the time the "installer" plugin method was available we had ironed out all issues with the extra_query method and it was working very well. If it ain't broken, don't fix it as the saying goes. Second, the "installer" plugin method has a fatal flaw: updates only work if you have published the "installer" plugin and set its access to Public. In any other case the update fails with a 403. We had seen many a user disabling plugins indiscriminately. Disabling the extra_query is not possible since there are no user visible controls to modify it. Third, the Rebuild Update Sites feature is a recent addition (Joomla! 3.8). It did not exist before, meaning that once you got the extra_query in the database nothing would remove it.
We are still wary that moving to an "installer" plugin is a bad idea because the plugin can and will be disabled by many users. However, we found a way to use BOTH the extra_query AND the "installer" method, the latter acting as a backup in case the extra_query is empty. The new combined approach is introduced in Akeeba Backup Professional 6.6.1, Admin Tools Professional 5.3.4 and Akeeba Ticket System 3.0.4. Please note that since the plugin is only included as of the aforementioned versions updating our components from an earlier version may still require the workaround. Once you have installed the aforementioned versions or any future version AND have the relevant installer plugin(s) published and their access set to Public (default state when you install these versions) then any future update will not require the workaround even if something or someone deletes the extra_query from the #__update_sites table.
Dale L. Brackin
Support Specialist
English: native
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!
????
My time zone is
EST (UTC -5) (click here to see my current time in Philadelphia, PA)
