Support

Akeeba Backup for Joomla!

#38940 503 Error After Prod Restore to Dev/Test, New Since J4.x Update

Posted in ‘Akeeba Backup for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
4.2.9
PHP version
8.1.17
Akeeba Backup version
9.5.1

Latest post by hodgie on Thursday, 04 May 2023 18:17 CDT

hodgie

I need some suggestions on where next to look for the source of a 503 error I'm getting on the front end (back end seems fine) after restoring a backup from my production server (shared hosting) to a development server that I fully control.

 

The production server is a InMotionHosting.com shared hosting plan running Apache, fpm-fcgi, PHP 8.1.17, Joomla! 4.2.9, Akeeba Backup Professional for Joomla!™ 9.5.1, and Admin Tools Professional 7.3.2.

 

My development/test server is Fedora 37 running PHP 8.1.18 (cli), Apache, and I used Kickstart Pro 7.2.0 to do the restore.

 

The extensions we use are RegularLabs (many of their suite), Tassos.gr (many of their suite), JCE editor, DJ-megamenu, and NextEndWeb's Smart Slider 3. They've been pretty solid to date, but I don't have a lot of history yet on how everything works with Joomla 4.x.

 

Under Joomla 3.10.x, restoring a production Akeeba backup to this dev/test server always worked flawlessly to refresh the environment. I just recently upgraded the production server from Joomla 3.10.x to Joomla 4.2.9, and since then restores to my dev/test server have never worked. The back end always seems to work fine, but the front end always instantly yields a 503 error.

 

As a result, I'm getting concerned that in the event of a production disaster, a restore to production might not work. I feel unable to update any software on the production machine now (like to Joomla 4.3.0 or 4.3.1), because I haven't yet been able to prove out a successful restore of my Joomla 4.x site to my dev/test server. I don't dare touch production until that happens.

 

On the dev/test server after restore, any time I request what I know to be a valid URL on the front end, I get the 503 error instantly. But if I enter a URL I know to be non-existent, the front end displays the site's header and footer correctly, but also a 404 "requested page not found" error in the main body.

 

When preparing for the Joomla 4.x upgrade, I regularly restored the production Joomla 3.x site to dev/test, ran the Joomla 4 upgrade (including required extension upgrades) on the dev/test server, and everything worked fine, back end and front end. So I sort of doubt Joomla core or extensions or content are the issue, but the previous paragraph makes me wonder a bit. And the production site is working well. I just can't prove out a successful production backup and restore to dev/test.

 

I've read and followed most of the related advice provided in https://www.akeeba.com/documentation/akeeba-backup-joomla/restoration-troubleshooting.html. I've tried lots of combinations and permutations of the following:

  • rerun Admin Tools' .htaccess Maker
  • delete .htaccess
  • use .htaccess.txt from a fresh Joomla install .zip as the content for .htaccess
  • In configuration.php I've ensured: public $cookie_domain = ''; public $cookie_path = ''; public $cache_handler = 'file'; public $caching = '0'; public $session_handler = 'database'; public $live_site is pointed to the correct URL
  • The php.ini file shows memory_limit is 256M, upload_max_filesize is 32M, post_max_size is 32M, and max_execution_time is 30
  • Apache's mod_rewrite seems to be loading (I tested with a custom .htaccess file that forced a rewrite to Google.com)

What I haven't exhaustively done yet, because the combinations and permutations of what to try are nearly infinite, is to add/delete various individual lines in .htaccess and enable/disable various Joomla plugins. But I've done quite a bit of fiddling in this way, with no success. If this is the only thing left for me to try, I could use some targeting help.

 

I'm no expert at knowing and finding all the linux logs squirreled away in various places, but those that I've looked at (after turning on maximum logging) don't tell me much, because the site never seems to respond very far- it seems to bomb immediately with the 503 before it does (or logs) anything else.

 

What perplexes me is that a) this always worked a charm under Joomla 3.x, b) the production and dev/test environments haven't changed except for the production Joomla 4.x upgrade and commensurate plug-in upgrades, c) the back-end seems to work fine on dev/test after restoring the production Joomla 4.x backup, and d) the dev/test header and footer display correctly if I request an invalid URL.

 

So, of course I'm happy to do more legwork here, but I thought I'd report in with all the above and maybe you could help me narrow down the universe of reasonable possibilities, given what I've described above.

 

Honestly, I think it's a .htaccess or a php.ini or a configuration.php issue, but I'm a bit lost beyond the basics discussed in the documentation. Where might I focus my next many hours of frustration and worry?

 

Thanks in advance for any ideas.

nicholas
Akeeba Staff
Manager

HTTP 503 Service Unavailable does not have anything to do with the software running at the application side. It's all about server configuration. This is something you would get if you exceed your allocated web server limits, or if your server cannot run PHP.

I suspect that your server has a maximum number of requests per second (or minute). The backend of Joomla 4 uses multiple AJAX requests to populate the status quick icons (privacy requests, Joomla core updates, extension updates, etc). Even though I've contributed a workaround so that these requests are sent serially (instead of all at once, in previous Joomla versions) it's still possible that the number of requests causes your server to reach its limit, thereby putting your site offline until this limit is restored. If there's any sort of cache (Varnish, Squid, a CDN, …) in front of the site it will cache the 503 page and return it every time you try to access it.

Another reason, as I said, is that your server cannot run PHP. If you are using mod_php (instead of PHP FastCGI, or PHP-FPM) on your server with the default pre-forking MPM in Apache it's possible that you do run out of usable threads on your test server. Use PHP-FPM, set up a pool with pm=dynamic, pm.max_children=10, pm.start_servers=2, pm.spare_servers=5 and set up Apache to use PHP-FPM. This gives you AMPLE room to play; I use a smaller configuration on my Linux VM to test concurrent backups with Akeeba Remote CLI.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

hodgie

Thank you for your advice. I will attempt to implement the Apache + PHP-FPM dev/test server and settings you suggest.

 

Honestly, I still don't understand why my existing dev/test server was able to run our Joomla 4 site without issue, after I updated from Joomla 3.10 on it, but it was not able to run the site after doing a Joomla 4 restore onto it.

 

But you are the expert and I am not, so I will follow your advice. Thank you.

nicholas
Akeeba Staff
Manager

If you had told me that it was happening consistently, on both the frontend and the backend, no matter what, I would have said that it's a .htaccess directive, namely the SetHandler or AddHandler line which tells Apache which PHP version to use. This can be a problem when restoring and definitely something we warn about and give you options in the restoration script to address. However, you said that the backend of the site works just fine and .htaccess files cascade. This narrowed down my troubleshooting to some server configuration issue. But you say you have not touched your server configuration.

Therefore, one of your statements has to be inaccurate — not out of malice, but because at the time you formed it you were still trying to make sense of a confounding situation (it's perfectly normal for our brains to try to fill in the gaps; our brains hate an information void!). You can't have a server where neither the server-side configuration, nor the client-provided server configuration is at fault, without a proxy / balancer / CDN / etc in front of the server, which is throwing an HTTP 503 error. One of those three is at fault; HTTP 503 is an error pertaining to server configuration.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

hodgie

Thank you for the follow up.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!