Automating with Joomla Scheduled Tasks

[Note]Note

This feature is only available on Joomla 4.1.0 and later. Make sure the plugin Task - Admin Tools is published.

Joomla 4.1 and later versions have a feature called Scheduled Tasks. It can be used to automate the scanning of your site with Admin Tools without having to create a CRON job just for the PHP File Change Scanner — or at all, in most cases.

You are responsible for setting up the Scheduled Tasks execution trigger

By default, Joomla Scheduled Tasks will not run at all, ever. Something needs to periodically remind Joomla to check if there are any scheduled tasks to run and, if so, execute them. Joomla provides three triggering methods for its scheduled tasks:

  1. CLI CRON jobs. This is the most reliable triggering method. You set up one CRON job which runs every minute and tells Joomla to look for any pending tasks and execute them. This method is only suitable for hosts with real, CLI CRON jobs.

    You need to set up a CRON job to run the command

    /usr/bin/php /path/to/site/cli/joomla.php scheduler:run --all

    where /usr/bin/php is the full filesystem path to the PHP CLI executable and /path/to/site is the full filesystem path to your site. These items are host-specific. We are not your host, therefore we cannot possibly know what these are; please ask your host, that's what you are paying them for. Likewise for how to set up CRON jobs on your site's server.

    Important! This CRON job must be set up to run every minute of every hour of every day.

  2. “Web Cron” (URL-based CRON job). This is the second most reliable triggering method. You set up a CRON job which accesses a special URL on your site every minute and tells Joomla to look for any pending tasks and execute them. This method is suitable for hosts which only allow URL-based pseudo-CRON, i.e. accessing a URL periodically. We DO NOT recommend using this method with a third party service (such as webcron.org) because it gets very expensive, very fast. If your host does not support real or URL-based CRON jobs you are better off using the other automation methods mentioned earlier in this documentation.

    1. Go to your site's backend, System, Managed, Scheduled Tasks.

    2. Click on the Options button.

    3. Click on the Web Cron tab.

    4. Set the Web Cron option to Enabled.

    5. Click on the Save button on the toolbar.

    6. When the page reloads click on the Web Cron tab again.

    7. Copy the “Webcron Link (Base)” contents. It's a URL similar to https://www.example.com/component/ajax/?plugin=RunSchedulerWebcron&group=system&format=json&hash=f0oB4r.

    8. Create a CRON job to access this URL every minute of every hour of every day.

  3. Lazy Scheduling. This is the least reliable method. Your scheduled tasks are only executed when there is visitor traffic on your site. This means that your scheduled maintenance tasks may not be executed exactly when you have scheduled them to run or, in the case of low traffic or intermittent traffic sites with a large number of tasks, not at all. Furthermore, they may cause resumable tasks like the PHP File Change Scanner to take a very long time to complete or in some cases never complete at all without an error. As a result we DO NOT recommend using Lazy Scheduling for the automation of the PHP File Change Scanner.

    1. Go to your site's backend, System, Managed, Scheduled Tasks.

    2. Click on the Options button.

    3. Click on the Lazy Scheduler tab.

    4. Set the Lazy Scheduler option to Enabled.

    5. Click on the Save & Close button on the toolbar.

You are responsible for setting up one of these trigger methods on your site. Again, bear in mind that Joomla does not have any of these set up by default, meaning that no scheduled tasks will execute.

Unfortunately, we cannot provide any support for setting up the execution of Joomla Scheduled Tasks. It's your responsibility to ensure that Scheduled Tasks do work on your site and you need to understand the limitations of each method, especially Lazy Scheduling. As a result we cannot offer any support for scheduled tasks which have not started at all; start at the “wrong” time; take “too long” to finish; or do start but not finish at all without returning an error. This is all part of how Joomla operates.

Depending on the trigger method, scans may be too slow, or fail to complete

You should also keep in mind that scanning your site is a process which normally takes several minutes of work, even on a relatively small site: Admin Tools has to find all .php files on your site, read each and every one of them and evaluate it for changes and threat score. This cannot be executed in a single page load; it would time out. Instead, we split the work in smaller chunks. When the file scanner scheduled task is executed it will perform a chunk of work. If it needs to perform more work it will tell Joomla that it's not done yet and please execute me again at your earliest convenience. This will continue happening until the scan is complete.

Because of the chunked method of execution, scanning a site will need between a few to several hundred scheduled task execution triggers. When using the CLI CRON job or “Web Cron” these triggers happen once a minute which is much longer than once every 5 to 7 seconds when you run a scan from the backend of your site. As a result your scan will very likely take much longer to complete, even tens of times longer, than a scan made through the backend of your site or through one of the other scan automation methods. We do not provide any support about scans which take “too long” besides what is written in this documentation; it's how Joomla works.

When using the Lazy Scheduling the scheduled task triggers only happen when there is enough traffic on your site meaning that your site scanning might not complete in a reasonable amount of time or at all; or it might not even start at all. As a result we offer no support whatsoever for using the Lazy Scheduling triggering method for Joomla Scheduled Tasks. If it works for you, that's great news. If it doesn't work well or at all, sorry, we told you so and there's nothing we can do, it's how this core Joomla feature works.

Preparing Scheduled tasks

Make sure that you are using Joomla 4.1 or later. Scheduled Tasks were first introduced in Joomla 4.1; older versions do not have that feature.

Go to your site's administrator backend, System, Manage column, Scheduled Tasks.

Click on Options.

Set the Task Timeout to 300 — that's the Joomla default value. If your server has another time limit that's lower than this setting, e.g. a maximum CPU usage time limit, you will need to lower this setting to that value. If you are unsure you can ask your host. Click on Save.

If you are going to use Lazy Scheduling (NOT RECOMMENDED, NOT SUPPORTED) you need to go to the Lazy Scheduler tab and set Lazy Scheduler to Enabled. Set Request Interval (seconds) to 60. Then click on Save & Close.

If you are going to use a URL-based CRON jobto trigger Joomla Scheduled Tasks you need to go to the Web Cron tab and set Web Cron to Enabled. Then click on Save. Copy the “Webcron Link (Base)”. You will have to set up a CRON job to access this URL every minute. If you are unsure how to do that, please ask your host. Kindly note that the information we told you to get from your host is host-specific, we do not know it and cannot help you with it because we are not your host.

If you are going to use a CLI-based CRON job to trigger Joomla Scheduled Tasks you need to run Joomla's cli/joomla.php file with the command line parameters scheduler:run --all . For example:

/usr/local/bin/php /path/to/your/site/cli/joomla.php scheduler:run --all 1>/dev/null 2>/dev/null

Where /usr/local/bin/php is the path to the PHP CLI executable on your server and /path/to/your/site is the absolute path to your site's web root directory. Both of these pieces of information as well as instructions for setting up a CRON job can be provided by your host. Kindly note that the information we told you to get from your host is host-specific, we do not know it and cannot help you with it because we are not your host.

Setting up a scheduled task

First, make sure that the Task - Admin Tools plugin is installed and enabled on your site and that you are using Joomla 4.1 or later.

Go to your site's administrator backend, System, Manage column, Scheduled Tasks.

Click on New in the toolbar.

Select “Admin Tools – PHP File Change Scanner ”

There are two configuration options specific to this task under “Task Parameters”. Both are used to override the maximum execution time. First, let's talk about it a little bit. By default, when these options are set to 0, the scheduled task will execute a scan step for the “Maximum Work Time” set up in Components, Admin Tools for Joomla!, Control Panel, Options, File Scanner tab. This is typically small, by default 5 seconds. This makes sense because the scan is meant to be run from the backend of the site and we want many small steps. This is not a problem for a backend scan since each scan step starts a few hundred milliseconds after the last scan step completed. There is not a lot of time lost there and the scan completes as fast as possible.

As we said earlier, when using Joomla Scheduled Tasks the time between steps is substantially longer. When using CLI or URL based CRON jobs the task is triggered once every minute. With a Maximum Work Time of 5 seconds it means that the scanner works for 3 to 5 seconds, then nothing happens for 55 to 57 seconds. Rinse and repeat until the scan is done. If you do the math, that's an 8.33% duty cycle. This means that if scanning your site through the backend takes about 8 minutes, doing the same through a Joomla Scheduled Task would take 1 hour and 36 minutes. That's insane, right?

It gets worse with Lazy Scheduling because the scheduled tasks will be trigger at most one every minute, depending on traffic. If there is not a lot of traffic they might be triggered once every 10 minutes on average. As a result the aforementioned site scan which takes 8 minutes through the backend of your site will now be taking 16 hours to complete. This makes it completely impractical and that's assuming that you get an average of one visitor every 10 minutes which translates to at least 144 page loads evenly spaced out over the day or, more realistically, at least 2000 page loads a day.

The only way to improve this situation is to increase the duty cycle, i.e. make more work in each scan step. However, you cannot go and change the component-wide Maximum Work Time option; setting it very high (e.g. 60 seconds) would most likely result in the backend scans failing. Hence the override options.

In the examples above, setting the maximum execution time override to 50 seconds would allow the scan to complete in around 9 minutes for the CLI and URL trigger methods; and somewhere between 9 and 90 minutes with the Lazy Scheduling triggering method. While not ideal it's not too bad either; it becomes a realistic scheduling solution.

There is a reason we have two different override options. Most hosts apply different time limits for CLI scripts (CLI CRON jobs) and for scripts running on the web (URL based CRON and Lazy Scheduling). It is possible, though not recommended, to enable all three triggering methods for Joomla scheduled tasks. As a result we need two different overrides to make sure that the correct one will be used depending on where the task is being triggered from.

Finally note that the overrides can be automatically capped at runtime. If they are greater than the Task Timeout you set up in the Joomla Scheduled Tasks' Options page the override will be reduced to the Task Timeout. If the override is higher than the PHP timeout it will be reduced to the PHP timeout. This ensures that an override that's a bit too much isn't as likely to cause the scan to fail.

Scans can still fail if you set a maximum execution time override which is higher than the CPU maximum usage time per process enforced by your host. Typically this is in the 30 to 600 seconds range. Unfortunately, there is no good way to determine that which works across all hosts. If you see your scans failing without an error (the task appears as hung) contact your host and ask them what is the CPU maximum usage time per process (if they have no idea what you're talking about, please ask them to escalate the ticket to an engineer and ask them what's the ulimit -t for your user account; that's the proper jargon to tell the engineer EXACTLY what you need). If your host replies there's no such limit set the override to 30 and keep in mind that your scans automated with Joomla Scheduled Tasks will last at least twice as long as the backend scans.

Maximum execution time (CLI)

This applies only for tasks executed with the CLI CRON job triggering method. Set to 0 to use the Maximum Work Time set in the component options. Non-zero options are used instead of the Maximum Work Time as long as they are greater than the Maximum Work Time. We recommend setting this to 300. Set this to 50, 30 or even 10 if your scans appear to be hung (but they will become slower).

Maximum execution time (web)

This applies only for tasks executed with the URL CRON job and Lazy Scheduling triggering methods. Set to 0 to use the Maximum Work Time set in the component options. Non-zero options are used instead of the Maximum Work Time as long as they are greater than the Maximum Work Time. We recommend setting this to 50. Decrease to 30 or even 10 if your scans appear to be hung (but they will become slower).