Support

Admin Tools

#20084 How long does php file scanner usually take?

Posted in ‘Admin Tools for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Admin Tools version
n/a

Latest post by on Saturday, 21 June 2014 18:00 CDT

Wednesday, 14 May 2014 11:33 CDT

user73146
 How long does this PHP File Scanner usually take?
I left scanner running by about 18:00 and it apparently finished 04:01:08 the next day, is this usual?

Is there php settings i should change to improve the speed of scan, i have wordpress site included with joomla, but i excluded these folders, but it seems to take forever, i can remember running much older version before and it never took this long.

Tried FF, then IE because it was mentioned FF was problematic when used for PFS, compared to IE which i find VERY VERY hard to believe, as I usually find its the other way round, saying well there is IE, BUT! there are better browser, which ones any of them.

Thursday, 15 May 2014 02:18 CDT

tampe125
Hello Gustavo,

it depends by the size of your site. Moreover did you enabled the option to calculate file diff?

Davide Tampellini

Developer and Support Staff

🇮🇹Italian: native 🇬🇧English: good • 🕐 My time zone is Europe / Rome (UTC +1)
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 15 May 2014 06:35 CDT

user73146
Total size of all files 1.48GB
size of Wordpress directories: (excluded from search i might add, at moment)
600mb

So should be about 800mb for Joomla site, which would mainly image, pdf files, very few flash files as we have our flash movies linked from brightcove, which should not be a problem as i thought it would as the name suggests 'php file scanner' scan php files only (and inc apparently).

and No! i have not enabled the option to calculate file difference.

Thursday, 15 May 2014 10:17 CDT

tampe125
Can you please post here the maximum and minimum execution time of the scanner?

Davide Tampellini

Developer and Support Staff

🇮🇹Italian: native 🇬🇧English: good • 🕐 My time zone is Europe / Rome (UTC +1)
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 16 May 2014 06:39 CDT

user73146
As suggested here https://www.akeebabackup.com/support/admin-tools/15075-php-file-change-scanner-not-working.html

Minimum execution time: 1 second
Maximum execution time: 5 seconds
Runtime bias: 75%

Edit:
files scanned with wordpress: 5676
files scanned excluding wordpress: 3152

I presume if it shows, that it actually finished the scan completely, and not to the point it was stopped prematurely for whatever reason.
Edited by user73146 on 2014-05-16 06:46 CDT

Friday, 16 May 2014 10:22 CDT

nicholas
These are too few files and too much time. Scanning 5676 files in 10 hours (600 minutes) gives us 9.46 files per minute, or 6 seconds per file. That's WAY too much time. The usual speed is 2000 to 4000 files per minute, depending on your server speed. From there to less than 10 files per minute it's a huge difference. The only thing I can think of is the setting of the "File types to be scanned". Normally it should be:
php
phps
php3
inc

If you've added other file types, e.g. PDF, with enormous files then there's a chance that the scan will be very slow. Of course in none of my tests was any scan that slow.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 16 May 2014 10:56 CDT

user73146
But i have not added pdf or any other file type for scanning and why would I? its the server languages files I.E php files, i want scanned, all i have added is php5 to that list, as we had upgraded to 5.4.xx,

Friday, 16 May 2014 11:30 CDT

nicholas
I frankly can't see any reason why the scanner is so slow on your site. I just run one more local test on a very large development site. 7262 files scanned in 1 minute 27 seconds.

Is it possible that you are running some antivirus / Internet security software which makes AJAX requests unbearably slow? I've seen something similar with Akeeba Backup when some users had been using certain antivirus products (AVAST and AVG to the best of my knowledge).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Friday, 16 May 2014 12:17 CDT

user73146
I never had problem with slow ajax, but to clear the number of files mentioned before IS number of files the php scanner had scanned and reported.

The approx number of all files excluding wordpress files is about

34,565

we are using jotcache which does produce large php cache files someimes, but as it was stated, PFS ignores these folders by default, and i think i added them (well just main cache folder not subfolders), to exclude listing as well.
I have noticed that two files are created when pfs is running in tmp directory where one is a tmp file, which i thought would pass details to main file to it would gradually increase in size, but they go up and down in size, so its not a gradual increase ( and no i dont do this everytime pfs runs, this was one off to see if it actually doing anything).

Saturday, 17 May 2014 03:22 CDT

nicholas
the number of files mentioned before IS number of files the php scanner had scanned and reported.


Yes, I understood that. The number of files I reported (7262) is the total number of PHP files scanned on my dev site. The total number of files is nearly 20000.

The only thing that comes to mind is that you have one or more directories with several hundreds or thousands of files. This might cause bad performance, but still not 10 hours to scan a site. On top of that, you already told me that you excluded the directories with these files:
we are using jotcache which does produce large php cache files someimes, but as it was stated, PFS ignores these folders by default, and i think i added them (well just main cache folder not subfolders), to exclude listing as well.


I have noticed that two files are created when pfs is running in tmp directory where one is a tmp file, which i thought would pass details to main file to it would gradually increase in size, but they go up and down in size, so its not a gradual increase ( and no i dont do this everytime pfs runs, this was one off to see if it actually doing anything).


Is it a PHP file? I mean, does it have an extension which would cause the PHP File Change Scanner to scan it? If not, this is irrelevant. If it is a huge file and it is being scanned I would expect the scanner to hang, not stall for 8 hours.

I have an idea. The scanner takes a very long time when you're using the web to run it. Can you please try using a CRON job to run it and tell me how long it takes?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 19 May 2014 11:10 CDT

user73146
Success! less than 2min, thought it was 1hr 2mns, but turns out the server time is 1 hour behind.

So this great! and just what we want, thanks for the help, i can add the wordpress directories now, shuu! had me worried for moment, but finally got there, again thanks.

Monday, 19 May 2014 11:24 CDT

nicholas
Cheers!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 20 May 2014 06:52 CDT

user73146
Just a addition, now we know it does work, is there a way to run PFS without going through cron in case i want a instant report? if it worked through backend web interface this would not be a problem.

Also when running scan through joomla backend what i did notice that when i first open PFS, it seem to show multiple pagination pages (23+), but no new report, i did a refresh and multiple pagination pages disappeared.

Through joomla backend
The first completed scan id ref is 655
The second completed scan id ref is 4174
(this is a big jump)
since cron job
5102
5103
5104

This seems to tell me that it is not completely finishing a scan and loops again and again, and given a new id ref each time on every loop.
Edited by user73146 on 2014-05-20 07:05 CDT

Tuesday, 20 May 2014 10:26 CDT

nicholas
if it worked through backend web interface this would not be a problem.


Yes, normally it DOES run through the back-end interface. The CRON interface is supposed to be used only for automation.

This seems to tell me that it is not completely finishing a scan and loops again and again, and given a new id ref each time on every loop.


It does sound like this is indeed happening. Can you please try disabling automatic cleaning of your site's tmp directory? I have a strong suspicion that this is the problem. PHP File Scanner stores a file with its internal state in your site's temporary directory when it's running through the back-end of your site. If between two page loads this file is removed a new scan will begin.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Tuesday, 20 May 2014 12:01 CDT

user73146
IF you mean the admin tools plugin 'Enable Temp-directory Cleaning' i have not enabled it, i have not enabled any of them, cache is taken care off by jotcache, and it does not touch tmp directory.

Else if you mean some other feature in joomla or php setting you will have to explain more.

Tuesday, 20 May 2014 16:21 CDT

nicholas
I was thinking about Admin Tools' feature. That said, I have seen some servers automatically clearing the temporary directory. That's why Akeeba Backup no longer uses Joomla!'s temporary directory for storage (Admin Tools' PHP File Scanner uses the same engine as Akeeba Backup, with the difference that each file is scanned instead of being added to a backup archive).

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Thursday, 22 May 2014 06:37 CDT

user73146
Not for joomla tmp directory surely? i did check max_execution_limit and it was set to 60 sec, i changed it to 300, but i discovered the filescanner php reset it to unlimited anyhow, so that idea went out the window.

Is there any other php ssettings i should consider looking at?

Thursday, 22 May 2014 09:37 CDT

nicholas
Not for joomla tmp directory surely?


As a matter of fact I HAVE seen hosts clear that exact directory on a schedule. Old versions of Akeeba Backup (pre-3.4) were using Joomla!'s temporary directory as the place to store temporary files – makes sense, right? However due to this problem of vanishing files I had to move the location of the temporary files somewhere else.

As for the reasons of the disappearing files, it's something the server does (empty all tmp directories), something a plugin does (periodically clear the Joomla! temporary directory) or something the user does with a CRON job.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Saturday, 21 June 2014 18:00 CDT

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!