Support

Admin Tools

#9869 Is user agent blocking a resource hog?

Posted in ‘Admin Tools for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Admin Tools version
n/a

Latest post by nicholas on Monday, 09 May 2011 11:22 CDT

Saturday, 07 May 2011 13:39 CDT

user33310
I have yet to enable the user agent blocking feature in .htaccess maker, but am considering it after finding the following guide on the matter:

http://www.htaccess-guide.com/blocking-offline-browsers-and-bad-bots/

The list of agents in the htaccess-guide page above is considerably longer than what's in .htaccess maker. I trust Akeeba's work more than some unknown guide on the web, but I would like to inquire about the potential performance hit of enabling the blocking of such a long list of user agents.

Thanks for responses from people that have had experience with this particular feature of Apache.

Sunday, 08 May 2011 18:37 CDT

nicholas
The more you add to the .htaccess, the slower things get. How slower? It depends on the server and the sites it serves (not just your site), as well as how many requests per second hit the server. The practical slowdown is negligible unless you're talking about a site with dozens of millions of pageviews per month.

If you have that kind of site, just take the .htaccess directives, put them in your vhost config, set AllowOverrides None and you're good. The performance impact will be virtually zero, as everything is cached in memory instead of being loaded off the disk all the time. If this sounds like Chinese, you really don't have to worry about the performance impact of your .htaccess rules. No joking meant here; in order to run a site like that you need to have great experience in setting up and configuring web server, so you should already have known about this stuff. That said, even if you have this kind of beast of a site, you can still not worry about .htaccess performance by putting the site entirely behind a CDN.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Sunday, 08 May 2011 19:04 CDT

slaes
For those who don't know, it may also be worth mentioning that although blocking user agents is good and relevant for many objectives, eg saving bandwidth among others, most good malicious scripts identify them-self as Firefox or Chrome or whatever, so this type of blocking would not stop access.

So bottom line, using user agent alone is almost impossible to detect the difference between, humans, malicious scripts, data scrapers, bots etc. In the past past i have played with some interesting things, relating to loading certain js which i knew 99% of scripts would not be able to do, and bang i knew they were not human. However they are very intelligent now days and its a losing battle, and hardly worth the time. My 2 cents.

Sunday, 08 May 2011 19:06 CDT

nicholas
I agree with that. The feature is mostly added as a way to prevent resource hogging scripts from reaching your site, as well as to stop the very common and badly (read: horridly) coded scripts most script kiddies use.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 09 May 2011 09:44 CDT

user33310
Thanks for the responses.

I am going to enable the feature with just the list included in AT Pro. If it doesn't hurt performance I may add more.

I'm trying to develop a standard "platform" on which I build sites to maximize security while keeping things running smoothly.

Monday, 09 May 2011 10:56 CDT

nicholas
Yes, working on a "master" set of rules is a good idea. If you want to take the idea further, you can create a "master" Joomla! site, configure Admin Tools to your liking, install and keep all of your favourite extensions up-to-date and use it to spin off new sites using Akeeba Backup Professional. From what I've heard from fellow J and Beyond attendees that is the professionals' preferred method of creating new sites :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Monday, 09 May 2011 11:12 CDT

user33310
That's exactly what I've been doing!

To borrow a term from my VMware days, I create a Gold Image to deploy from.

Also, I'm not sure if Akeeba users are really exploiting cloud storage but I sure am. A single Amazon S3 account can handle an unbelievable amount of data for a really low cost. I keep at least 5 backups of every site I host or maintain, plus backups of all my personal and development sites. I'm not running any VM's there, yet. But, my storage and bandwidth costs alone at S3 are averaging about a dime per month. That's not a typo - 10 cents per month! Better than any tape drive or VTL.

Monday, 09 May 2011 11:14 CDT

user33310
Hey, Nicholas. I thought this was a "No Support" period on the calendar. :) Can't stay away, eh?

Monday, 09 May 2011 11:22 CDT

nicholas
Yes, similar setup here. Multiple copies of multiple backups for a fraction of the cost of downloading and keeping them on a local hard disk. Cloud storage has certainly changed the way we even think about storage these days.

Yes, today is a no support day, but I'm stuck for 7 hours at the Schiphol airport in Amsterdam waiting for my flight back home, so I thought I might do some work instead of boring myself to death ;) You're right, I can't stay off the job for too long. Deep down, it seems I am a masochist - or, at least, a workaholic :D

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!