Support

Admin Tools

#13049 Sudden increase in Google crawl errors coincident with AT installation

Posted in ‘Admin Tools for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Admin Tools version
n/a

Latest post by nicholas on Thursday, 26 July 2012 12:35 CDT

user7806
Mandatory information about my setup:

Have I read the related troubleshooter articles above before posting (which pages?)? Yes
Have I searched the tickets before posting? Yes
Have I read the documentation before posting (which pages?)? Yes
Joomla! version: 1.5.26
PHP version: 5.2.9
MySQL version: 5.0.41
Host: www.oldottawsouth.ca
Admin Tools version: 2.2.10

Description of my issue:
I installed Admin Tools about 10 days ago. Since then I have gotten a couple of warnings from Google Webmaster Tools about "Increase in authorization permission errors". When I log in to Webmaster Tools and look at the data for Access Denied (403) errors, I see the error rate was consistent at zero per day until the day after I installed Admin Tools, at which point it jumps to a consistent 136 errors per day.

The vast majority of the failed URLs listed are of this form:
http://www.oldottawasouth.ca/index.php/component/user/?task=login&323ed0194f9c59eed6070bc7bf56ccfc=1

(but with unique strings after "task=login&")

This URL corresponds to the "Login" link in the top-right corner of my site. Is there any reason why Admin Tools would be causing an access denied error here?

(For what it's worth, it doesn't seem useful for Google to crawl that link. Still, it has been there for a long time and the errors have only just started now that Admin Tools is active.)

Thanks, Chris.

nicholas
Akeeba Staff
Manager
First go to Admin Tools, Web Application Firewall, Configure WAF. Make sure "Log security exceptions" is set to Yes; if it's not, set it to Yes and click on Save, then wait for a new Google crawl. Now please go to Admin Tools, Web Application Firewall, Security Exceptions Log and look for the URLs Google reports as blocked. Please copy the Reason and Target URL here so that I can help you.

Regarding your concerns, yes, Google should not crawl that URL. In fact, the login and logout URLs should have rel="nofollow" appended to them. They are not indexable pages. Even though I can help you turn off the security feature which may be blocking them it's not a good idea. The best way to deal with this is adding the nofollow rel attribute to those links, or use a login form instead of a login link.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user7806
I will see what I can do about adding nofollow to the login link. The link is being generated by a module included with my JoomlArt site template, but with some digging I should be able to find the code.

As for the logs, I did already have WAF set up to log exceptions. I can't find anything in there from the google crawler, though. There are entries with similar-looking URLs, but I believe they are the result of people actually attempting to log in, not just following the link to the login form. They are listed in the AT log as "login failure", and the IPs are Chinese, Russian, etc.. -- not googlebot.

nicholas
Akeeba Staff
Manager
Since you don't get a log entry in WAF it has to be something being blocked by the .htaccess file generated by .htaccess Maker. If you're using the .htaccess Maker feature please try setting the "Disable PHP Easter Eggs" option to No and then click on Save and Create .htaccess. Normally this should solve this issue.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

user7806
Actually, I'm not using the .htaccess maker.
I did manage to add the nofollow parameter to the login link, so hopefully that will solve the immediate problem.

nicholas
Akeeba Staff
Manager
If you're not using the .htaccess Maker and you don't see any log entries for those crawl errors then they are not related to Admin Tools. All access attempts blocked by Admin Tools' WAF result in a log message.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!