Support

Admin Tools

#27134 WAF Blacklist and Bing?

Posted in ‘Admin Tools for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
n/a
PHP version
n/a
Admin Tools version
n/a

Latest post by on Friday, 17 March 2017 18:17 CDT

[email protected]
 Good Evening,

I've been investigating an issue with Yahoo/Bing in which our website isn't being crawled very much. I've been racking my brain trying to uncover the root issue and, just today, I was able to narrow down the date the issue begin to on/around 11/01/2016.

While reviewing my logs for that range I found that one of the changes I made was adding WAF Blacklist for com_users. Before adding the exception I had spoken with support (#26435). Considering I am still crawled by other engines and haven't seen any other issues I can't see how the WAF exception would cause a problem, but I wanted to ask the professionals. Could a WAF exception cause an issue with a bot?

I've attached a screenshot of my WAF for reference.

nicholas
Akeeba Staff
Manager
You had asked us how to completely block the password reminder pages of Joomla. We told you how to do that. Here's the thing: if you have not removed these links from your login modules on your site then the search engines may see and follow them (depending on whether you're using rel="nofollow,noindex" on those links). However, if search engines did trigger WAF you'd see them in the security exceptions log. You don't see anything like that, therefore what you did and what you observe are unrelated.

It would also not make sense that only some of the search engines would get blocked. More so since the default value of "Whitelisted domains" in the Configure WAF page reads
.googlebot.com,.search.msn.com
i.e. it allows BOTH Google AND Bing's bots. So we're really sure that the observed effect has nothing to do with what you did on the site.

I also see that between the last crawl and you filing the support request it's roughly a month. This is too low to decide that your site is not being indexed by search engines. Depending on the traffic of your site and the rate of content change on it the indexing will take place anywhere between a few hours to a few months. My blog gets indexed about once every two months. Our business site is indexed several times a day.

If you doubt that just disable the password reset rules we told you you could use in the previous ticket. Do remember that what you wanted to do really made no sense to begin with so we DID give you a workaround that's not a very good idea. Password security is maintained even in the case of account resets since the attacker would need to know the email address of the victim AND have full access to it. Also, you could always use Two Factor Authentication (TFA) which is built into Joomla! since version 3.2 (I know because I wrote that feature myself) for additional account security. Even if an attacker successfully resets someone's password they'll never be able to log in because they don't have the TFA secret. If they get their hands on the TFA secret they have already hacked your site, making the point of password security moot.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!