Support

Admin Tools for WordPress

#40282 development website protection

Posted in ‘Admin Tools for WordPress’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

WordPress version
n/a
PHP version
n/a
Admin Tools version
n/a

Latest post by nicholas on Wednesday, 13 March 2024 08:35 CDT

houba_houbi

Hello

For my developement websites I used a protection against robots indexation like this :

AuthUserFile ".../.htpasswd"
AuthName "ACCES PRIVE"
AuthType Basic
require valid-user
RewriteEngine On
RewriteRule \.htpasswd$ - [F,L]

So now with admintool where may I put my rules please ? In Custom .htaccess rules at the top or bottom ? Thanks

nicholas
Akeeba Staff
Manager

You only need the first four line. You can put them in the .htaccess Maker's top rules.

IMPORTANT! Do NOT copy the last two lines you sent me. These will conflict with the .htaccess Maker rules, causing your site to stop responding with Internal Server Error 500.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

Thanks Nicholas

So I do that and save my htaccess file but the protection is not in place. I try with another browser too. May I do something else? May I need an exception for the .htpasswd file?

I thought of this because I got fooled with the Google validation file (Google Search Console). It was already in place but as I installed Admintoots afterwards, I did not realize that Google could no longer access this file and it was not indexing my pages of the site that I had just put online. By putting this exception it seems to work. Is there anything else for Google?

nicholas
Akeeba Staff
Manager

You definitely should NOT put an exception for the .htpasswd file. This file is never served over the web. It is read by the web browser, but it should never be served as a file to any random stranger. Adding an exception for it does not make any difference on whether your site is password-protected or not. It only controls whether any random person on the Internet can download the file containing the (plaintext) username and (weakly) hashed password, which would allow them to crack that very weak hashing, therefore view your site if they so wished.

The reason you experience it as the protection “not taking place” is that your browser remembers the login credentials for the site and sends them with your request. This is true even if you use the private browsing / incognito mode of your browser (yeah, I know, right?!!). You can use a different browser you don't normally use, e.g. Chromium, making sure you have cleared all stored data.

 I did not realize that Google could no longer access this file and it was not indexing my pages of the site that I had just put online. By putting this exception it seems to work. Is there anything else for Google?

The code you asked me about adding to the .htaccess file puts your entire site behind a username and password. Nobody can access anything on the site unless they provide the correct username and password combination. This concept is antithetical to indexing your site. You said so yourself when you described it as “I used a protection against robots indexation” [sic]. Yes, this code prevents any robot from indexing your site, including Google Bot which is used by all of Google's services.

So, the question is, was that your intention or not? You can't be hiding your site from everyone by using a username and password to access it, and expect it to be indexed at the same time. Do you want it hidden or not?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

Hello

I'm really sorry for the mix of issues, I realize it's confusing.

Problem 1: add password protection to the test site (only to avoid indexing by a search engine and therefore harmful duplication of content)
So I added this code as indicated here is what it gives on htaccess:
# +++CustHead+++
##### Custom Rules (Top of File)
###CW### B/Protection Indexation
AuthUserFile "/...(url masked)/.htpasswd"
AuthName "WEBSITE NAME MASKED"
AuthType Basic
require valid-user
###CW### E/Protection Indexation
# ---CustHead---
I have cleared the cache and cookies, closed Firefox and restarted, I do not see active protection. I also tested on Chrome and Safari.

Problem 2: the same site but be careful this time the one in production (it is no longer the test site that I want to protect by password), I had put a Google verification page before installing Admintools: indexing = OK. Since installing Admintools, indexing no longer takes place, I added an exception to the Google file which is on the root of the site. Is there anything else to do for Google services? I know this is another topic but since the Google file couldn't be read, I thought of this for my problem 1 and now understand it's not the same thing.

Thanks

 

nicholas
Akeeba Staff
Manager

1. This will work (tested), provided that a. your host supports it and b. the path to the .htpasswd file (AuthUserFile directive) is correct.

2. You need to add these files into the "Allow direct access to these files" feature in the .htaccess Maker configuration page.

Needless to say, in both cases you must click on the Save And Create .htaccess button for the changes to take effect.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

1. Yes my host supports it because I will worked before admintools with my custom htaccess. So to test I just create a custom htaccess only with the four lines for the protection and Wordpress lines (I delete all admintools rules) : it works. I put the complete htaccess with admintools rules : it doesn't work.

2. Thanks

nicholas
Akeeba Staff
Manager

I can tell you it definitely works. I tested on both Linux and Windows servers before replying back to you.

A few things that you may have missed, though.

The custom code needs to go into the "Custom .htaccess rules at the top of the file" section.

You must click on the Save & Create .htaccess button, NOT just the Save button. The former saves the configuration changes to the database and applies them to your site's .htaccess file. The latter only saves the configuration changes to the database, but does NOT apply them to the site's .htaccess file. Open your .htaccess file and make sure these lines were added. If not, go back to the .htaccess Maker and click on the Save & Create .htaccess button.

The AuthUserFile MUST have the full path to the .htpasswd file. If it doesn't, you'll see the login box in the browser but it will result in a 500 Internal Server Error page.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

I don't understand I do everything like you explain me. The lines are well in the htaccess file. Theses lines works fine with a blank htaccess file with just wordpress rules. But all the htaccess generates from the htaccess maker didnt't work. May I send you the htacess file in the attachments ? I am not sure to send you here because the ticket is public ?

nicholas
Akeeba Staff
Manager

Put the .htaccess file in a ZIP file and attach the ZIP file in your next reply. Attachments are always private, even on public tickets.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

Thanks please my file is enclosed.

nicholas
Akeeba Staff
Manager

Ah, now I understand the problem. You need to disable the "Block access from specific user agents" feature. This will not work together with password protecting the entire site. This feature allows all requests as long as the user agent does not contain any of the words or phrases in the block list.

It just did not occur to me that someone would be trying to use both features at once. To me, it was "obvious" that if you are password protecting your site you don't need or want to apply any other features limiting access to it since, well, you are already blocking access with a password.

Another way that should work (but I have not tested myself) is placing the password protection code in the "Custom .htaccess rules at the bottom of the file" area of the .htaccess Maker. Since that is evaluated last thing it should be applied to all requests which have not been blocked by any of the other rules.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

So the code placed in bottom = that doesn't work.

But the solution by disabling the "Block access from specific user agents" feature = works.

Thank you

nicholas
Akeeba Staff
Manager

You're welcome!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

Hello, sorry again me.

I have another development website : same webhost, same cms Joomla.

I do exactly the same settings with the htaccess maker for the 2 websites

For the second website when I clik on "save and create the htaccess" just after adding my code for the htaccess/password protection in the top section for custom rules, I have a 404 error page and my code is not saving.

I don't understand. If you see the problem ? Thanks

nicholas
Akeeba Staff
Manager

If you remove that code, does the site work? Or does it still say 404?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

It's just the htaccess maker page : when I add my code and click on the save button (for the 2 button options), the htaccess maker page is redirected to a 404 error page and the code is not saving and the htacess file not changed. But all the website works fine.

Without my code if I click on save and create the htacces, it works great.

nicholas
Akeeba Staff
Manager

To sum it up: If you do not add your custom code and create a .htaccess it works. If you add your custom code and create a .htaccess it crashes the site with a 404 (which is an error 500 in disguise; that's a different discussion). Assuming that I get your problem description right, the  root cause of the problem is the code that you added.

I hear you saying "But, why?! It's the same host!". Do keep in mind that every server, even on the same host, may have a slightly different configuration which makes the AddHandler / SetHandler line from one server not work on the other. For example, our business site is hosted on a managed virtual server on Rochen, whereas my blog is hosted on a standard hosting account on Rochen. Both sites are on Rochen, but the two servers are ever so slightly differently configured. I can't take the AddHandler line from the site hosted on one server and put it on the site hosted on the other server.

Please do read https://www.akeeba.com/documentation/admin-tools-joomla/custom-htaccess-rules.html. The idea is that 1. you change the PHP version using the hosting control panel 2. you copy the code it added to the .htaccess 3. you paste it in the .htaccess Maker and 4. you click on Save & Create .htaccess.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

It's not all the website in error 404, just this page after saving : https://test.xxx/administrator/index.php?option=com_admintools&view=Htaccessmaker

I have the php 8,2 version. May I need to verify if a php module must be activate (I can modify in cpanel) ?

nicholas
Akeeba Staff
Manager

Oh, now I get it! You get the 404 only when trying to save the .htaccess Maker configuration.

This is something you need to talk to your host about. Tell them that they have a mod_security2 rule in their Apache configuration which prevents you from saving a web form which includes your custom code in a text field. Send them your custom code so they know what it looks like. They can then find out which mod_security2 rule causes the problem and either disable it, or give you instructions to do so.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

houba_houbi

Hello

I checked with my host : he disabled mod-security and it works now.

I am surprised that on other installations: same version of Joomla/Admintools with the same host, I have mod-security activated, I do not encounter this problem (it is the same mod-security on all their hosting) . They answered me this:

"Modsecurity is a WAF with predefined attack type rules.
If one triggers security and the other doesn't, you must have a difference between the two that triggers security.
If you already have a WAF on your side, it can only conflict as well.
I don't have the information on my side of what causes this trigger unfortunately."

 

nicholas
Akeeba Staff
Manager

mod_security2, on its own, does nothing. Hosts need to install mod_security2 rules. The most widely used is OWASP ModSecurity Core Rule Set. However, since this includes rules for every version of everything under the sun, it ends up being unusable if used unmodified. THerefore, hosts need to curate the rules, and disable some.

I would neither enable everything, nor disable everything. The happy ground is somewhere in the middle. As a matter of fact, our site uses CloudFlare which also uses the OWASP core rule set, and I have indeed disabled the irrelevant sections and select rules which interfered with the operation of our site. As I said, the happy ground is somewhere in the middle :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!