Support

Hi,
i get a lot of Errors in my web logs:
2018-02-27 15:27:18 Error 216.244.66.227 403 GET /robots.txt HTTP/1.0 Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected]) 1.25 K Apache-Zugriff
2018-02-27 15:27:18 Error 216.244.66.227 client denied by server configuration: /var/www/vhosts/domain.de/root/cms/robots.txt Apache-Fehler

My Hoster says, it is possible from the Admin Tools generated htaccess
Is there a way to solved this prob?

WBR,
deltapapa

I woulldn't expect the enhanced .htaccess file to block the robots.txt file. Let's test it out. Please swap your current .htaccess file with htaccess.txt in the root of your site. This is the standard file that Joomla! ships, it is pretty plain. Does that solve the problem?

Hey,
after swap the admin tools htaccess to the default htaccess, the prob is solved. (See picture)
So, what can i do to use admin tools htaccess?
I have the issue on a couple of sites!
WBR,
deltapapa

I think I see the problem. It isn't the file itself that is being blocked, it is the request for the file. Please swap your .htaccess files back the way they were (I just want to save the clean Joomla! file as htaccess.txt). Then go to .htaccess Maker and set "Block access from specific user agents" to No. Then regenerate your .htaccess file and test. If that works, we need to figure out which specific user agent we want to allow and delete it from the blocked list. Turning the list off is just a test.

Hi,
jepp, after disable "Block access from specific user agents" no more errors with the robots.txt.
What can i do next?
Thank you and wbr,
deltapapa

We are not quite finished yet. We want to turn the "Block access from specific user agents" back on and delete "DotBot" from the list of agents to block. The name of the specific agent was in the original message but I was looking for something else and missed it at the time. That minimizes what we are allowing to access your site while allowing the site to function as it should.

Hi,
thank you. I did it, and now i will check the Logs to the next day!

Please let me know if you have any more problems.

This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.