Support

Admin Tools

#40613 Bad Bots

Posted in ‘Admin Tools for Joomla! 4 & 5’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

Joomla! version
4.4.x
PHP version
8.5.8
Admin Tools version
7.4.3

Latest post by nicholas on Saturday, 20 April 2024 09:20 CDT

Friday, 19 April 2024 18:05 CDT

myaviation

Today I had some server issues and I contacted my support - and they said that I have lots of issues wtih this BOT ClaudeBot

They told me to put this in my .htaccess file

Does the admin tools do this for us for the BOTS - I looked at your documents and other tickets but they where all very old

 

Thanks

 

#BEGIN IMH Block Bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(seznambot|semrushbot|sogou|ahrefs|mj12bot|petalbot|opensiteexplorer|blexbot|bytedance|amazonbot|applebot|ClaudeBot).*$ [NC]
RewriteRule .* - [F,L]
#END IMH Block Bots

Saturday, 20 April 2024 09:20 CDT

nicholas

Do not put this code into your .htaccess file.

Instead, go to the .htaccess Maker feature and set "Block access from specific user agents" to Yes. 

You will now see the "User agents to block area". You can now add different (partial) user agents there.

DANGER! Your host gave you potentially bad advice about what to block. Here's what you block with each string

  • seznambot blocks a Polish search engine from indexing your site.
  • semrushbot blocks some SEO tools.
  • sogou block's Tencent's search engine from indexing your site.
  • ahrefs blocks some SEO tools.
  • mj12bot blocks a search engine indexing backlinks.
  • petalbot blocks Huawei's Assistant and search services, used by popular Huawei phones.
  • opensiteexplorer blocks another SEO tool.
  • blexbot blocks a marketing tool.
  • bytedance blocks TikTok previews of your site.
  • amazonbot prevents Amazon from indexing your site, making it invisible to Alexa.
  • applebot prevents Apple's iMessage -- the most popular messaging app in North America -- from rendering previews of your site. It also appears to block indexing for Apple Search which does not seem to be publicly available yet. Basically, it will make your site invisible to all Apple iOS / iPadOS / tvOS / macOS devices.
  • ClaudeBot blocks an IRC bot, NOT Claude AI. I believe your host mixed up the two.

Out of all these, I would maybe add the three SEO ones and blexbot. Everything else is more likely to hurt you more than help you.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!