TL;DR It does not matter why a request is blocked, it participates to automatic IP blocking all the same. When your site gets many requests at the same time the consistency model of the database (BASE instead of ACID, if you want to be technical) means that it may take dozen to hundreds of requests before an IP-level block is established. When the IP-level block takes place there are no longer any log entries in Admin Tools to show you these requests are blocked. You can still see them being blocked in your web server's access log, therefore you can see that Admin Tools DOES work.
Your analysis was wrong. You have not read the documentation and you are jumping to false conclusions.
Regardless of what the blocking reason is, the request is blocked. That's why it's called a blocking reason.
Moreover, regardless of what the blocking reason is, the fact that the request was blocked (including the target URL, blocking reason, and IP address) is recorded into the Blocked Exceptions Log unless you have explicitly told Admin Tools not to do that in Web Application Firewall, Configure WAF, Logging & Reporting, “Do not log these reasons”.
Every time there is a blocked request, Admin Tools evaluates the auto-blocking rules set up in Web Application Firewall, Configure WAF, Auto-ban. It will automatically block an IP address for a limited amount of time based on the “IP blocking of repeat offenders” settings and the contents of the Blocked Requests Log. If the same IP address is receiving temporary bans and you have set up “Add persistent offenders to the IP Disallow List” then Admin Tools will permanently block it.
There are two critical things you need to understand about how a request block turns into a temporary IP-level block, and eventually a permanent IP block.
The evaluation of the rules to block an IP address does not happen until there is a blocked request. Since you have a lot of requests coming in at about the same time, they are queued for processing by your web server, and are processed in parallel according to the number of PHP threads available on your server. When each request is processed, Admin Tools loads the list of temporarily and permanently blocked IP addresses. There is a small lag between an IP block record being written to the database and the database being able to return it when we ask it to list the contents of that table. This comes down to MySQL's consistency model which favours performance over consistency (all RBDMS work like that; consistency-first approach is only common across key-value storage databases like Redis and memcached which is why they hold the data in memory, so as not to obliterate performance due to the orders of magnitude slower permanent storage I/O).
The evaluation of IP blocking reasons means that Admin Tools asks the database to return the number of blocked requests from this IP address within a specific time period. Remember the database is being accessed (read to and written from) several threads handling different requests at the same time. As I explained, this means that for a very short period of time after a write, a thread reading from the database will see stale data i.e. what the data looked like before the write operation. Therefore, a number of requests are individually blocked without being able to “see” that there were a few dozen or a few hundred other requests being blocked at the same time in other threads.
The combination of these two factors means that if there are “too many” requests at the same time they will be blocked individually, but the IP block won't be triggered until some of the last requests are being processed, when the database finally returns the entire list of blocked requests, at which point we write another record in the temporary IP block table. Then a few more requests are individually blocked until the database finally returns the new temporary IP block record to Admin Tools, at which point any further requests are blocked without being logged.
If you check your access logs you will see that following the small barrage of requests blocked individually by Admin Tools and logged as blocked requests there's a continued deluge of requests being blocked by Admin Tools (at the IP address level) without being logged. Not logging requests blocked at the IP address level is a deliberate feature, implemented as a default value for the “Add persistent offenders to the IP Disallow List” option, to prevent an attacker from overwhelming your site's database by sending requests from a blocked IP address.
Technically, it is possible to have MySQL operate in a more-or-less consistency-first mode, by locking tables for reading and writing. The problem is that this approach is detrimental to performance. The web server can no longer process requests in parallel as the table lock means the database is waiting to be told it has the go-ahead to read from or write to the Admin Tools tables. When you have someone sending a few thousand requests within a few seconds this would mean that instead of processing them (and blocking individually) these requests within a couple of seconds we'd need several minutes to process each one of them individually, and arrive at an IP-level block sooner. In both cases, all requests are blocked. In the consistency-first mode, though, your web server's CPU usage spikes, and your site is unavailable for legitimate visitors for several minutes. That's why we did not choose to do this. Using the less consistent mode of operation we are STILL blocking all requests, and your server is unavailable for legitimate visitors for a few milliseconds to a couple of seconds.
Nicholas K. Dionysopoulos
Lead Developer and Director
🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!