This is a bug in Joomla which I have reported months ago but they said they won't fix. If an extension does not have a public update URL or Joomla cannot find it it will pretend that the extension is incompatible with Joomla 4. This is wrong. Both FOF 3 and FOF 4 are compatible with Joomla 4. They were the FIRST third party extensions to be compatible with Joomla 4 since Joomla 4 alpha 2 as a matter of fact.
As for the FOF extension, this is a very old version (2.x) we stopped developing in 2015 and became end of life in 2016. It is still shipped with Joomla 3 and is most definitely not compatible with Joomla 4 at all. It's also automatically removed on update.
So, don't worry, you can definitely update your test site to Joomla 4. Let me tell you that my live blog site (https://www.Dionysopoulos.me) is already using Joomla 4 and uses our software: Akeeba Backup, Admin Tools, Akeeba LoginGuard, Akeeba Engage, Akeeba SocialLogin. That's how confident I am that our Joomla 4 support is working: I put my own site where my mouth is ;)
Nicholas K. Dionysopoulos
Lead Developer and Director
🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!