Actually, let me rephrase that last sentence. The ability to work without Fabric API is just one of the reasons the aforementioned mods are allowed, there are plenty of other reasons too.

The fabric loader is allowed. In fact, a good majority of speedrunners use it to run minecraft with a several mods like CaffeineMC mods, LazyDFU, atum, ect. Those mods do not require the Fabric API and are not banned because of that.