Is all castles just the castles at the end of each world or is it the castles as well as the towers?

no towers

You just need to beat all the castles at the end of each world, though for the optimal all castles route you need to beat every tower except 5-T anyways.

nerd

imagine having the NSMBU Any% Normal world record

having