I don't know if this is entirely useful for most speedruns but if you're using wired controllers and undock then dock the switch all the wired controlled characters disconnect instantly, it won't take any time if you do it while on the scoreboard or taking the flag. It's faster than going into the menu especially if you need to remove 3 players after wanting to get the flagpole with them all.

If you want to disconnect multiple players the fastest way I've found is using the gamecube adapter and unplugging the usb, letting you disconnect all 3 players without losing video signal by removing the switch from the dock.