Hi, TASer here, was bored so I decided to say my piece. It is possible for humans to reach PU speed in the castle grounds, unfortunately, that will take a couple of hours. Secondly, it is too, too precise. One tiny angle difference in a PU is a MASSIVE angle change. It will never be humanly viable. It is possible to go to a PU and back to the main map, by simply turning mario 180 degrees but thats as good as you will get.
I could go on and on for an embarrassingly long time, but I will really only contribute to this thread by saying this: Technical questions etc you should really just bring to the SM64 TAS discord server. There's a channel there where you can ask questions, and often people are online and someone can answer. Asking this there would have changed a 2 page thread into a 2 message conversation.