Everytime after andys I get the above message instead of "would you like to save yes/no" thought maybe it was the controller plugin, but na. Tried with and without mem pak selected, didn't do anything. Only happens after andys house.

That screen always comes up after exiting a level if you've collected any tokens/new potato head items. It doesn't just happen on Andy's House, it happens after every level. Nothing you can do about it, the screen isn't optional. You don't really 'lose time' because everyone has to view that screen, just watch literally any other N64 run to confirm this.