I have the Xbox One version, as well as the 360 version, but my 360 copy is super slow. Is it possible to downgrade past the Horse tutorial update on Xbox One?

no, the first version of the xbox one edition had potions tutorial, you can only ever play on versions older than potion tutorial if you have a xbox 360

My bad, the horse tutorial was the first one I played on Xbox One. Anyways, how can I downgrade the Xbox One version, and should I make backups?

i don't remember as to how you can downgrade but this vid might be useful, and if you have any important worlds its better to back them up.