Got lucky to have not going to the Joust which takes up quite some time (also I suck at this minigame). I was able to build a huge enough army to conquer all the other looser kingdoms.