Maybe create a transport from the dock and transport like 3 trebuchets over there and destroy both of the castles at the same time would be faster, since you don't need to deal with the army that way.