The start is quite RNG as the red won't move the axeman around very often, but it's quite awkward to prevent them from going to the yellow territory so later it's not necessary to search for them. Hard to find those villagers as they keep gathering resources, sometimes they will come out when you started attacking their buildings and they don't have any military, but this time most of them didn't show up.