If they're still remaining you haven't won yet; did you capture or destroy the big central building (CC)? Sometimes there's just a civilian standing where they last worked, sometimes all remaining units run somewhere to build a port.
newb here
I've built a huge army and stamped out basically the entirety of the bot and there's just bits of houses and stuff remaining. What do I do so I can finally win?
It should be. After all, did the ancients really do things as efficiently as possible? We are thousands of years more advanced, and we still don't do things as efficiently as we could. We know how to be efficient, we just choose not to. Besides, real merchants would travel from one market to the next, then on to another one, not just back and forth between two markets.
I would find that odd. Finishing a building already damaged doesn't make any sense. Furthermore I think it is good that the enemy can prevent you from building. Otherwise it is too easy to build a fortress or towers to defend yourself.