Well once you noticed how the costs work, you should have slowed down and upgraded everything before expanding. That's the quirk of this game, you want to upgrade everything before you build next copy..
I agree with the military point completely, 1 barrack, upgraded to 3 lasts almost entire game, then maybe a one or two STR upgrades. And to compare, one of the "epic" relics adds.... 1STR. Not that great...