You are not really wasting your time as you already have the connection when you expand into their area, thus getting the trade from early on. Also you gain a valuable insight into the other factions' standings, are they winning are they losing, are they strong/weak and so on. I urge you to also get their maps when you get the traderights, thus you can see their extent well, getting a feel on which faction will win the coming wars.

Reply