It means you have to have a governor in town to build or recruit.

I think it's a good rule to slow you down. But you can still move your guys around and make things happen. Or bribe enemy generals to get more governors. Just another strategy element.

At least it will give people something to do in the beginning. After all you will need to ships to do that. And you will be taking a risk of being killed by pirates.