Well, the history of mankind has never been a pretty one. Explusion, enslavement, genocide, conquest are large parts of our past. Whether we want this in our game is another question.

Another option which would be just as unpopular is enslavement not to speed up population growth but as a production bonus to areas receiving its benefits. Of course in our more civilized age we abhor this kind of abuse but it certainly was a reality of our past. Then again you could argue that every serf from that time period was just as easily a slave.