While usually tribal societies are indeed not as capable to field as large forces as more organized ones, we are talking about armies of a few thousand men at max in the game, which a dominant tribal confederacy (which you are playing) could realistically field without much trouble. Seeing that if you use a smaller unit size, civilized societies will also field smaller armies, I would say that if you want to reduce numbers, in my opinion the more accurate way would be using less unit slots.