Yep, it's all about quantity. Units that are larger (120 vs 80 members) give better total effect. If you can consider an area completely safe from attack, you could garrison with nothing but peasants, but because of rebel threats, I almost always have 3 or 4 units that are combat able, like a single cav, 2 archers and a respectable infantry unit. Else, you might have a long trip to get troops to destroy short rebel stacks.