I've just done, what I should have done before - downloaded my own mod - and indeed it is correct that it's positive values that increase rebelliousness. I had all of the provinces set to +2 and some at +3 and +4.

So I was correct in assuming that increasing the rebellious factor causes the AI to stay put and not invade. Garnier's results also seem to confirm this.

It's probably best left at 0 as the AI does seem to struggle to deal with anything higher.