There are vices and virtues and retainers that affect your popularity with the people and the senate.

I think I've seen some that are -5% popularity to senate +10% to people. These usually get given to good generals iirc.

That could be your problem.