But academy and its upgrades give retinues only.

Also, exemot ordinary retinues you can get famous people which are unique (only one per campaign).

Benefits of upgraded buildings is a bit limited though.
Scriptorium add greatly to poor of famous people, but just several retinue members.

While Ludus Magnus is almost useless.
Total two of them, including one famous person.




P.S.
CVP and bug-fixer do give some chance to increase some of the traits.