Another question on Retinue.

Is there a limit to the number of retainers a faction member can have? I've got an old faction leader with a pot full of retainers, so I moved some of the other generals over to his house and started dividing up the retinue. Most of them went over OK. Some who didn't were obviously duplicates, Priests especially.

But, a couple wouldn't transfer. A mathematician, a librarian and some other dude. If memory serves, these guys weren't named beyond their speciality. Are these generic retainers non transferrable?