You can edit the EDB. You can decrease the pool (number 3) and the replenishment rate (number 0.4) of a unit in the recruitment pool of a building. See for example the recruitment pool of Christian guards in this code
Code:
recruit_pool "Christian Guard"  1   0.4   3  0  requires factions { moors, }
PS I didn't have the EDB from crusades since that is the only campaign I haven't unpacked.