In that case, you always attack the unit closest to your calvary. Often they have a line of crossbowmen, that's what you attack first. If you attack the infantry behind the crossbowmen, your charge will be broken up if a couple of retreating crossbowmen get in the way. A good charge into a line of non-spear infantry should result in 70-100% casualties.

For maximum effect, surround the enemy, and then charge. That way more units are charging rather than standing around at the back of your army doing nothing.