It means the attack values increase when the unit attacked is armoured, giving a bonus against armoured troops. If it's applied to missiles, then the missile will have greater effectiveness against armour in similar vein. Hence vanilla archers won't do much damage to say halbardiers, but longbows will. Crossbows and arbalests are also armour-piercing.

For melee units it's generally axes and polearms that have AP bonuses.

There is a more detailed breakdown of the actual formulas in Frogbeastegg's wonderful guide (follow links from the MTW Guides section), but generally the more the armour, the better the AP bonus!