The shield seems to be working as intended against missile fire, as is armor. I ran lots of peasant bowmen target tests when researching the armor upgrade problem and units with shields were consistently tougher. Spear militia and pike militia both have 1 skill and no armor, yet the spears take 30% less casualties on average from normal missile fire due to their 6 points of shield. If they have armor in addition to a shield (like Italian spear militia with 4 points of padded) then it can cut their losses by 50%.

Melee is a very tricky thing to test. Often one side gets hopelessly muddled and doesn't even strike back when charged, giving the attackers a 'free' strike at the enemy, regardless of whether you set the charge bonus high or low. This tends to mean that combat tests are inaccurate as the side who didn't bug and charged successfully first inflicts disproportionate casualties. I've run tests where vanilla JHI massacred everything and tests where they got stomped badly. Unless a dev comments on this, I am reluctant to rely on combat tests to prove anything major.