Note that not all units produced in a city qualify for free upkeep. I believe units must be 'militia' to qualify, but I'm not sure. For example, broken lances cannot be 'kept up' for free. It is probably in some file.
If you have more than enough qualifying units, the game will conveniently choose the costliest for you.