Another downside to releasing prisoners is that they run to the nearest town owned by their faction, no matter how far they have to walk. If that town was previous empty of a garrison, now it has one.

For example, sometimes you can attack an army standing outside a town, and the town's garrison will come to their aid. You can crush both in the field without a prolongued siege and stroll into the undefended town without a battle. If you were to release your prisoners from the field battle, they'd run into town, and you'd have to siege it.