". is there any kind of mod and/or txt file change that makes the AI produce better units, and pass on the weaker ones?"

WesW's entire MedMod is designed to do that. He's spent endless hours tweaking the column 15 AI build probabilities and the tech tree in build_prod13.txt to produce that exact result. Works pretty well. You should give it a try.