Ooooh... long and rambly. Better spoiler it and hope no one reads it.
Ah but you miss the later injection into the gene pool , the giants from on high . They came down and had lots of sex with humans because they like the food . But humans being humans liked the sex but didn't like the big fellas taking the food so they killed the giants .
But then of course the giants hadn't been killed and were evil and ungodly so when the really big fella got annoyed he killed them all too with his really big rainstorm(together with all the humans apart from a really amazingly good boat builder and his little family) which somewhat diminished the gene pool ever so slightly.
But then the gene pool expanded again because lo it is written there were more giants breeding with humans.
So what we really need to settle this debate is for a cretinist to dig up a fossilised giant.
It should be easy , after all they were all killed along with the dinosaurs in the flood so all the fossils should be together , well apart from those who were around after the flood but I think they must have been illegal immigrants so don't really have any standing despite being very tall.