Then you are aware that the Sahara used to be a rain forest and have been drying up since pretty much forever. Maybe a asteroid landing in it helped it out but as far as long trends go, its been going on for a pretty long chunk of human history. That being said, the expansion of the Sahara waxes and wanes from time to time and maybe climate change has something to do with it. As for agricultural production in those areas at that particular time, I'm not aware that there were food production issues.

Well not including political things like the conquest of Carthage.