Simple apathy towards any religion is fine and not a religion, but I would rather call that agnosticism.
Atheists have a belief that no supreme being exists, so i am more critical of calling that a lack of religion. When they start preaching Athiesm then the preachers, then it has adopted religious trappings and I would certainly not refrain from calling it a religion.