Just because it can be used as a rascist symbol doesn't mean it is. The Swastika is religious in some cases, thats why it shouldn't be banned totally. The Hammer of Thor is religious to some people, but it's also a skinhead symbol. Almost anything can be interpeted as rascist, doesn't mean we should ban it.
Oh, and the 1st Confederate Flag was very union like, the 2nd looked like a white flag when it was in still winds, and the 3rd was the most liked cause it was the most recognizable (I have one in my room, on my wall right below a pic of jeff davis).