The problem with Lee is that, really, he is morally equivalent with Thomas Jefferson - both personally, morally, objected to slavery and yet both wholeheartedly participated in it and refused to free their slaves. Jefferson didn't even free his coloured children.

So - if Lee must go, so must Jefferson, doubly so because as one of the Founding Fathers he literally institutionalised slavery.