I don't think it's quite so easy to say that.
What should our whole attitude be toward the Civil War? Should we talk about it as a great crusade where the Good Guys overcame evil? At the battlefields, should we rewrite the presentations so that they aren't so neutral? Edit the films to show the triumph of freedom against the forces of oppression and slavery?
There would be a lot of truth to that version of events, I am sure you will admit.
Should we not only lower the flag at that South Carolina monument to fallen Confederate soldiers, but also destroy the monument itself? After all, they were fighting in a cause that was, frankly, evil, were they not? Why should we memorialize them at all? Isn't it time those monuments came down?
I don't think so. They're part of history. The men who fought for the Confederacy fought for slavery, sort of, but for the most part they fought because it was their team. It was their home town. In the case of my grandmother's grandfather, it was because his father informed him that he and his brothers would be disinherited unless they joined the Confederate army. (And they had a financial stake in it directly as well, since some of their "property" consisted of people.)
At what point does it stop? How far should we go in purging those symbols that we associate with racism or slavery? That's a serious question, by the way. Is it just that flag, or should it include the monuments? I was somewhat serious about the question of music as well. Should a government band ever play "The Bonnie Blue Flag"? I know that no one is getting irate about music like that, but, well, where do you draw the line?
I think there's something very odd about flying a rebel flag from a government building, so it seems to make sense to me to take it down from an official display, but it's hard for me to get worked up over a coffee cup with a confederate battle flag on it. I feel more nervous about efforts to purge unpopular symbols than I feel about the symbols themselves.