And when a white person calls his cheese by the name of the man whose name is the same as one of these words, well, I think the answer is pretty obvious.
ETA: In all seriousness, both you and rockinkt could be right. The words, by themselves, really do hold no power, but they are used in ways that can project power and result in real harm.
In my opinion, when you go after the words, all by themselves, it's a distraction from the real issue, and it alienates people who might otherwise be allies. Demanding that someone change the name of a kind of cheese is a good way to get people to not take you seriously when you also demand that the actual racial slur not be used as a racial slur.